Skip to content

Commit

Permalink
docs(patching): introduce new patching methods and modes, update docu…
Browse files Browse the repository at this point in the history
…mentation (#431)
  • Loading branch information
jxnl authored Feb 12, 2024
1 parent 2cabb76 commit 56e69d4
Showing 1 changed file with 42 additions and 33 deletions.
75 changes: 42 additions & 33 deletions docs/concepts/patching.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,49 @@
# Patching
# Patching the OpenAI Client

Instructor enhances client functionality with three new keywords for backwards compatibility. This allows use of the enhanced client as usual, with structured output benefits.

- `response_model`: Defines the response type for `chat.completions.create`.
- `max_retries`: Determines retry attempts for failed `chat.completions.create` validations.
- `validation_context`: Provides extra context to the validation process.

There are three methods for structured output:
There are two ways to patch the OpenAI client: patching the client itself, or patching a specific function. The former is more general, while the latter is more specific.

Then there are a handful of modes to choose from:

1. **Function Calling**: The primary method. Use this for stability and testing.
2. **Tool Calling**: Useful in specific scenarios; lacks the reasking feature of OpenAI's tool calling API.
3. **JSON Mode**: Offers closer adherence to JSON but with more potential validation errors. Suitable for specific non-function calling clients.
4. **Markdown JSON Mode**: Experimental, not recommended.
5. **JSON Schema Mode**: Only available with the [Together](../blog/posts/together.md) and [Anyscale](../blog/posts/anyscale.md) patches.

## Patching Modes

We support two kinds of patches One that patches anything isomorphic to the OpenAI client and the other to a specific create call

### Patch Client

```python
import instructor
from openai import OpenAI

client = instructor.patch(OpenAI())
```

### Patch Create

## Function Calling
This allows you to patch any function that is isomorphic to `chat.completions.create`. function with messages and other parameters.

```python
import instructor
from openai import OpenAI

create_fn = OpenAI().chat.completions.create
create = instructor.patch(create=create_fn)
```

## Patching Modes

### Function Calling

```python
import instructor
Expand All @@ -21,7 +52,7 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.FUNCTIONS)
```

## Tool Calling
### Tool Calling

```python
import instructor
Expand All @@ -30,7 +61,7 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.TOOLS)
```

## JSON Mode
### JSON Mode

```python
import instructor
Expand All @@ -40,7 +71,7 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=Mode.JSON)
```

## Markdown JSON Mode
### Markdown JSON Mode

!!! warning "Experimental"

Expand All @@ -53,37 +84,15 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.MD_JSON)
```

### Schema Integration
### JSON Schema Mode

In JSON Mode, the schema is part of the system message:
This is only available with the [Together](../blog/posts/together.md) and [Anyscale](../blog/posts/anyscale.md) patches.

```python
import instructor
from openai import OpenAI

client = instructor.patch(OpenAI())


class UserExtract(instructor.OpenAISchema):
name: str
age: int


response = client.chat.completions.create(
model="gpt-3.5-turbo-1106",
response_format={"type": "json_object"},
messages=[
{
"role": "system",
"content": f"Match your response to this json_schema: \n{UserExtract.model_json_schema()['properties']}",
},
{
"role": "user",
"content": "Extract jason is 25 years old",
},
],
)
user = UserExtract.from_response(response, mode=instructor.Mode.JSON)
print(user)
#> name='Jason' age=25
client = instructor.patch(OpenAI(
...
), mode=instructor.Mode.JSON_SCHEMA)
```

0 comments on commit 56e69d4

Please sign in to comment.