multi-shot prompting

[palako]

Is there documentation or examples on how to use multi-shot prompting with instructor? Since the pydantic classes abstract from the actual message passed to the LLM, how do we craft examples as part of the multi-shot conversation in the prompt?

It would be sweet to be able to use example values to manually instantiate objects of the same type that are going to be used as response_model and then just add them pass them to the chat/completion methods, and have Instructor take care of the translation into what the model expects.

[Mr-Ruben]

Is there documentation or examples on how to use multi-shot prompting with instructor?

I haven't seen.

Since the pydantic classes abstract from the actual message passed to the LLM, how do we craft examples as part of the multi-shot conversation in the prompt?

Simple:

1. Do the multi-shot just as if Instructor were never there (example below).
2. Once you have that working, do the same call but providing something to the response_model parameter and Instructor will kick in.

Copied from https://platform.openai.com/docs/guides/text-generation/chat-completions-api

response = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "<Include here your multi-shot examples>\nQ: Who won the world series in 2020?"},
    {"role": "assistant", "content": "A: The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Q: Where was it played?"}
  ]
)

PS: If you show what you have tried and the results... you may get better help.

[Mr-Ruben]

Here
(old) #653
(new) https://python.useinstructor.com/examples/examples

But you you can just add the examples on the prompt, together with the instructions and data. Except for the response_model, everything is the same as if Instructor were never there.

In fact, I have a function where I do this:

call1: prompt -> LLM  -> text_response  
call2: prompt (which is only the text_response)  -> LLM (with response_model) -> response_model_Object

I do this so that the 1st call has 'less work to do' when responding.
I know the retry may not work well as the original prompt is lost, but it works fine for me.

[pablospe]

What do you mean by "call2: prompt (which is only the text_response)"? Do you mean that new prompt is the "original prompt + text_response"? Or do you add it as the reply like this:

    {"role": "user", "content": prompt},
    {"role": "assistant", "content": text_response},

?

Wouldn't the text_response be obtained from the raw response (here)?

[Mr-Ruben]

Sorry If I mislead you. What you are asking about has nothing to do with 'providing examples' (links above) but rather with a technique I use to get (in my opinion) 'better answers' when the response format is simple.

What do you mean by "call2: prompt (which is only the text_response)"?

Exactly what it says. The prompt is ONLY the text_response. That is, not an object processed by Instructor.

Wouldn't the text_response be obtained from the raw response (here)?

No if you want to avoid the LLM to get distracted by the response_model.
It is just a call without Instructor, so the output is just plain text.

Then you take that text_output and send it on a call with Instructor. Example here

# Define your desired output structure
class UserInfo(BaseModel):
    name: str
    age: int


# Patch the OpenAI client
client = instructor.from_openai(OpenAI())

# Extract structured data from natural language
user_info = client.chat.completions.create(
    model="gpt-3.5-turbo",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)
#> John Doe
print(user_info.age)
#> 30

"John Doe is 30 years old." is here the text_output.

More on that trick I use -> #956

---

Now, going back to the examples on the prompt.

Basically & briefly:

• Task
• Examples
• Input
• Output format
  (you may also use system prompt, but that goes in other message)

[pablospe]

Thanks for the clarifications! I got confused with this comment: #497 (reply in thread)

Where you were suggesting:

# Pseudocode
response1= call(prompt=p1)
response2= call(prompt= response1 + p2 )
...
responseN= call(prompt= responseN-1  + pN  , response_model=Output )

[pablospe]

In fact, I have a function where I do this:

call1: prompt -> LLM  -> text_response  
call2: prompt (which is only the text_response)  -> LLM (with response_model) -> response_model_Object

Can this approach be used for generating a jsonl dataset (for fine-tuning)? Since with this approach, one could get the system/user/assistant as unstructured messages. See discussion: #1026

Any other ideas?

[thomasnormal]

I wrote a small library here using Instructor: https://github.com/thomasnormal/fewshot
This is similar to DSPy in that it will optimize the few-shot examples for you.