Abstract from API idiosyncrasies

[palako]

Instructor does a beautiful job of abstracting model API specifics, and allows to use OpenAI or Anthropic with an almost identical setup after the client/create has been constructed.
However, the APIs sometimes behave differently in ways that things break. For example, you can have a set of messages working for OpenAI where you alternate the roles of user and assistant, and nothing prevents you from sending consecutive messages from the user (for example you can conditionally add messages to the conversation). However, Claude doesn't like this, and will fail with an error complaining that messages between user and assistant must alternate.

We can of course refactor our code to concatenate messages from "user" which will then work for both models, but I open for discussion if Instructor would be the right place for abstracting the client code from these kinds of specificities.

This is specially relevant when testing the performance of several models with the same code, or when dispatching the same request in parallel to several models to then compare responses, where ideally the code written to generate the prompts is reused. I would not like to have to move all over my code "if Claude" type of statements.

Thoughts?

[palako]

The same applies to some data that would be lovely if the API would be model independent, for example, to use or simply log the token usage, I understand that the only way is to look at the _raw_response. For OpenAI there's _raw_response.usage.prompt_tokens and _raw_response.usage.completion_tokens while for anthropic there's _raw_response.usage.input_tokens and _raw_response.usage_output_tokens.

[jxnl]

we've updated this to handle consequeitves

[palako]

Do you mean it has been taken care of already? In the 1.0.1 you just published? That's great!

[Mr-Ruben]

I guess @jxnl is talking about
https://python.useinstructor.com/#returning-the-original-completion-create_with_completion

[jxnl]

could heve room for some

create_with_usage