Support multiple openai versions in python #27
Conversation
| @@ -18,7 +21,7 @@ jobs: | |||
| python-version: "3.11" | |||
| - name: Install dependencies | |||
| run: | | |||
| python -m pip install --upgrade pip setuptools build twine | |||
| python -m pip install --upgrade pip setuptools build twine openai | |||
There was a problem hiding this comment.
Should we do something where we add a version number to the matrix and pin that version number here?
There was a problem hiding this comment.
No i think we want the test to fail if openai releases another breaking change :)
There was a problem hiding this comment.
Sure I just think we could also make sure it's tested across all the versions we do support. Perhaps there's a way to encode "no version" as one of the options?
py/autoevals/util.py
Outdated
|
|
||
| openai_obj = openai | ||
| is_v1 = False | ||
| if hasattr(openai, "chat") and hasattr(openai.chat, "completions"): |
There was a problem hiding this comment.
Are we expecting that someone has already imported openai and set openai.api_key before getting here? Because if I just import openai from scratch and call `hasattr(openai.chat, "completions"), I see the following error:
91 api_key = os.environ.get("OPENAI_API_KEY")
92 if api_key is None:
---> 93 raise OpenAIError(
94 "The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable"
95 )
96 self.api_key = api_key
98 if organization is None:
OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
There was a problem hiding this comment.
Yeah weird, I changed it to be hasattr(openai, "OpenAI").
This change updates autoevals to support both the v0 and v1 versions of OpenAI's python library. To accomplish this, we:
openaias a dependency. This enables autoevals to be included flexibly as a dependency in systems which use either version of openai.braintrustsdk instead of a custom one (except for cached requests...).Tested using end-to-end scripts with both versions of python.