You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Jan Server does not include a tool call id for called tools.
This causes an issue with using langchain/langgraph which requires a tool call id.
Included is a python script that tests OpenAI and Jan server tool calls (using qwen2.5-coder-7b-instruct model for Jan)
Expected: tool call id for tool calls.
Steps to Reproduce
Install Jan
Download Qwen2.5 Coder 7B Instruct Q4 from the Hub
Activate Qwen2.5 Coder 7B Instruct Q4
Start Local API Server
Note: It says it's an OpenAI compatible local HTTP server
Install python
pip install openai to install library dependencies
Set your OPENAI_API_KEY environment variable to your OpenAI Api Key
Add code to test.py
fromopenaiimportOpenAIdefmain():
jan=OpenAI(base_url="http://localhost:1337/v1")
oai=OpenAI()
TEST(oai, "gpt-4o-mini")
TEST(jan, "qwen2.5-coder-7b-instruct")
defTEST(client, m):
try:
history= []
history.append({"role": "system", "content": "You are a helpful assistant."})
history.append({"role": "user", "content": "Add 1 and 2"})
response=client.chat.completions.create(
model=m,
messages=history,
tools=[
schema_add(),
],
tool_choice="auto"
)
fortoolinresponse.choices[0].message.tool_calls:
iftool.idisNone:
raiseException(f"{m}: tool call id is None for {tool.function.name}")
print(f"Pass: {m}")
exceptExceptionase:
print(f"Fail: {e}")
# A function that we can use as a tooldefadd(a: float, b: float) ->float:
print(f"{a} + {b} = {a+b}")
returna+b# A schema that lets the LLM know that we can use the add function as a tooldefschema_add() ->dict:
return {
"type": "function",
"function": {
"name": "add",
"description": "Add two numbers",
"parameters": {
"type": "object",
"properties": {
"a": {"type": "number"},
"b": {"type": "number"}
},
"required": ["a", "b"]
},
},
}
if__name__=="__main__":
main()
python test.py
Screenshots / Logs
Pass: gpt-4o-mini
Fail: qwen2.5-coder-7b-instruct: tool call id is None for add
What is your OS?
MacOS
Windows
Linux
The text was updated successfully, but these errors were encountered:
Jan version
0.5.13
Describe the Bug
Jan Server does not include a tool call id for called tools.
This causes an issue with using langchain/langgraph which requires a tool call id.
Included is a python script that tests OpenAI and Jan server tool calls (using qwen2.5-coder-7b-instruct model for Jan)
Expected: tool call id for tool calls.
Steps to Reproduce
pip install openai
to install library dependenciestest.py
python test.py
Screenshots / Logs
What is your OS?
The text was updated successfully, but these errors were encountered: