You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe it
Currently, Jan only offers four predefined OpenAI models, which may not reflect the full range of models available to users with different API key access levels. This limitation prevents users from accessing their full range of accessible models, including older, more powerful models.
Describe the solution
I would recommend implementing a dynamic model discovery feature that runs every time the user edits their OpenAI API key that queries the OpenAI API to retrieve all available models for the new key.
Maybe you update the AIEngine class (in core/src/browser/extensions/engines/AIEngine.ts:35-59) to incorporate this dynamic model discovery
Definitely add a new field to the model object in models.json to track whether a model is enabled (extensions/inference-openai-extension/resources/models.json): {"enabled": true}
To account for the possible drawback:
Update the UI to include a toggle for each model in the OpenAI settings, allowing users to enable/disable models (optional, to account for the drawback mentioned in issue 3115).
Modify the model loading process (core/src/node/api/restful/helper/builder.ts:300-316) to filter out disabled models.
Potential Drawbacks
Handling the case of no internet when the API key is changed to fail gracefully
Managing updates when OpenAI introduces or deprecates models (actually, this helps with the case of adding new models)
If we don't include the configurable model options, this could add more clutter by adding more API "models" to the list
Enable the Feature:
Navigate to the settings page.
Enable/disable the "Dynamic Model Discovery" option (should be enabled by default).
Configure Models:
Go to Settings ⌘, > OpenAI > "Model Configuration" section (this is a new section that needs to be added)
Toggle the models you want to enable or disable.
What is the motivation / use case for changing the behavior?
Use gpt-4-0314, currently the smartest, least restricted model available, to answer questions on a local machine not powerful enough to run high-end models locally.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe it
Currently, Jan only offers four predefined OpenAI models, which may not reflect the full range of models available to users with different API key access levels. This limitation prevents users from accessing their full range of accessible models, including older, more powerful models.
Describe the solution
I would recommend implementing a dynamic model discovery feature that runs every time the user edits their OpenAI API key that queries the OpenAI API to retrieve all available models for the new key.
Maybe you update the
AIEngine
class (incore/src/browser/extensions/engines/AIEngine.ts:35-59
) to incorporate this dynamic model discoveryDefinitely add a new field to the model object in
models.json
to track whether a model is enabled (extensions/inference-openai-extension/resources/models.json):{"enabled": true}
To account for the possible drawback:
Update the UI to include a toggle for each model in the OpenAI settings, allowing users to enable/disable models (optional, to account for the drawback mentioned in issue 3115).
Modify the model loading process (
core/src/node/api/restful/helper/builder.ts:300-316
) to filter out disabled models.Potential Drawbacks
Teachability, documentation, adoption, migration strategy
Teachability: The users won't even notice.
For docs:
Enable the Feature:
Navigate to the settings page.
Enable/disable the "Dynamic Model Discovery" option (should be enabled by default).
Configure Models:
Go to Settings
⌘,
> OpenAI > "Model Configuration" section (this is a new section that needs to be added)Toggle the models you want to enable or disable.
What is the motivation / use case for changing the behavior?
Use
gpt-4-0314
, currently the smartest, least restricted model available, to answer questions on a local machine not powerful enough to run high-end models locally.The text was updated successfully, but these errors were encountered: