Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add older/more OpenAI models #3392

Closed
1 task done
Tracked by #3374 ...
msmolkin opened this issue Aug 16, 2024 · 1 comment
Closed
1 task done
Tracked by #3374 ...

feat: add older/more OpenAI models #3392

msmolkin opened this issue Aug 16, 2024 · 1 comment
Labels

Comments

@msmolkin
Copy link

  • I have searched the existing issues

Is your feature request related to a problem? Please describe it

Currently, Jan only offers four predefined OpenAI models, which may not reflect the full range of models available to users with different API key access levels. This limitation prevents users from accessing their full range of accessible models, including older, more powerful models.

Describe the solution

I would recommend implementing a dynamic model discovery feature that runs every time the user edits their OpenAI API key that queries the OpenAI API to retrieve all available models for the new key.
Maybe you update the AIEngine class (in core/src/browser/extensions/engines/AIEngine.ts:35-59) to incorporate this dynamic model discovery
Definitely add a new field to the model object in models.json to track whether a model is enabled (extensions/inference-openai-extension/resources/models.json): {"enabled": true}

To account for the possible drawback:
Update the UI to include a toggle for each model in the OpenAI settings, allowing users to enable/disable models (optional, to account for the drawback mentioned in issue 3115).
Modify the model loading process (core/src/node/api/restful/helper/builder.ts:300-316) to filter out disabled models.

Potential Drawbacks

  • Handling the case of no internet when the API key is changed to fail gracefully
  • Managing updates when OpenAI introduces or deprecates models (actually, this helps with the case of adding new models)
  • If we don't include the configurable model options, this could add more clutter by adding more API "models" to the list

Teachability, documentation, adoption, migration strategy

Teachability: The users won't even notice.

For docs:

Enable the Feature:
Navigate to the settings page.
Enable/disable the "Dynamic Model Discovery" option (should be enabled by default).
Configure Models:
Go to Settings ⌘, > OpenAI > "Model Configuration" section (this is a new section that needs to be added)
Toggle the models you want to enable or disable.

What is the motivation / use case for changing the behavior?

Use gpt-4-0314, currently the smartest, least restricted model available, to answer questions on a local machine not powerful enough to run high-end models locally.

@msmolkin msmolkin added the type: feature request A new feature label Aug 16, 2024
@msmolkin msmolkin changed the title feat: [DESCRIPTION] feat: add older/more OpenAI models Aug 16, 2024
@imtuyethan imtuyethan moved this to Planned in Jan & Cortex Aug 19, 2024
@imtuyethan imtuyethan moved this from Scheduled to Planning in Jan & Cortex Aug 30, 2024
@freelerobot
Copy link
Contributor

Closing as dupe of #3505

@github-project-automation github-project-automation bot moved this from Planning to Completed in Jan & Cortex Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

4 participants