Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for OpenAISpec in litgpt deploy #1908

Open
bhimrazy opened this issue Jan 17, 2025 · 2 comments
Open

Add support for OpenAISpec in litgpt deploy #1908

bhimrazy opened this issue Jan 17, 2025 · 2 comments
Labels
question Further information is requested

Comments

@bhimrazy
Copy link

bhimrazy commented Jan 17, 2025 •
edited
Loading

Title: Add Support for OpenAISpec

Description:
Would it be feasible for LitGPT to support the OpenAISpec from litserve?

This feature could enable seamless interaction with OpenAI-compatible APIs,
reducing friction for users looking to integrate their applications or workflows.
Is this something planned? I'd be happy to contribute.
@bhimrazy bhimrazy added the question Further information is requested label Jan 17, 2025
@bhimrazy bhimrazy changed the title Add support for OpenAISpec Add support for OpenAISpec in litgpt deploy Jan 17, 2025
@craigpfeifer
Copy link

I have looked into this also. I have the code for an implementation in this PR: #1865

The catch is litgpt needs to update use v0.2.5 of litserve (where the OpenAI spec was introduced), and that version causes the build tests to fail. I have done some investigation, but haven't had time to focus on it.

I'd be happy to work with you on this.

@bhimrazy
Copy link
Author

Sure @craigpfeifer . I will check the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@craigpfeifer @bhimrazy