Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Where is the 4bit attention API? #52

Open
BirdChristopher opened this issue Nov 27, 2024 · 5 comments
Open

Where is the 4bit attention API? #52

BirdChristopher opened this issue Nov 27, 2024 · 5 comments

Comments

@BirdChristopher
Copy link

I read the SageAttention2 tech report and I'm really thirsty for trying 4bit kernel. However I cannot find any API introduction in your README. Is it still under testing and not released?
@jt-zhang
Copy link
Member

Hello. The 4-bit API will be released in the near future.

@laomao0
Copy link

laomao0 commented Nov 28, 2024

Hello. The 4-bit API will be released in the near future.

looking forward the 4-int kernel

@jason-huang03
Copy link
Member

We are currently exploring some new techniques that can further improve the accuracy of 4 bit kernel.

@marvin-0042
Copy link

We are currently exploring some new techniques that can further improve the accuracy of 4 bit kernel.

awesome! looking forward to!

do you have high level timeline that when 4 bit kernel or algorithm reference code might be ready?

@asahni04
Copy link

hi @jason-huang03 @jt-zhang any updates on 4bit kernel?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@laomao0 @jt-zhang @BirdChristopher @jason-huang03 @marvin-0042 @asahni04