-
-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Misc] Update to Transformers 4.48 #12120
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Tyler Michael Smith <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can do one of these:
🚀 |
FYI @mgoin, @dsikka, @robertgshaw2-redhat on the neuralmagic/Llama-3.2-1B-quantized.w8a8 issue |
@@ -2,7 +2,7 @@ | |||
# This file is autogenerated by pip-compile with Python 3.12 | |||
# by the following command: | |||
# | |||
# python3.12 -m piptools compile requirements-test.in -o requirements-test.txt | |||
# pip-compile --output-file=requirements-test.txt requirements-test.in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please keep the original command as we want to keep this in Python 3.12 to maintain consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@DarkLight1337 yea this is what I got when I used py 3.12 and the following versions to compile with the same command
python3.12 -m piptools compile requirements-test.in -o requirements-test.txt -P transformers
but what comes out is the command that you see above, can you suggest what is the preferred way to compile? Or just manually update the command?
versions:
build==1.2.2.post1
click==8.1.8
packaging==24.2
pip-tools==7.4.1
pyproject_hooks==1.2.0
setuptools==75.8.0
wheel==0.45.1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@DarkLight1337 I thought the same thing when I saw it in #10909, but when you run
python3.12 -m piptools compile requirements-test.in -o requirements-test.txt
it generates
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
# pip-compile --output-file=requirements-test.txt requirements-test.in
You still think I should update the comment? I can definitely see a case for being explicit about python 3.12 in the command. If so I'll update the comment indicating that the command should be left as-is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yea it doesnt print the command that was actually run, it messages it to this format, which is actually consistent with the pip-tools
docs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cc @khluu since you made this change.
The transformers 4.48 release has the required change in how Automodel is expected to handle compressed vs frozen models. The model in question has the frozen state, which really should be compressed. Likely just outdated I'll look into updating the model |
The custom model in question that is causing Language Model Test to fail is this one in particular. What can we do about this since this code is only modifiable but the its maintainers?.
|
The same issue has appeared in a few other models as a result of the release e.g Deepseek |
I put up a PR to fix those ch
I have a PR up to fix it just for MiniCMP here: https://huggingface.co/openbmb/MiniCPM3-4B/discussions/39 But alternatively if there are multiple models with the same issue, I think it would be better to fix via huggingface/transformers#35734 -- @dsikka do you have links to other models with the same issue? |
FYI the model has been updated and the tests now pass (at least locally) |
The basic correctness tests pass for me if using this: huggingface/transformers#35681. For now we can wait until it gets merged, or skip the tests until the new point release of transformers |
Added the ready label to see if the Multi GPU test is intermittent |
Looks like the multi gpu tests are passing so we should be good to go once huggingface/transformers#35681 lands. I'm inclined to wait for a transformers point release but lmk if you disagree |
@tlrmchlsmth thanks! Im absolutely on board with waiting for the point release |
@@ -533,6 +533,7 @@ tqdm==4.66.6 | |||
# lm-eval | |||
# nltk | |||
# peft | |||
# pqdm | |||
# sentence-transformers | |||
# tqdm-multiprocess | |||
# transformers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should update transformers
version in this file as well. Some of the resulting errors have been fixed in #12187
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tlrmchlsmth ah yes, when you generate the file you need to pass the -P
argument so that the transformers
version get updated.
python3.12 -m piptools compile requirements-test.in -o requirements-test.txt -P transformers
Update Transformers to 4.48. Split off from #10909 to isolate any 4.48-related changes here and for easier debugging.
From @fabianlim, we have the following open issues: