Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[adaround][mix-precicison][onnx] how to manually set some nodes to fp16 when applying adaround on one onnx model? #3736

Open
seriphdog opened this issue Jan 17, 2025 · 0 comments

Comments

@seriphdog
Copy link

Greetings.
I am dealing with a resnet_v2-7.onnx and I expect:

after applying adaround Alg. on all possible Conv/GeMM, this onnx will result in

  1. all non-fused BatchNorm nodes set as fp16,
  2. every Conv/GeMM set as W8A8/W4A8, (int4/8 here, not fp8 of course)
  3. accuracy drop < 3%

Typically, I might utilize a MixPrecisionSimModel class , and reset dtype && bitwidth manually.

Is there an example to achieve above functions?

For aimet-onnx, QuantizationSimModel enables me manually set different int-width for given nodes, only if aimet_onnx/quantsim.py and other necessary files properly revised.

However, any suggestions for fp16 case? aimet-torch verifies a MixedPrecisionConfigurator class, not sure if same concept has already been implemented for aimet-onnx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant