Skip to content

Commit

Permalink
Update quark quantizer command in fp8 instruction (#49)
Browse files Browse the repository at this point in the history
* update quark quantizer command

* typo
  • Loading branch information
charlifu authored Jun 13, 2024
1 parent dc60612 commit 12079c3
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion ROCm_performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ python3 quantize_quark.py --model_dir [llama2 checkpoint folder] \
--output_dir output_dir \
--quant_scheme w_fp8_a_fp8_o_fp8 \
--num_calib_data 128 \
--export_safetensors \
--model_export vllm_adopted_safetensors \
--no_weight_matrix_merge
```
For more details, please refer to Quark's documentation.
Expand Down

0 comments on commit 12079c3

Please sign in to comment.