Error: unsupported dtype BF16 for op matmul #32
Comments
|
@kadirnar can you please provide a mininmum-reproducible example? |
diffusion_rs_cli --scale 3.5 --num-steps 50 dduf -f FLUX.1-dev-Q4-bnb.dduf
2025-01-08T15:18:35.608729Z INFO diffusion_rs_core::pipelines: loading from source: dduf file: FLUX.1-dev-Q4-bnb.dduf.
2025-01-08T15:18:35.608954Z INFO diffusion_rs_core::pipelines: model architecture is: flux
2025-01-08T15:18:35.679229Z INFO diffusion_rs_core::pipelines::flux: loading CLIP model
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 196/196 [00:00<00:00, 1147.15it/s]
2025-01-08T15:18:35.922026Z INFO diffusion_rs_core::pipelines::flux: loading T5 model
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 138/138 [00:00<00:00, 306.10it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 513/513 [00:04<00:00, 348.27it/s]
2025-01-08T15:18:42.869190Z INFO diffusion_rs_core::pipelines::flux: loading VAE model
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 244/244 [00:00<00:00, 3129.41it/s]
2025-01-08T15:18:43.007140Z INFO diffusion_rs_core::pipelines::flux: loading FLUX model
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2672/2672 [00:04<00:00, 498.33it/s]
2025-01-08T15:18:47.849220Z INFO diffusion_rs_core::pipelines::flux: FLUX pipeline using a guidance-distilled model: true
◇ Height:
│ 1024
│
◇ Width:
│ 1024
│
◇ Prompt:
│ a woman
│
Error: unsupported dtype BF16 for op matmul |
|
@kadirnar what feature flags did you use to install the CLI? |
cargo install diffusion_rs_cli --features cudaI don't know Rust but I solved it by using GPT4 to create src and cargo files. I might have done the installation incorrectly. Error Message error: failed to run custom build command for `diffusion_rs_common v0.1.0`
Caused by:
process didn't exit successfully: `/tmp/cargo-installzcx4CF/release/build/diffusion_rs_common-9ed7d75a0ac27c5a/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-changed=src/cuda_kernels/compatibility.cuh
cargo:rerun-if-changed=src/cuda_kernels/cuda_utils.cuh
cargo:rerun-if-changed=src/cuda_kernels/binary_op_macros.cuh
cargo:info=["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]
cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
cargo:rustc-env=CUDA_COMPUTE_CAP=90
--- stderr
thread 'main' panicked at /home/ubuntu/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen_cuda-0.1.5/src/lib.rs:527:9:
nvcc cannot target gpu arch 90. Available nvcc targets are [35, 37, 50, 52, 53, 60, 61, 62, 70, 72, 75, 80, 86, 87].
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `diffusion_rs_cli v0.1.0`, intermediate artifacts can be found at `/tmp/cargo-installzcx4CF`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.GPU: H100 |
That commend is correct for a CUDA machine. Since you have an H100, bf16 should definently be supported. I think I'm a bit unclear, though: it seems you were able to build and run, which is how you got the Can you please run and let me know the output of: |
|
Yes, can you please update nvcc to a major version of 12 (11.5 most likely does not support compute cap 9.0)? |
|
I'm having the exact same issue, but with the CPU backend (I don't have cuda, I installed with
|
|
@EricLBuehler I updated the nvcc version. I ran it again and I'm getting the same error cmd: Error Message: |
|
@kadirnar I merged #33 which should report some useful information on CUDA, can you please install the CLI from source to use this latest version and let me know the output? @Aloso I merged #33 which should hopefully resolve the issue that you are having! Can you please install the CLI from source to use this latest version? I'll let you know when the next release comes out with this feature (shortly). |
|
I'm now getting a different error:
|
|
I merged #34 which should fix this, can you please reinstall and try again?
This is confusing, I cannot reproduce the issue when I build for plain CPU on my machine after #34. I merged #34 which fixed a different bug, and now it runs on my machine. Can you please reinstall and try it again? |
|
Build: Error Message: Compiling zip v2.2.2
error[E0599]: no variant or associated item named `CUBLASLT_MATMUL_DESC_A_SCALE_POINTER` found for enum `cublasLtMatmulDescAttributes_t` in the current scope
--> diffusion_rs_backend/src/cublaslt/matmul.rs:183:63
|
183 | Matrix::A => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_A_SCALE_POINTER,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `cublasLtMatmulDescAttributes_t`
|
help: there is a variant with a similar name
|
183 | Matrix::A => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_BIAS_POINTER,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0599]: no variant or associated item named `CUBLASLT_MATMUL_DESC_B_SCALE_POINTER` found for enum `cublasLtMatmulDescAttributes_t` in the current scope
--> diffusion_rs_backend/src/cublaslt/matmul.rs:184:63
|
184 | Matrix::B => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_B_SCALE_POINTER,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `cublasLtMatmulDescAttributes_t`
|
help: there is a variant with a similar name
|
184 | Matrix::B => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_BIAS_POINTER,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0599]: no variant or associated item named `CUBLASLT_MATMUL_DESC_C_SCALE_POINTER` found for enum `cublasLtMatmulDescAttributes_t` in the current scope
--> diffusion_rs_backend/src/cublaslt/matmul.rs:185:63
|
185 | Matrix::C => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_C_SCALE_POINTER,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `cublasLtMatmulDescAttributes_t`
|
help: there is a variant with a similar name
|
185 | Matrix::C => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_BIAS_POINTER,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0599]: no variant or associated item named `CUBLASLT_MATMUL_DESC_D_SCALE_POINTER` found for enum `cublasLtMatmulDescAttributes_t` in the current scope
--> diffusion_rs_backend/src/cublaslt/matmul.rs:186:63
|
186 | Matrix::D => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_D_SCALE_POINTER,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant or associated item not found in `cublasLtMatmulDescAttributes_t`
|
help: there is a variant with a similar name
|
186 | Matrix::D => sys::cublasLtMatmulDescAttributes_t::CUBLASLT_MATMUL_DESC_BIAS_POINTER,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error[E0599]: no variant or associated item named `CUDA_R_8F_E4M3` found for enum `diffusion_rs_common::core::cuda_backend::mistralrs_cudarc_fork::cublaslt::sys::cudaDataType_t` in the current scope
--> diffusion_rs_backend/src/cublaslt/matmul.rs:634:30
|
634 | sys::cudaDataType_t::CUDA_R_8F_E4M3
| ^^^^^^^^^^^^^^ variant or associated item not found in `cudaDataType_t`
For more information about this error, try `rustc --explain E0599`.
error: could not compile `diffusion_rs_backend` (lib) due to 5 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `diffusion_rs_cli v0.1.0 (/home/ubuntu/kadir_dev/diff_opt/diffusion-rs/diffusion_rs_cli)`, intermediate artifacts can be found at `/home/ubuntu/kadir_dev/diff_opt/diffusion-rs/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path. |
I tested fp16, fp8, f4 models. However, I'm getting this error. Can you help?
The text was updated successfully, but these errors were encountered: