Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sparse tensor support on CPU requires MKL #8219

Closed
agitter opened this issue Oct 18, 2023 · 7 comments
Closed

Sparse tensor support on CPU requires MKL #8219

agitter opened this issue Oct 18, 2023 · 7 comments
Labels
installation

Comments

@agitter
Copy link

agitter commented Oct 18, 2023

😵 Describe the installation problem

I am installing PyG 2.4 on Windows 10 in a cpuonly setting. When running the examples/graph_gps.py example code, I encountered a RuntimeError related to MKL:

Processing train dataset:   0%|          | 0/10000 [00:00<?, ?it/s]C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\utils\sparse.py:264: UserWarning: Sparse CSR tensor support is in beta state. If you miss a functionality in the sparse tensor support, please submit a feature request to https://github.com/pytorch/pytorch/issues. (Triggered internally at  C:\cb\pytorch_1000000000000\work\aten\src\ATen\SparseCsrTensorImpl.cpp:68.)
  adj = torch.sparse_csr_tensor(
Traceback (most recent call last):
  File "C:\xxxxx\Desktop\madison\teaching\bmi826-f21\homework\pyg-yeast-hw\graph_gps.py", line 26, in <module>
    train_dataset = ZINC(path, subset=True, split='train', pre_transform=transform)
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\datasets\zinc.py", line 100, in __init__
    super().__init__(root, transform, pre_transform, pre_filter)
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\data\in_memory_dataset.py", line 76, in __init__
    super().__init__(root, transform, pre_transform, pre_filter, log)
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\data\dataset.py", line 102, in __init__
    self._process()
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\data\dataset.py", line 235, in _process
    self.process()
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\datasets\zinc.py", line 162, in process
    data = self.pre_transform(data)
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\transforms\base_transform.py", line 32, in __call__
    return self.forward(copy.copy(data))
  File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\transforms\add_positional_encoding.py", line 144, in forward
    out = out @ adj
RuntimeError: Calling addmm on CPU with sparse output requires MKL.

I'm unsure what this error message means. I have mkl 2022.1.0 from conda-forge in my conda environment.

The above error was encountered with PyTorch 1.12.0, the most recent version that supports pytorch-sparse for Windows. However, when I first tried with PyG 2.4, PyTorch 2.1, and no pytorch-sparse I received a similar error:

RuntimeError: addmm: computation on CPU is not implemented for SparseCsr + SparseCsr @ SparseCsr without MKL. PyTorch built with MKL has better support for addmm with sparse CPU tensors.

Environment

  • PyG version: 2.4.0
  • PyTorch version: 1.12.0
  • OS: Windows 10
  • Python version: 3.9.13
  • CUDA/cuDNN version: CPU
  • How you installed PyTorch and PyG (conda, pip, source): conda
  • Any other relevant information (e.g., version of torch-scatter): pytorch-scatter 2.0.9, pytorch-sparse 0.6.15
Complete environment 
# Name                    Version                   Build  Channel
blas                      2.116                       mkl    conda-forge
blas-devel                3.9.0              16_win64_mkl    conda-forge
brotli                    1.0.9                hcfcfb64_8    conda-forge
brotli-bin                1.0.9                hcfcfb64_8    conda-forge
brotlipy                  0.7.0           py39ha55989b_1005    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.9.24            h5b45459_0    conda-forge
certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39h68f70e3_2    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.6            py39h1f6ef14_0    conda-forge
cpuonly                   2.0                           0    pytorch
cryptography              38.0.3           py39h58e9bdb_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
fonttools                 4.38.0           py39ha55989b_1    conda-forge
freetype                  2.12.1               h546665d_0    conda-forge
gettext                   0.21.1               h5728263_0    conda-forge
glib                      2.74.1               h12be248_1    conda-forge
glib-tools                2.74.1               h12be248_1    conda-forge
gst-plugins-base          1.21.1               h001b923_1    conda-forge
gstreamer                 1.21.1               h6b5321d_1    conda-forge
icu                       70.1                 h0e60522_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.2.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h8ffe710_2    conda-forge
kiwisolver                1.4.4            py39h1f6ef14_1    conda-forge
krb5                      1.19.3               h1176d77_0    conda-forge
lcms2                     2.14                 h90d422f_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libblas                   3.9.0              16_win64_mkl    conda-forge
libbrotlicommon           1.0.9                hcfcfb64_8    conda-forge
libbrotlidec              1.0.9                hcfcfb64_8    conda-forge
libbrotlienc              1.0.9                hcfcfb64_8    conda-forge
libcblas                  3.9.0              16_win64_mkl    conda-forge
libclang                  15.0.5          default_h77d9078_0    conda-forge
libclang13                15.0.5          default_h77d9078_0    conda-forge
libdeflate                1.14                 hcfcfb64_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libglib                   2.74.1               he8f3873_1    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
liblapack                 3.9.0              16_win64_mkl    conda-forge
liblapacke                3.9.0              16_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.38               h19919ed_0    conda-forge
libsqlite                 3.40.0               hcfcfb64_0    conda-forge
libtiff                   4.4.0                h8e97e67_4    conda-forge
libuv                     1.44.2               h8ffe710_0    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp-base              1.2.4                h8ffe710_0    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.13               hcfcfb64_4    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markupsafe                2.1.1            py39ha55989b_2    conda-forge
matplotlib                3.6.2            py39hcbf5309_0    conda-forge
matplotlib-base           3.6.2            py39haf65ace_0    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
mkl-devel                 2022.1.0           h57928b3_875    conda-forge
mkl-include               2022.1.0           h6a75c08_874    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
networkx                  2.8.8              pyhd8ed1ab_0    conda-forge
numpy                     1.23.4           py39hbccbffa_1    conda-forge
openjpeg                  2.5.0                hc9384bd_1    conda-forge
openssl                   1.1.1s               hcfcfb64_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.5.1            py39h2ba5b7c_1    conda-forge
pcre2                     10.40                h17e33f8_0    conda-forge
pillow                    9.2.0            py39h595c93f_3    conda-forge
pip                       22.3.1             pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
psutil                    5.9.4            py39ha55989b_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyg                       2.4.0           py39_torch_1.12.0_cpu    pyg
pyopenssl                 22.1.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyqt                      5.15.7           py39hb77abff_2    conda-forge
pyqt5-sip                 12.11.0          py39h99910a6_2    conda-forge
pysocks                   1.7.1            py39hcbf5309_5    conda-forge
python                    3.9.13          h9a09f29_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytorch                   1.12.0              py3.9_cpu_0    pytorch
pytorch-mutex             1.0                         cpu    pytorch
pytorch-scatter           2.0.9           py39_torch_1.12.0_cpu    pyg
pytorch-sparse            0.6.15          py39_torch_1.12.0_cpu    pyg
pytz                      2022.6             pyhd8ed1ab_0    conda-forge
qt-main                   5.15.6               h9c3277a_1    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
scikit-learn              1.1.3            py39h6fe01c0_1    conda-forge
scipy                     1.9.3            py39hfbf2dce_2    conda-forge
setuptools                65.5.1             pyhd8ed1ab_0    conda-forge
sip                       6.7.4            py39h99910a6_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.40.0               hcfcfb64_0    conda-forge
tbb                       2021.7.0             h91493d7_0    conda-forge
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.2              py39ha55989b_1    conda-forge
tqdm                      4.64.1             pyhd8ed1ab_0    conda-forge
typing_extensions         4.4.0              pyha770c72_0    conda-forge
tzdata                    2022f                h191b570_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
unicodedata2              15.0.0           py39ha55989b_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
vc                        14.3                 h3d8a991_9    conda-forge
vs2015_runtime            14.32.31332          h1d6e394_9    conda-forge
wheel                     0.38.4             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0            py39hcbf5309_5    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
zstd                      1.5.2                h7755175_4    conda-forge
@rachitk
Copy link
Contributor

rachitk commented Oct 19, 2023

It may be useful to report what

python -m torch.utils.collect_env

returns as well as what

import torch
torch.backends.mkl.is_available()

returns. At first glance, it seems that your torch version was not built with MKL (which may happen if you installed MKL after torch or if torch for some reason didn't recognize your MKL install when you installed it).

@agitter
Copy link
Author

agitter commented Oct 19, 2023 

>>> import torch
>>> torch.backends.mkl.is_available()
True

$ python -m torch.utils.collect_env
Collecting environment information...
PyTorch version: 1.12.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Education
GCC version: (x86_64-posix-seh, Built by strawberryperl.com project) 8.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:50:36) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] torch==1.12.0
[pip3] torch_geometric==2.4.0
[pip3] torch-scatter==2.0.9
[pip3] torch-sparse==0.6.15
[conda] blas                      2.116                       mkl    conda-forge
[conda] blas-devel                3.9.0              16_win64_mkl    conda-forge
[conda] cpuonly                   2.0                           0    pytorch
[conda] libblas                   3.9.0              16_win64_mkl    conda-forge
[conda] libcblas                  3.9.0              16_win64_mkl    conda-forge
[conda] liblapack                 3.9.0              16_win64_mkl    conda-forge
[conda] liblapacke                3.9.0              16_win64_mkl    conda-forge
[conda] mkl                       2022.1.0           h6a75c08_874    conda-forge
[conda] mkl-devel                 2022.1.0           h57928b3_875    conda-forge
[conda] mkl-include               2022.1.0           h6a75c08_874    conda-forge
[conda] numpy                     1.23.4           py39hbccbffa_1    conda-forge
[conda] pyg                       2.4.0           py39_torch_1.12.0_cpu    pyg
[conda] pytorch                   1.12.0              py3.9_cpu_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] pytorch-scatter           2.0.9           py39_torch_1.12.0_cpu    pyg
[conda] pytorch-sparse            0.6.15          py39_torch_1.12.0_cpu    pyg

I installed all of the packages in one command from an environment file, so MKL wasn't added after torch. I was able to install PyG 2.4 on a different Linux system, so this is not a high priority issue.

@EdisonLeeeee
Copy link
Contributor

This issue has been resolved in #8197. You need to install PyG nightly for it.

@agitter
Copy link
Author

agitter commented Oct 19, 2023

Thanks for pointing me to that @EdisonLeeeee. I searched "MKL" but didn't find that pull request. Installing PyG from master did indeed resolve the original error.

I'm still having problems with sparse tensors on Windows in the GraphGPS example though:

File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\transforms\add_positional_encoding.py", line 150, in forward
    pe_list.append(get_self_loop_attr(*to_edge_index(out), N))
File "C:\xxxxx\.conda\envs\pyg-2023\lib\site-packages\torch_geometric\utils\sparse.py", line 410, in to_edge_index
    return adj.indices().detach().long(), adj.values()
RuntimeError: Cannot get indices on an uncoalesced tensor, please call .coalesce() first

@EdisonLeeeee EdisonLeeeee mentioned this issue Oct 19, 2023
Merged
rusty1s added a commit that referenced this issue Oct 19, 2023
@EdisonLeeeee @rusty1s
[Transforms] Fix sparse-sparse matrix multiplication support on Windo…
5a7d501 
…ws (Part 2) (#8225)

Address #8219

---------

Co-authored-by: Matthias Fey <[email protected]>
@EdisonLeeeee
Copy link
Contributor

@agitter Fixed via #8225. Thank you for pointing it out :)

@agitter
Copy link
Author

agitter commented Oct 20, 2023

Thank you! AddRandomWalkPE now works for me in this environment after reinstalling version 5a7d501.

@agitter agitter closed this as completed Oct 20, 2023
@SongZan222
Copy link

I also encountered the above error when running my own GNN-model on a CPU-only environment. (I'm using a laptop, I don't know if the program uses CPU or kernel)
Here is my environment：

OS：Windows 10
CPU： i7-1260P
GPU：Intel Iris Xe Graphics

python 3.10.2
Package Version

aiohttp 3.9.1
aiosignal 1.3.1
async-timeout 4.0.3
attrs 23.1.0
certifi 2023.11.17
charset-normalizer 3.3.2
colorama 0.4.6
contourpy 1.2.0
cycler 0.12.1
filelock 3.13.1
fonttools 4.46.0
frozenlist 1.4.0
fsspec 2023.12.2
idna 3.6
Jinja2 3.1.2
joblib 1.3.2
kiwisolver 1.4.5
lightning-utilities 0.10.0
MarkupSafe 2.1.3
matplotlib 3.8.2
mpmath 1.3.0
multidict 6.0.4
networkx 3.2.1
numpy 1.26.2
packaging 23.2
Pillow 10.1.0
pip 22.3.1
psutil 5.9.6
pyparsing 3.1.1
python-dateutil 2.8.2
pytorch-lightning 2.1.2
PyYAML 6.0.1
requests 2.31.0
scikit-learn 1.3.2
scipy 1.11.4
setuptools 65.5.1
six 1.16.0
sympy 1.12
threadpoolctl 3.2.0
torch 2.1.0
torch_geometric 2.4.0
torch-scatter 2.1.2+pt21cpu
torchmetrics 1.2.1
tqdm 4.66.1
typing_extensions 4.9.0
urllib3 2.1.0
wheel 0.38.4
yacs 0.1.8
yarl 1.9.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
installation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@agitter @rachitk @EdisonLeeeee @SongZan222