Unable to compute metrics for model output/lego #481

yangqinhui0423 · 2023-11-18T10:36:55Z

Hello，everyone！
I trained it on the NeRF-synthetic dataset, and it worked, and it rendered. But when I run python metrics.py -m output/lego, I get an error saying Unable to compute metrics for model output/lego.

Has anyone encountered a similar problem and knew how to solve it?
PS：I was able to compute metrics on the Mip-NeRF360 dataset after colmap.

RetroFuturism7 · 2024-01-05T07:28:05Z

same issue.

curious-energy · 2024-01-24T08:48:10Z

I made a simple change, please see #632.

curious-energy · 2024-01-24T12:13:37Z

I made a simple change, please see #632.

and another reason is VRAM OOM when the progress bar is interrupted.

LakeofRain · 2024-01-25T07:16:38Z

I made a simple change, please see #632.

I have solved the problem of 'Unable to compute metrics for model ' by 'pip install torchmetrics', although don't know why, but it does work well. Now I can obtain the metrics.

leblond14u · 2024-03-25T14:21:47Z

Same problem here metrics.py does not work. Tried to install torchmetrics but:

No improvements with pip install
And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation :

All of the metrics are NaN ...

Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

shenqianxiang · 2024-06-26T14:13:34Z

我做了一个简单的更改，请参阅#632。

你好请问你是怎么解决的

shenqianxiang · 2024-06-26T15:06:46Z

同样的问题。

你解决了吗

LakeofRain · 2024-06-26T15:19:50Z

我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang ***@***.***>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting ***@***.***>抄送： LakeofRain ***@***.***>, Comment ***@***.***>主题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

biter0088 · 2024-09-05T17:16:36Z

I re-found a new environment on my computer with the torch package, ran pip install torchmetrics and re-executed python metrics.py -m “/home/xiu/Learning/gaussian-splatting/output/train” and was able to output the following results:

# Metric evaluation progress: 100%|█████████████████████████████████████████████████████████████████████████| 38/38 [00:39<00:00, 1.05 s/it]
# here
# SSIM : 0.8076466
# PSNR : 21.7869835
# LPIPS: 0.2143032

kellofcn · 2024-10-15T08:24:58Z

Same problem here metrics.py does not work. Tried to install torchmetrics but:

No improvements with pip install
And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation : All of the metrics are NaN ...

Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is what I have executed python metrics.py -m /home/lwk/gaussian-splatting/output/980df71c-a , metric evaluation progress is 0it , do u know why?
Scene: /home/lwk/gaussian-splatting/output/980df71c-a Method: ours_18000 Metric evaluation progress: 0it [00:00, ?it/s] SSIM : nan PSNR : nan LPIPS: nan

HanYan012 · 2024-11-23T18:04:18Z

Same problem here metrics.py does not work. Tried to install torchmetrics but:

No improvements with pip install
And the conda install of torchmetrics is messing with my conda environment

This is what I have after a full metrics evaluation : All of the metrics are NaN ...
Sometimes it also crashes :

Traceback (most recent call last):
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 150, in <module>
    evaluate(args.model_paths)
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/metrics.py", line 116, in evaluate
    lpipss.append(lpips(renders[idx], gts[idx], net_type='vgg'))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/__init__.py", line 20, in lpips
    criterion = LPIPS(net_type, version).to(device)
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/lpips.py", line 28, in __init__
    self.lin.load_state_dict(get_state_dict(net_type, version))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/Bureau/PhD/Depth_3DGS/lpipsPyTorch/modules/utils.py", line 17, in get_state_dict
    old_state_dict = torch.hub.load_state_dict_from_url(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/hub.py", line 770, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1028, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hleblond/anaconda3/envs/GS/lib/python3.11/site-packages/torch/serialization.py", line 1264, in _legacy_load
    typed_storage._untyped_storage._set_from_file(
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

This is what I have executed python metrics.py -m /home/lwk/gaussian-splatting/output/980df71c-a , metric evaluation progress is 0it , do u know why? Scene: /home/lwk/gaussian-splatting/output/980df71c-a Method: ours_18000 Metric evaluation progress: 0it [00:00, ?it/s] SSIM : nan PSNR : nan LPIPS: nan

Same. Did you solve this?

tapowanliwuyun · 2024-12-24T11:45:25Z

Same. Did you solve this?

tapowanliwuyun · 2024-12-24T13:30:12Z

solved！please chekck /home/dkcs/.cache/torch/hub/checkpoints/vgg.pth ....

micki-37 · 2025-01-10T02:00:50Z

我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang @.>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting @.>抄送： LakeofRain @.>, Comment @.>主题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

我也是vgg网络没下完 重新跑metrics就显示算不了了 但并不会重新下vgg的权重 是得重新训练完重新下vgg吗

LakeofRain · 2025-01-10T05:12:30Z

我换了个torchvision的版本就解决了，没重新训练在 2025-01-10 10:01:12，"micki-37" ***@***.***> 写道：我是因为torchvision版本不对，算lpips时vgg网络没下载下来。发自我的荣耀手机-------- 原始邮件 --------发件人： shenqianxiang @.>日期： 2024年6月26日周三 23:07收件人： graphdeco-inria/gaussian-splatting @.>抄送： LakeofRain @.>, Comment @.>主题： Re: [graphdeco-inria/gaussian-splatting] Unable to compute metrics for model output/lego (Issue #481) 同样的问题。你解决了吗 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***> 我也是vgg网络没下完重新跑metrics就显示算不了了但并不会重新下vgg的权重是得重新训练完重新下vgg吗 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to compute metrics for model output/lego #481

Unable to compute metrics for model output/lego #481

yangqinhui0423 commented Nov 18, 2023 •

edited

Loading

RetroFuturism7 commented Jan 5, 2024

curious-energy commented Jan 24, 2024

curious-energy commented Jan 24, 2024

LakeofRain commented Jan 25, 2024

leblond14u commented Mar 25, 2024 •

edited

Loading

shenqianxiang commented Jun 26, 2024

shenqianxiang commented Jun 26, 2024

LakeofRain commented Jun 26, 2024 via email

biter0088 commented Sep 5, 2024 •

edited

Loading

kellofcn commented Oct 15, 2024

HanYan012 commented Nov 23, 2024

tapowanliwuyun commented Dec 24, 2024

tapowanliwuyun commented Dec 24, 2024

micki-37 commented Jan 10, 2025

LakeofRain commented Jan 10, 2025 via email

Unable to compute metrics for model output/lego #481

Unable to compute metrics for model output/lego #481

Comments

yangqinhui0423 commented Nov 18, 2023 • edited Loading

RetroFuturism7 commented Jan 5, 2024

curious-energy commented Jan 24, 2024

curious-energy commented Jan 24, 2024

LakeofRain commented Jan 25, 2024

leblond14u commented Mar 25, 2024 • edited Loading

shenqianxiang commented Jun 26, 2024

shenqianxiang commented Jun 26, 2024

LakeofRain commented Jun 26, 2024 via email

biter0088 commented Sep 5, 2024 • edited Loading

kellofcn commented Oct 15, 2024

HanYan012 commented Nov 23, 2024

tapowanliwuyun commented Dec 24, 2024

tapowanliwuyun commented Dec 24, 2024

micki-37 commented Jan 10, 2025

LakeofRain commented Jan 10, 2025 via email

yangqinhui0423 commented Nov 18, 2023 •

edited

Loading

leblond14u commented Mar 25, 2024 •

edited

Loading

biter0088 commented Sep 5, 2024 •

edited

Loading