Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrap vec size 8 with USE_ROCM #1795

Open
wants to merge 2 commits into
base: 2.5_perf_fix
Choose a base branch
from
+4 −0

update

f69cc7e
Select commit
Loading
Failed to load commit list.
Open

Wrap vec size 8 with USE_ROCM #1795

update
f69cc7e
Select commit
Loading
Failed to load commit list.
ROCm Repo Management API / Tests / Tests / Test PyTorch / Run pytorch_test1 failed Jan 19, 2025 in 0s

failed: 39, skipped: 3620, passed: 37670

Send us feedback

Details

TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (48884.81632451079) from the precise computation than the torch result was (0.0009053018231455212)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (24.550849061126804) from the precise computation than the torch result was (0.004248719329927653)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (nan) from the precise computation than the torch result was (0.004607466882615279)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (24.550849061126804) from the precise computation than the torch result was (0.004248719329927653)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (125894081380380.61) from the precise computation than the torch result was (0.02007240644097054)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1371, in only_fn
    return fn(slf, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 564, in test_python_ref_executor
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (451.9713069588776) from the precise computation than the torch result was (0.0013502447967150277)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_var_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_var_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (1.572183849892433e+29) from the precise computation than the torch result was (0.054682756076321226)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_var_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_var_mean_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_var_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (8524.012307069732) from the precise computation than the torch result was (0.007912538481738451)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_var_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (nan) from the precise computation than the torch result was (0.004607466882615279)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (nan) from the precise computation than the torch result was (0.004607466882615279)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_std_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (59.79646238029621) from the precise computation than the torch result was (0.046462380563603745)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (59.79646238029621) from the precise computation than the torch result was (0.046462380563603745)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_linalg_norm_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1371, in only_fn
    return fn(slf, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 564, in test_python_ref_executor
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (57.64001618392368) from the precise computation than the torch result was (0.10998381607632268)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (48884.81632451079) from the precise computation than the torch result was (0.0009053018231455212)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_std_mean_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1371, in only_fn
    return fn(slf, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 564, in test_python_ref_executor
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (451.9713069588776) from the precise computation than the torch result was (0.0013502447967150277)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1371, in only_fn
    return fn(slf, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 564, in test_python_ref_executor
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (57.64001618392368) from the precise computation than the torch result was (0.10998381607632268)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 75: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=75 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_linalg_norm_executor_aten_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 520, in test_python_ref
    self._ref_test_helper(lambda: TorchRefsMode(strict=True), device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (24.550849061126804) from the precise computation than the torch result was (0.004248719329927653)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 20: SampleInput(input=Tensor[size=(2, 3, 2), device="cuda:0", dtype=torch.float16], args=('fro',(-2,-1),True), kwargs={}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=20 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref__refs_linalg_matrix_norm_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1371, in only_fn
    return fn(slf, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 564, in test_python_ref_executor
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (451.9713069588776) from the precise computation than the torch result was (0.0013502447967150277)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.float16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_executor__refs_std_mean_executor_aten_cuda_float16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
Stack trace 
Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1140, in test_wrapper
    return test(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1426, in only_fn
    return fn(self, *args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1547, in wrapper
    fn(*args, **kwargs)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 534, in test_python_ref_torch_fallback
    self._ref_test_helper(contextlib.nullcontext, device, dtype, op)
  File "/var/lib/jenkins/pytorch/test/test_ops.py", line 497, in _ref_test_helper
    self.assertTrue(ref_distance <= torch_distance, msg=msg)
  File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 687, in assertTrue
    raise self.failureException(msg)
AssertionError: tensor(False, device='cuda:0') is not true : Reference result was farther (125894081380380.61) from the precise computation than the torch result was (0.02007240644097054)!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 2979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 448, in instantiated_test
    result = test(self, **param_kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_utils.py", line 1526, in wrapper
    fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/testing/_internal/common_device_type.py", line 1152, in test_wrapper
    raise e_tracked from e
Exception: Caused by reference input at index 7: SampleInput(input=Tensor[size=(5, 5, 5), device="cuda:0", dtype=torch.bfloat16], args=(), kwargs={'dim': 'None', 'correction': '0', 'keepdim': 'True'}, broadcasts_input=False, name='')

To execute this test, run the following from the base repo dir:
    PYTORCH_OPINFO_SAMPLE_INPUT_INDEX=7 PYTORCH_TEST_WITH_ROCM=1 python test/test_ops.py TestCommonCUDA.test_python_ref_torch_fallback__refs_var_mean_cuda_bfloat16

This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0

more test results are not shown here, view them on Jenkins

View more details on ROCm Repo Management API