Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid interpolation examples fail on pandas 2.2.0. #83

Open
juseg opened this issue Jun 7, 2024 · 1 comment
Open

Grid interpolation examples fail on pandas 2.2.0. #83

juseg opened this issue Jun 7, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@juseg
Copy link
Owner

juseg commented Jun 7, 2024

Documentation build raises an OverflowError on pandas>=2.0.0 pandas==2.2.0.

python examples/interp/plot_interpolated_surface.py

Traceback (most recent call last):
  File "conversion.pyx", line 156, in pandas._libs.tslibs.conversion.cast_from_unit_vectorized
OverflowError: value too large

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/julien/git/code/hyoga/examples/interp/plot_interpolated_surface.py", line 31, in <module>
    ds = ds.hyoga.interp(hyoga.open.example('pism.alps.vis.refined.nc'))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/git/code/hyoga/hyoga/core/accessor.py", line 383, in interp
    ds = ds.interp(x=x, y=y)
         ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/dataset.py", line 3377, in interp
    if is_duck_dask_array(var.data):
                          ^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 434, in data
    return self.values
           ^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 607, in values
    return _as_array_or_item(self._data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 313, in _as_array_or_item
    data = np.asarray(data)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 658, in __array__
    self._ensure_cached()
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 655, in _ensure_cached
    self.array = NumpyIndexingAdapter(np.asarray(self.array))
                                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 628, in __array__
    return np.asarray(self.array, dtype=dtype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 529, in __array__
    return np.asarray(array[self.key], dtype=None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/variables.py", line 72, in __array__
    return self.func(self.array)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/times.py", line 333, in decode_cf_timedelta
    result = to_timedelta_unboxed(num_timedeltas.ravel(), unit=units)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/times.py", line 316, in to_timedelta_unboxed
    result = pd.to_timedelta(value, **kwargs).to_numpy()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/tools/timedeltas.py", line 213, in to_timedelta
    return _convert_listlike(arg, unit=unit, errors=errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/tools/timedeltas.py", line 266, in _convert_listlike
    td64arr = sequence_to_td64ns(arg, unit=unit, errors=errors, copy=False)[0]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/arrays/timedeltas.py", line 1059, in sequence_to_td64ns
    data = cast_from_unit_vectorized(data, unit or "ns")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "conversion.pyx", line 160, in pandas._libs.tslibs.conversion.cast_from_unit_vectorized
pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: cannot convert input -775154880000.0 with the unit 's'
@juseg juseg added the bug Something isn't working label Jun 7, 2024
@juseg juseg added this to the v0.3.1 milestone Jun 7, 2024
@juseg juseg self-assigned this Jun 7, 2024
@juseg juseg changed the title Grid interpolation examples fail on pandas 2. Grid interpolation examples fail on pandas 2.2.0. Jun 10, 2024
@juseg
Copy link
Owner Author

juseg commented Jun 10, 2024

The issue only concerns the latest release of pandas 2.2.0 and has been documented in:

According to the trace above ds.hyoga.interp() somehow tries to decode CF times and that triggers the bug.

@juseg juseg removed this from the v0.3.1 milestone Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant