Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splitting general and CORDEX-Checks, adding further checks #5

Merged
merged 34 commits into from
Jul 17, 2024
Merged

Conversation

sol1105
Copy link
Collaborator

@sol1105 sol1105 commented Jun 24, 2024

  • Create general CV Checker
    • Read CV, MIP tables
    • Check DRS (compliance with CV and consistency)
    • Check variables, coordinate variables (illegal names)
    • Check presence of required global attributes
    • Check value of required global attributes
    • Check version_date
    • Check creation_date
    • Check missing value of main variable
    • Check time axis, time bounds and time_range
    • Optionally writing JSON file with coordinate checksums, global and variable attrs for external consistency checks across files
  • Create specific CORDEX-CMIP6 Checker inheriting checks from the general CV Checker
    • Load tables from disk or download CORDEX-CMIP6 MIP tables
    • CORDEX specific checks for file format and compression
    • CORDEX specific check for time_range
    • CORDEX specific checks regarding the file chunking
    • CORDEX specific checks for calendar
    • CORDEX specific checks for time:units
    • CORDEX specific checks for driving_variant_label, driving_source_id and driving_experiment_id
    • CORDEX specific checks for domain_id
    • Check institution
    • CORDEX specific checks for optional/recommended attributes
      • grid * with * grid spacing *
      • version_realization_info
      • references
  • Test datasets and tests

In further PRs:

  • Check coordinates (incl. bounds) and variable
    • variable attributes, coordinate variable attributes
    • vertical coordinates / formula_terms
    • horizontal coordinates
    • CORDEX-specific: The 2-dimensional geographic latitudes and longitudes of the model grid cells ( lon and lat ) must be also provided as auxiliary coordinates. Longitude coordinates must be strictly monotonically increasing and confined to the range -180 to 360; they must also have absolute values as small as possible given the first two constraints. (e.g., store 170 E to 170 W as 170 to 190, but store 150 W to 130 W as -150 to -130, not 210 to 230.)
    • CORDEX-specific: grid_mapping variable crs: char or int, see CF-1.11 Appendix F
  • Check domain
  • Make use of logger library
  • Introduce id for each check
  • add tests and (manipulated) test datasets

sol1105 and others added 6 commits June 19, 2024 17:39
- added check for DRS path, DRS file name and suffix
- generalized reading of CMOR tables
Adding download of miptables similar to euro-cordex/cmor-check
@larsbuntemeyer
Copy link
Contributor

I can do the the domain coordinates checks!

@sol1105 sol1105 requested a review from larsbuntemeyer July 10, 2024 08:03
@sol1105
Copy link
Collaborator Author

sol1105 commented Jul 10, 2024

@larsbuntemeyer I think it is in a state that you could have a look. Would be great if you provided some general feedback, also for the "roadmap" sketched above.

I am yet uncertain whether dependent checks should generally throw an error when they cannot be conducted or simply should not run (i.e. the frequency cannot be identified, so most of the time checks will either not run or throw errors). I mostly opted for "not running".

@larsbuntemeyer
Copy link
Contributor

Thanks! I'll have a look asap!

@larsbuntemeyer
Copy link
Contributor

Would be great if you provided some general feedback, also for the "roadmap" sketched above.

I think it is straightforward to check the coordinate using the domain_id attribute and comparing this using cx.domain and using my former checks. But let's keep that for another PR...

@larsbuntemeyer
Copy link
Contributor

I think, this could be merged, if you like. This is lots of new stuff, we can then start from a common code base to tackle all of the rest.

@sol1105
Copy link
Collaborator Author

sol1105 commented Jul 17, 2024

Sounds good, thanks

@sol1105 sol1105 merged commit dfb23b3 into main Jul 17, 2024
6 checks passed
@sol1105 sol1105 deleted the drs branch July 17, 2024 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants