Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix execution of WPS remote providers and corresponding metadata #342

Merged
merged 17 commits into from
Sep 29, 2021

Conversation

fmigneault
Copy link
Collaborator

Changes

  • add auto-handling of recoverable invalid XML characters (<, <=, etc.)
    Help support remote providers, temp fix until properly resolved by pywps/finch
    (see Escape XML responses geopython/pywps#628)
  • add end-to-end test of WPS provider registration, process description, execution and result retrieval
    (same process as tested by https://github.com/Ouranosinc/pavics-sdi/blob/master/docs/source/notebook-components/weaver_example.ipynb, but using mocked 'remote' WPS)
  • block provider operations when not applicable (Weaver ADES/DEFAULT config modes)
  • detect and block (better message of cause) invalid combinations of Weaver mode vs Remote provider execution
  • fix HYBRID mode not calling WPS remote provider CWL handler as expected
  • refactor provider functions into separate utils vs views modules for easier reuse and to avoid circular imports
  • fix URLs reported in links by processes nested under a provider Service.
  • fixes to OpenPAI schema definitions and resolution during validation

Fixes

@github-actions github-actions bot added ci/doc Issue related to documentation of the package ci/tests Tests of the package and features feature/CWL Issue related to CWL support feature/db Related to database or datatype manipulation. feature/oas Issues related to OpenAPI specifications. process/wps3 Issue related to WPS 3.x (REST-JSON) processes support feature/job Issues related to job execution, reporting and logging. labels Sep 28, 2021
@codecov
Copy link

codecov bot commented Sep 28, 2021

Codecov Report

Merging #342 (30e4c5e) into master (d229afd) will increase coverage by 3.70%.
The diff coverage is 87.79%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #342      +/-   ##
==========================================
+ Coverage   74.09%   77.79%   +3.70%     
==========================================
  Files          62       63       +1     
  Lines        9863     9981     +118     
  Branches     1487     1508      +21     
==========================================
+ Hits         7308     7765     +457     
+ Misses       2133     1732     -401     
- Partials      422      484      +62     
Impacted Files Coverage Δ
weaver/processes/execution.py 82.28% <ø> (+4.34%) ⬆️
weaver/wps_restapi/processes/__init__.py 100.00% <ø> (ø)
weaver/tweens.py 77.94% <50.00%> (ø)
weaver/processes/utils.py 78.47% <55.55%> (+0.09%) ⬆️
weaver/processes/wps_package.py 71.49% <62.50%> (+4.46%) ⬆️
weaver/datatype.py 70.13% <71.42%> (+1.96%) ⬆️
weaver/xml_util.py 92.30% <92.30%> (+3.41%) ⬆️
weaver/wps_restapi/providers/providers.py 79.67% <92.45%> (+25.03%) ⬆️
weaver/exceptions.py 86.00% <100.00%> (+0.14%) ⬆️
weaver/processes/convert.py 77.11% <100.00%> (+1.05%) ⬆️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b035cc5...30e4c5e. Read the comment docs.

matprov
matprov previously approved these changes Sep 29, 2021
dbyrns
dbyrns previously approved these changes Sep 29, 2021
Copy link
Contributor

@dbyrns dbyrns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few questions as usual, but overall everything looks fine :)

tests/utils.py Show resolved Hide resolved
weaver/wps_restapi/providers/providers.py Show resolved Hide resolved
…WPS providers + add 'ignore' query to raise on any failure during listing
@fmigneault fmigneault dismissed stale reviews from dbyrns and matprov via f989d82 September 29, 2021 20:25
@fmigneault fmigneault merged commit 9ef6a14 into master Sep 29, 2021
@fmigneault fmigneault deleted the fixes-wps-providers branch September 29, 2021 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/doc Issue related to documentation of the package ci/tests Tests of the package and features feature/CWL Issue related to CWL support feature/db Related to database or datatype manipulation. feature/job Issues related to job execution, reporting and logging. feature/oas Issues related to OpenAPI specifications. process/wps3 Issue related to WPS 3.x (REST-JSON) processes support
Projects
None yet
3 participants