Implement EnsurePath.for_dataset() and resolve against that dataset #271
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I have used this check in datalad/datalad-metalad#362 and it was very useful to have! |
Previously this method only got a `Dataset`-type argument. However, for path resolution (for example), this information is insufficient, because DataLad needs to know whether the type of the original argument. For this reason `DatasetParameter` already existed, and is now also used for this purpose. This also simplified the type-annotation -- no need for custom typevars. It also made sense to move the `DatasetParameter` class to `base.py`, because every single constraint implementation has to support it (or use the default implementation from `base.py`.
This implements DataLad's standard path resolution approach (relative
is relative to CWD, unless a `Dataset` instance is given).
This now makes it possible to perform this standard resolution pattern
during command parameter validation, like so:
```py
\# example validator for a command with two args
EnsureCommandParameterization(
param_constraints=dict(
dataset=EnsureDataset(),
path=EnsurePath(),
),
tailor_for_dataset=dict(path='dataset'),
)
```
This will:
- cause the 'dataset' validator to run first
- auto-generate an `EnsurePath` variant that resolves against that
dataset
- ensures that the command always receives absolute paths, resolved
against the dataset whenever the rules require it.
Closes datalad#270
Now also all aspects of `path` resolution, and deriving a filename from an input URL are done outside the command implementation and in respective constraints. Improtantly, this now also uses a new feature of `EnsureGeneratorFromFileLike` that can now simply relay per-item exceptions to a caller (the `download()` command in this case). This allows for performing comprehensive item constraint evaluation, have that use regular exceptions (as if done upfront, but here actually during the runtime of a command, due to the long-running generator providing command input AKA batch mode). The command gets the exception yielded (not raised) and can decide what to do with it. `download()` maintains its previous behavior and reports an 'impossible' result.
The command can now immediately error when it is clear from the input arguments that no destination filename can be derived (when none is provided). When this is not possible, the command continue to yield an 'impossible' result. RF tests on invalid command calls into a dedicated function. No need for an http server in this case.
|
Failures are unrelated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements DataLad's standard path resolution approach (relative
is relative to CWD, unless a
Datasetinstance is given).This now makes it possible to perform this standard resolution pattern
during command parameter validation, like so:
This will:
EnsurePathvariant that resolves against thatdataset
against the dataset whenever the rules require it.
Closes #270
The PR also contains an update of the
downloadcommand to make use of these new abilities. As a result a substantial amount of custom code could be removed that partially duplicated path resolving and argument inspection (between upfront and runtime aspects).