[ENH] Add derivatives command and pipeline-catalog submodule

[alyssadai]

• Closes Investigate schema and examples for the Nipoppy imaging bagel #344
• Closes Add new sub-command for bagel-CLI derivatives: #345
• Closes update bagel bids subcommand to play nice with new derivatives #346
• See also [ENH] Update JSONLD examples with @context and outputs from bagel derivatives neurobagel_examples#34
  • should be merged first, currently causing test failures due to examples in this repo being outdated (old @context)

Changes proposed in this pull request:

• Add nipoppy/pipeline-catalog as submodule
• Create new bagel derivatives command
  • Add basic input checks of processing status TSV: no missing subject IDs, no new (unique) subjects, only recognized pipeline names + vers
  • Add command logic to extract completed pipelines per subject-session based on step statuses
  • Aggregate valid pipeline info for missing sessions for a subject under a new custom ses-nb01 ImagingSession
• Move I/O utils shared between commands into a new module file_utils.py to avoid circular imports
• Add new derivative_utils.py to hold helper funcs for bagel derivatives
• Add new example processing status TSVs (7 total)
• Switch to re-generating the @context in each command (previously, we were stripping it from an input JSONLD and re-adding it at the end. This led to sometimes ending up with an incomplete @context without all the terms used/added to the dataset by the command that was just run)

Checklist

This section is for the PR reviewer

• PR has an interpretable title with a prefix ([ENH], [FIX], [REF], [TST], [CI], [MNT], [INF], [MODEL], [DOC]) (see our Contributing Guidelines for more info)
• PR has a label for the release changelog or skip-release (to be applied by maintainers only)
• PR links to GitHub issue with mention Closes #XXXX
• Tests pass
• Checks pass

For new features:

• Tests have been added

For bug fixes:

• There is at least one test that would fail under the original bug conditions.

[codecov]

Codecov Report

Attention: Patch coverage is 99.32660% with 2 lines in your changes missing coverage. Please review.

Project coverage is 98.37%. Comparing base (5d15125) to head (e0a6b10).
Report is 11 commits behind head on main.

Files with missing lines	Patch %	Lines
bagel/utility.py	95.45%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #349      +/-   ##
==========================================
+ Coverage   98.08%   98.37%   +0.29%     
==========================================
  Files          13       16       +3     
  Lines         782      983     +201     
==========================================
+ Hits          767      967     +200     
- Misses         15       16       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[surchs]

Thanks very much @alyssadai, looks great.

🧑‍🍳

[alyssadai]

Keeping this PR open for now to merge branch for #346 back into this one

[neurobagel-bot]

🚀 PR was released in v0.3.0 🚀