Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Add missing _id suffix to processing status file column mappings #370

Merged
merged 3 commits into from
Nov 4, 2024
Merged

[FIX] Add missing _id suffix to processing status file column mappings #370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2f607fd
update proc status column name mappings
alyssadai Nov 1, 2024
cb4ccbe
add _id to identifier col names in proc status files
alyssadai Nov 3, 2024
4d576e6
update comments
alyssadai Nov 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions bagel/derivatives_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@

# Shorthands for expected column names in a Nipoppy processing status file
# TODO: While there are multiple session ID columns in a Nipoppy processing status file,
# we only only look at `bids_session` right now. We should revisit this after the schema is finalized,
# we only only look at `bids_session_id` right now. We should revisit this after the schema is finalized,
# to see if any other logic is needed to avoid issues with session ID discrepancies across columns.
PROC_STATUS_COLS = {
"participant": "bids_participant",
"session": "bids_session",
"participant": "bids_participant_id",
"session": "bids_session_id",
"pipeline_name": "pipeline_name",
"pipeline_version": "pipeline_version",
"status": "status",
Expand Down
4 changes: 2 additions & 2 deletions bagel/tests/data/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,10 @@ Example file `proc_status`... | Description | Expected result
_synthetic.tsv | Captures a subset of subject-sessions represented in the BIDS examples synthetic dataset | Pass
_synthetic.csv | Same as proc_status_synthetic.csv, but is a CSV file | Fail
_unique_subs.tsv | Includes subjects not found in the phenotypic dataset | Fail
_incomplete.tsv | Has a missing value in the `bids_participant` column | Fail
_incomplete.tsv | Has a missing value in the `bids_participant_id` column | Fail
_unique_sessions.csv | Includes a unique subject-session (`sub-01`, `ses-03`) not found in the synthetic dataset | Pass
_missing_sessions.tsv | One subject (`sub-02`) is missing all session labels | Pass
_no_bids_sessions.tsv | Has session labels in all rows for `session_id`, but no values in `bids_session` column | Pass
_no_bids_sessions.tsv | Has session labels in all rows for `session_id`, but no values in `bids_session_id` column | Pass


## Example expected CLI outputs
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_missing_sessions.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
01 sub-01 01 ses-01 fmriprep 20.2.7 step1 SUCCESS
01 sub-01 01 ses-01 fmriprep 20.2.7 step2 SUCCESS
01 sub-01 01 ses-01 fmriprep 23.1.3 default SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_no_bids_sessions.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
01 sub-01 01 fmriprep 20.2.7 step1 SUCCESS
01 sub-01 01 fmriprep 20.2.7 step2 SUCCESS
01 sub-01 01 fmriprep 23.1.3 default SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_synthetic.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id,bids_participant,session_id,bids_session,pipeline_name,pipeline_version,pipeline_step,status
participant_id,bids_participant_id,session_id,bids_session_id,pipeline_name,pipeline_version,pipeline_step,status
01,sub-01,1,ses-01,fmriprep,20.2.7,step1,FAIL
01,sub-01,1,ses-01,fmriprep,20.2.7,step2,INCOMPLETE
01,sub-01,1,ses-01,fmriprep,23.1.3,default,SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_synthetic.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
01 sub-01 01 ses-01 fmriprep 20.2.7 step1 FAIL
01 sub-01 01 ses-01 fmriprep 20.2.7 step2 INCOMPLETE
01 sub-01 01 ses-01 fmriprep 23.1.3 default SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_synthetic_incomplete.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
01 sub-01 01 ses-01 fmriprep 20.2.7 step1 FAIL
01 sub-01 01 ses-01 fmriprep 20.2.7 step2 INCOMPLETE
01 sub-01 01 ses-01 fmriprep 23.1.3 default SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_unique_sessions.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
01 sub-01 01 ses-01 fmriprep 20.2.7 step1 FAIL
01 sub-01 01 ses-01 fmriprep 20.2.7 step2 INCOMPLETE
01 sub-01 03 ses-03 fmriprep 23.1.3 default SUCCESS
Expand Down
2 changes: 1 addition & 1 deletion bagel/tests/data/proc_status_unique_subs.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
participant_id bids_participant session_id bids_session pipeline_name pipeline_version pipeline_step status
participant_id bids_participant_id session_id bids_session_id pipeline_name pipeline_version pipeline_step status
pd1 sub-pd1 01 ses-01 fmriprep 20.2.7 step1 FAIL
pd1 sub-pd1 01 ses-01 fmriprep 20.2.7 step2 INCOMPLETE
pd2 sub-pd2 01 ses-01 fmriprep 23.1.3 default SUCCESS
Expand Down
4 changes: 2 additions & 2 deletions bagel/tests/test_cli_derivatives.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,8 @@ def test_derivatives_invalid_inputs_fail(
("proc_status_missing_sessions.tsv", {"sub-02": 2}),
# TODO: Revisit this example once the updated Nipoppy proc status file schema is available
# This example assumes that
# 1. It is possible to have a subject with missing values in bids_session but not in session_id
# 2. Duplicate entries of pipeline name, version, and step for an apparent subject-session based on bids_participant and bids_session
# 1. It is possible to have a subject with missing values in bids_session_id but not in session_id
# 2. Duplicate entries of pipeline name, version, and step for an apparent subject-session based on bids_participant_id and bids_session_id
# (i.e., the two columns Neurobagel looks at) are allowed (see rows 8 and 9)
("proc_status_no_bids_sessions.tsv", {"sub-01": 3, "sub-02": 2}),
],
Expand Down
4 changes: 2 additions & 2 deletions bagel/tests/test_utility.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -761,9 +761,9 @@ def test_create_completed_pipelines():
example_ses_proc_df = pd.DataFrame.from_records(
columns=[
"participant_id",
"bids_participant",
"bids_participant_id",
"session_id",
"bids_session",
"bids_session_id",
"pipeline_name",
"pipeline_version",
"pipeline_step",
Expand Down
Loading