Adding READMEs to all directories #584

skeating · 2024-12-24T10:30:27Z

Description

Adds a README to all directories to facilitate easier navigation of the tree
Also adds comments where existing README did not make sense/was confusing
Corrected a few areas that were slightly confusing to read

Type of change

Please delete options accordingly to the description.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Suggested Checklist

I have performed a self-review of my own code.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have commented my code, particularly in hard-to-understand areas.
I have passed on my local host device. (see further details at the CONTRIBUTING document)
Make sure your branch is up-to-date with main branch. See CONTRIBUTING for a general example to syncronise your branch with the main branch.
I have requested review to this PR.
I have addressed and marked as resolved all the review comments in my PR.
Finally, I have selected squash and merge

* Remove private tag used for pixl project name Pass the project name in payload when calling orthanc anon plugin * Update tests to reflect project name tag has been removed * Update export api to take project name in payload * Update system tests to reflect we've removed the project name tag * Remove unused function * Explicitly set dicom spec edition in pixl_dcmd * Update renovate config to monitor dicom spec edition * Rename original_study_info to identifiable_study_info * remove UCLH Project Name tag from tag configs * Remove Private Creator Data Element tag * Fix formatting of orthanc.json for orthanc-raw * specify version format when extracting latest edition of dicom spec

* Adds configs files for mr spectroscopy with relevant tags #511 * Adds Stef`s suggestion on expecting none in dicom at projects/configs/mr-spectroscopy.yaml Co-authored-by: Stef Piatek <[email protected]> * adds tags for 0002 and 0008 groups from `DICOM/csi_slaser_40_20_MR/1.dcm` * adds some ~20% of the tags from `dicom.dcmread(DICOM/csi_slaser_40_20_MR/1.dcm` which might be or not included (also needs to be formated). Do we need to include all tags with a config format? * remove ms-pinpoint from base tag operations for mr-spectroscopy * Keep all tags unique to mr spectroscopy We're not yet keeping nested tags except for those that cause validation errors * Add manufacturer overrides for mr-spectroscopy * Rename mr-spectroscopy project to test-mr-spectroscopy project * Use latest version of dicom-validator We're running into this issue in 0.6.3: pydicom/dicom-validator#123 * remove manufacturer overrides for mr-spectroscopy * remove patient id from spectroscopy anonymisation config * Move config Acquisition DateTime, Content Date, and Content Time to mri.yaml Also replace the tags rather than keep --------- Co-authored-by: Stef Piatek <[email protected]> Co-authored-by: Paul Smith <[email protected]>

#532) * Check the instance count for local studies before getting SOPInstanceUIDs * Add function to get statistics for local study in orthanc

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Stef Piatek <[email protected]>

* Renovate: dependency PyYAML to v6.0.2 * Renovate: dependency loguru to v0.7.3 * Renovate: dependency pandas to v2.2.3 * Renovate: dependency pytest to v8.3.4 * Renovate: dependency requests to v2.32.3 * Renovate: dependency sqlalchemy to v2.0.36 * Renovate: dependency xnat4tests to v0.3.12 * Renovate: rabbitmq Docker tag to v3.13.7 * Renovate: dependency aio_pika to v9.5.3 * Renovate: dependency alembic to v1.14.0 * Renovate: dependency azure-identity to v1.19.0 * Renovate: dependency azure-storage-blob to v12.24.0 * Renovate: dependency fastapi to v0.115.6 * Renovate: dependency httpx to ==0.28.* * Renovate: dependency hypothesis to v6.122.3 * Renovate: dependency nibabel to v5.3.2 * Renovate: dependency pyOpenSSL to v24.3.0 * Renovate: dependency pydantic to v2.10.3 * Renovate: dependency pytest-check to v2.4.1 * Renovate: dependency tqdm to v4.67.1 * Renovate: dependency uvicorn to v0.32.1 * Renovate: codecov/codecov-action to v5 * Renovate: dependency jsonpickle to v4 * Renovate: dependency pyarrow to v18 * Renovate: dependency pyftpdlib to v2 --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Renovate: dependency dicom-anonymizer to v1.0.13.post1 * Use ActionsMapNameFunctions when getting the tag functions --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Smith <[email protected]> Co-authored-by: Stef Piatek <[email protected]>

* Add configuration for ION Neuro DB project

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Smith <[email protected]>

…#581) * Ensure we're testing the correct modalities in the dcmd tests * ensure tags are not duplicated across base configs * Add missing tags to config files * remove unused fixtures * remove unused MODALITY_TAGS dict * update pixl_core test to reflect change in test project config * remove MR as a supported modality in the test project config otherwise system tests fail * Remove MR from list of modalities in project config tests in core * set mri diffusion dataset modality to DX otherwise instance is dropped due to unsupported modality * remove projects/configs/tag-operations/test-extract-uclh-omop-cdm.yaml as it's empty * remove mention of test-extract-uclh-omop-cdm.yaml tag operation file as it have been removed * move uclh-nasogastric-tube-project-ngt-only-full-dataset tags into xray config and move duplicated tags into base

* Add timeout for dicomweb server setup * Remove unused env varible * Count and log skipped instance reason * Log validation errors at the study level * Anonymise all studies and upload before export-api * Increase timeout of anon In line with raw * Cache hashed identifiers

* Start documenting design choices

…s-pub

codecov · 2024-12-24T10:31:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.13%. Comparing base (e4f3fcc) to head (8d7faf6).
Report is 23 commits behind head on sk/joss-publication.

Additional details and impacted files

@@                 Coverage Diff                  @@
##           sk/joss-publication     #584   +/-   ##
====================================================
  Coverage                85.13%   85.13%           
====================================================
  Files                       72       72           
  Lines                     3161     3161           
====================================================
  Hits                      2691     2691           
  Misses                     470      470

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* Log study info in anonymisation To enable easier tracking * Make dicom helpers public

Fixes #588 by de-duping the orthanc and PIXL dockerfiles. This was factored out from PR #400 which had become too big. * Merge three pixl python dockerfiles into one multi-stage dockerfile to avoid repetition. * Merge orthanc dockerfiles, downloading dicom spec at a more separate stage. * Specify healthcheck command in only one place

* Strip whitespace and remove empty identifiers

* Add initial de-identification profile for rt data * Add missing tags to keep for RTSTRUCT modality * remove redundant tags from rt-plan config * remove unnecessary tags from rt-dose config * Add default config for CT files * Add missing tags for rt-dose config * Add missing tags for rt-plan config * Ensure we keep linkages between rt images, dose, plan, and struct files by using the same uids after anonymisation * Keep tags related to multi-frame DICOM * Add mappings for RTDOSE, RTSTRUCT, and RTPLAN to MODALITY_TO_CLASS_UID dictionary * remove ct.yaml tag operation file all tags are duplicated in base * remove duplicated tags from rt tag operation files * remove ct.yaml from test rt project config * replace date and times in RT Plans and Structure Sets

…he exception if it exists (#590)

skeating · 2025-01-13T10:01:20Z

Note: I have only made changes to README files, but I have imported changing from the main branch so a lot of files appear to have changed

stefpiatek and others added 26 commits November 27, 2024 10:02

Tweak logging for anonymisation (#528)

76e35fc

update main read me

93dc6ba

Fix timeout issues when querying instances of a local study in Orthanc (

d0f809b

#532) * Check the instance count for local studies before getting SOPInstanceUIDs * Add function to get statistics for local study in orthanc

Renovate: ubuntu to v24 (#568)

87edbea

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: dependency pytest-asyncio to v0.24.0 (#556)

30bbd3b

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: dependency aiohttp to v3.10.11 [SECURITY] (#533)

b4d5888

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: dependency psycopg2-binary to v2.9.10 (#539)

2abbcfb

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: dependency jsonpickle to v3.4.2 (#552)

0ebd195

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: dependency pytest-cov to v6 (#565)

0d2f3a9

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Stef Piatek <[email protected]>

Group all non-major updates into a single PR (#569)

9dbb114

Add configuration for IoN Neuro DB project - prognosis ai (#525)

cf96120

* Add configuration for ION Neuro DB project

Renames file to prevent pytest conflict (#573)

019032f

Renovate: actions/checkout to 11bd719 (#575)

8ed7655

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Renovate: docker/setup-buildx-action to c47758b (#577)

83c1140

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Smith <[email protected]>

Renovate: actions/setup-python to 0b93645 (#576)

f1c63fb

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Smith <[email protected]>

Renovate: postgres:16-bookworm Docker digest to 5620f24 (#578)

6435ebb

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Paul Smith <[email protected]>

Start documenting design choices (#574)

1d62bf7

* Start documenting design choices

update main read me (#529)

edbf6dc

Merge branch 'sk/joss-1' into sk/joss-pub

235ca58

Merge remote-tracking branch 'origin/sk/joss-publication' into sk/jos…

8d7faf6

…s-pub

skeating added 3 commits December 24, 2024 14:39

made read mes consistent

65c7b5f

Made the read mes more consistent

3dcc4fb

made read mes more consistent

a48ec93

stefpiatek and others added 16 commits January 2, 2025 18:44

Log study info in anonymisation (#587)

208e74c

* Log study info in anonymisation To enable easier tracking * Make dicom helpers public

making readmes consistent

7c777c9

making readmes consistent

4109db4

making readmes consistent

08b4ca5

making readmes consistent

474894e

Strip whitespace and remove empty identifiers (#586)

ecd5abf

* Strip whitespace and remove empty identifiers

Making readmes more consistent

b0360ea

always try to create the output folder on the ftps server and catch t…

b3921b6

…he exception if it exists (#590)

Making readmes consistent

2babe4d

Merge remote-tracking branch 'origin/main' into sk/joss-pub

6bf6989

Making readmes consistent

415761f

Making readmes consistent

57f13b9

Making readmes consistent

e00eeae

Making readmes consistent

35c34b2

skeating requested a review from stefpiatek January 13, 2025 09:57

skeating marked this pull request as ready for review January 13, 2025 09:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding READMEs to all directories #584

Adding READMEs to all directories #584

skeating commented Dec 24, 2024 •

edited

Loading

codecov bot commented Dec 24, 2024 •

edited

Loading

skeating commented Jan 13, 2025

Adding READMEs to all directories #584

Are you sure you want to change the base?

Adding READMEs to all directories #584

Conversation

skeating commented Dec 24, 2024 • edited Loading

Description

Type of change

Suggested Checklist

codecov bot commented Dec 24, 2024 • edited Loading

Codecov Report

skeating commented Jan 13, 2025

skeating commented Dec 24, 2024 •

edited

Loading

codecov bot commented Dec 24, 2024 •

edited

Loading