Enables PIXL DB to operate as a service separate from Orthanc Raw

[tomaroberts]

Description

Fixes #517:

Adds the means to create two postgres services. Previously, PIXL DB was used by both Orthanc Raw and for storing patient metadata.

Now it is possible to have separate databases for Orthanc Raw and PIXL DB. By default and in keeping with previous design, the root ./docker-compose.yml brings up a single postgres service, shared by both Orthanc Raw and PIXL DB.

With this PR, test/docker-compose.yml now contains a new postgres service demonstrating how to bring up a separate service for PIXL DB.

Type of change

Please delete options accordingly to the description.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Suggested Checklist

• I have performed a self-review of my own code.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have commented my code, particularly in hard-to-understand areas.
• I have passed on my local host device. (see further details at the CONTRIBUTING document)
• Make sure your branch is up-to-date with main branch. See CONTRIBUTING for a general example to syncronise your branch with the main branch.
• I have requested review to this PR.
• I have addressed and marked as resolved all the review comments in my PR.
• Finally, I have selected squash and merge

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.41%. Comparing base (dca65ca) to head (502f27e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #593   +/-   ##
=======================================
  Coverage   87.41%   87.41%           
=======================================
  Files          76       76           
  Lines        3425     3425           
=======================================
  Hits         2994     2994           
  Misses        431      431           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[p-j-smith]

looking very nice!

I think the docs could do with updating in a couple of places:

• https://github.com/SAFEHR-data/PIXL/tree/main/cli#configuration
• https://github.com/SAFEHR-data/PIXL/tree/main/orthanc/orthanc-raw#step-1

And possibly some more places too. Which makes me wonder whether it would be worth describing all the environment variables in a single README rather than across the repo (as a separate PR)?

Also, if we use an external postgres instance, would we need to change how we handle alembic migrations? Currently we run them when the container is started, but I guess we'd need to apply them manually if postgres is running elsewhere?

[stefpiatek]

Looking good, thanks for getting this in 🎉 . Agree with Paul on the documentation places. Happy for you to merge once you've added that in (without another review) and resolved the other comments to your liking.

Currently we run them when the container is started, but I guess we'd need to apply them manually if postgres is running elsewhere?

I think its fine for us to still run migrations on imaging api going up, because it'll still need to check the migrations have happened anyway.

[stefpiatek]

Probably worth moving this elsewhere in the file as this port isn't necessarily what we're exposing it to (we can override this in the system test files, where we are exposing it).

[stefpiatek]

How about EXTERNAL_PIXL_DB_HOST or CLI_PIXL_DB_HOST etc?

[stefpiatek]

I know it wasn't you but should this move? Feels like its not to do with the export api

[stefpiatek]

Imagine you've seen but from CI: /home/runner/work/PIXL/docker/common.env: no such file or directory. I'm guessing because we've set the context to be ../ then it can just be ./docker...

[tomaroberts]

A note: my last commit removed the env_file: line from test/docker-compose.yml and now the system test is passing. But that's slightly unexpected?

Two thoughts:

1. I think the original problem was due to relative file paths. See here for a similar issue around relative paths and env-file:. I think the context: attribute is only applied to build: rather than the location of the file associated with env-file: attribute.
2. Reading the DC docs, there is some level of inheritance of env variables across compose files – this might explain the tests passing now because a) the env variables already exist and b) by removing the lines in 502f27e we remedy the broken file pathing.

[stefpiatek]

1. think the original problem was due to relative file paths. See here for a similar issue around relative paths and env-file:. I think the context: attribute is only applied to build: rather than the location of the file associated with env-file: attribute.

Yeah I think that's right - IIRC we have to do the test docker compose up --build in two steps because of this