Modifying the database schema and/or entries in-place

[JoanneBogart]

The procedure might go like this:

1. Write a script (preferably in python but may use raw sql if necessary) to make needed modifications to one or both of schema and entries. If schema is modified, the script should also add a new entry to Provenance including updated version info for the schema. The script should incorporate old and new db schema semantic version, e.g. migrate_1.2.3_to_1.3.0.py
2. Practice on a test schema (e.g., dump the real production or working schema and restore to a different schema name). Revise script if necessary and try again until results are satisfactory
3. If there are schema changes, on a branch update schema.yaml and version fields in create_registry_schema.py. Check that schema creation works properly.
4. If older client code does not work with the upgraded database some sort of warning should be issued. There could be different constraints depending on whether client is attempting to read or write. Most updates will be backwards compatible for readers. How do we keep track of this?
5. Run script on working and production schemas (should we lock people out while this is happening?)
6. If there were schema changes, merge in branch (and make code release?)

[JoanneBogart]

When inserting a new entry in the Provenance table, what is the difference between update methods 'MODIFY' and 'MIGRATE'?
Should _insert_provenance behave differently depending on update method? E.g., use old value of associated_production in the new row for anything other than 'CREATE'?