Index version management from interface

[lukavdplas]

#1004 made me realise that we have versioned index names in production, so indexing corpora from the interface (#985) should account for version management.

The minimal implementation would be that curators do not see the different versions at all, and the application just quietly updates the alias. It would be good to have an environment setting for production (i.e. versioned index names and aliases) vs. development (none of that, clear and overwrite the index with every update) - I don't think this ever needs to be set per corpus or, as we do now, per indexing action.

In this case, the older index versions could be useful when a curator contacts us about an issue. Still, it would be nice if they could restore older versions themselves. This would save unnecessary duplicates.

The indexing menu (step 3 in #982) could show a list of all indices matching the corpus name, with the option to delete inactive indices, or switch which version is currently active.

However, old indices may not be compatible with the current corpus definition. Ideally, the application will save a "snapshot" of the corpus at the time of indexing (doable with the export option #981), and restoring an old index also means restoring the (relevant) corpus settings.

[lukavdplas]

These issues cover preparations to database models, etc.

• Represent elasticsearch index in postgres database #1695
• IndexJob model #1696
• IndexJob lifecycle #1697

When these are done, we can create an API and an interface in the frontend. This should be much more streamlined; proper index management requires admin privileges.

@JeltevanBoheemen and I discussed this and made a rough outline. We're envisioning these functions for the user:

• Upload additional CSV files. The user has already uploaded one CSV file, but this may only be a sample. (Also, they may add more data later, break up a dataset into multiple files, etc.)
• Create a new index for the corpus and activate it. The user will be asked to select one or more files from their CSV uploads. These will be the source files. The backend should now (in order):
  • Create a new index in Elasticsearch
  • Extract data from the selected files and populate the index
  • Link the index to the corpus
  • If the corpus had existing indices, delete them
  • Send the user an email that their index is ready
• If the corpus is already indexed: delete the index.
• Toggle the corpus as private/public.

If a corpus has an index, other steps in the form will still be available, but fields that affect the index will be marked with a warning sign. (Or something like that.) When the user changes those fields and hits save, they'll get a confirmation window. The existing index will become invalid, and they'll have to create a new index, or undo their changes, before they can make the corpus public again.

Notes:

• Users cannot manage multiple index versions; they'll always use the latest.
• Deleting indices is a good idea for many reasons, but we should watch out that this won't result in situations where heavyweights like times get deleted by accident 🙃