Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronize deletes #102

Open
1 task
hackartisan opened this issue Sep 26, 2024 · 0 comments
Open
1 task

Synchronize deletes #102

hackartisan opened this issue Sep 26, 2024 · 0 comments

Comments

@hackartisan
Copy link
Member

hackartisan commented Sep 26, 2024

If hydrator encounters a deletion marker for an indexed record, the existing record should be deleted from the solr index.

Acceptance Criteria

  • When an EphemeraFolder record is deleted from Figgy, it should delete itself from the DPUL Collections solr index.

Technical Notes

One way this could happen is the following:

  1. A deletion marker found by the HydrationConsumer is written to the HydrationCache with the record_id set as the DeletionMarker's resource_id
  2. The TransformationConsumer, when it finds a HydrationCacheEntry for a DeletionMarker, writes a TransformationCacheEntry with the data {id: <record_id>, deleted: true}
  3. The IndexingConsumer, when it finds an entry with deleted equal to true, will delete that record from the index. Maybe it should batch those up, actually - set a batcher in handle_message and then have the deletion batcher delete every message it gets in one query, if that's possible.

First step

Find a way to write a test.

@hackartisan hackartisan added this to the Stable Synchronization milestone Oct 23, 2024
@hackartisan hackartisan changed the title If hydrator encouters a deletion marker for an indexed record, the existing record should be deleted from the solr index. Synchronize deletes Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant