Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly purge state groups tables when purging a room #18024

Merged

Conversation

MatMaul
Copy link
Contributor

@MatMaul MatMaul commented Dec 10, 2024

Currently purging a complex room can lead to a lot of orphaned rows left behind in the state groups tables.
It seems it is because we are loosing track of state groups sometimes.

This change uses the room_id indexed column of state_groups table to decide what to delete instead of doing an indirection through event_to_state_groups.

Related to #3364.

Pull Request Checklist

@MatMaul MatMaul force-pushed the fix-purge-room-state-groups branch 2 times, most recently from 81cadd8 to 7945610 Compare December 10, 2024 16:06
@MatMaul MatMaul marked this pull request as ready for review December 10, 2024 16:30
@MatMaul MatMaul requested a review from a team as a code owner December 10, 2024 16:30
@MatMaul MatMaul requested a review from erikjohnston December 18, 2024 23:37
Copy link
Member

@erikjohnston erikjohnston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

synapse/storage/databases/state/store.py Outdated Show resolved Hide resolved
synapse/storage/databases/state/store.py Outdated Show resolved Hide resolved
Copy link
Member

@erikjohnston erikjohnston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, bah I can't edit the PR. Can you commit the simplifications to the delete?

@MatMaul MatMaul requested a review from erikjohnston December 29, 2024 13:19
Copy link
Member

@erikjohnston erikjohnston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woo, thanks!

@erikjohnston
Copy link
Member

@MatMaul sorry, looks like the tests didn't trigger, could you push an empty commit or something?

@erikjohnston
Copy link
Member

Ooooh, there are conflicts, sorry!

@MatMaul MatMaul force-pushed the fix-purge-room-state-groups branch from 0875f1c to aab137d Compare January 6, 2025 14:13
@MatMaul
Copy link
Contributor Author

MatMaul commented Jan 6, 2025

Argh sorry, self inflicted wound (the other conflicting PR is mine 😅 ).

@MatMaul MatMaul requested a review from erikjohnston January 6, 2025 14:14
@erikjohnston erikjohnston enabled auto-merge (squash) January 6, 2025 14:30
@erikjohnston erikjohnston merged commit b3ba501 into element-hq:develop Jan 6, 2025
39 checks passed
@olmari
Copy link
Contributor

olmari commented Jan 7, 2025

Ooh, does this mean the https://github.com/erikjohnston/synapse-find-unreferenced-state-groups is getting less in the future?

yingziwu added a commit to yingziwu/synapse that referenced this pull request Jan 16, 2025
Please note that this version of Synapse drops support for PostgreSQL 11 and 12. The minimum version of PostgreSQL supported is now version 13.

No significant changes since 1.122.0rc1.

- Remove support for PostgreSQL 11 and 12. Contributed by @clokep. ([\#18034](element-hq/synapse#18034))

- Added the `email.tlsname` config option.  This allows specifying the domain name used to validate the SMTP server's TLS certificate separately from the `email.smtp_host` to connect to. ([\#17849](element-hq/synapse#17849))
- Module developers will have access to the user ID of the requester when adding `check_username_for_spam` callbacks to `spam_checker_module_callbacks`. Contributed by [email protected]. ([\#17916](element-hq/synapse#17916))
- Add endpoints to the Admin API to fetch the number of invites the provided user has sent after a given timestamp,
  fetch the number of rooms the provided user has joined after a given timestamp, and get report IDs of event
  reports against a provided user (i.e. where the user was the sender of the reported event). ([\#17948](element-hq/synapse#17948))
- Support stable account suspension from [MSC3823](matrix-org/matrix-spec-proposals#3823). ([\#17964](element-hq/synapse#17964))
- Add `macaroon_secret_key_path` config option. ([\#17983](element-hq/synapse#17983))

- Fix bug when rejecting withdrew invite with a `third_party_rules` module, where the invite would be stuck for the client. ([\#17930](element-hq/synapse#17930))
- Properly purge state groups tables when purging a room with the Admin API. ([\#18024](element-hq/synapse#18024))
- Fix a bug preventing the admin redaction endpoint from working on messages from remote users. ([\#18029](element-hq/synapse#18029), [\#18043](element-hq/synapse#18043))

- Update `synapse.app.generic_worker` documentation to only recommend `GET` requests for stream writer routes by default, unless the worker is also configured as a stream writer. Contributed by @evoL. ([\#17954](element-hq/synapse#17954))
- Add documentation for the previously-undocumented `last_seen_ts` query parameter to the query user Admin API. ([\#17976](element-hq/synapse#17976))
- Improve documentation for the `TaskScheduler` class. ([\#17992](element-hq/synapse#17992))
- Fix example in reverse proxy docs to include server port. ([\#17994](element-hq/synapse#17994))
- Update Alpine Linux Synapse Package Maintainer within the installation instructions. ([\#17846](element-hq/synapse#17846))

- Add `RoomID` & `EventID` rust types. ([\#17996](element-hq/synapse#17996))
- Fix various type errors across the codebase. ([\#17998](element-hq/synapse#17998))
- Disable DB statement timeout when doing a room purge since it can be quite long. ([\#18017](element-hq/synapse#18017))
- Remove some remaining uses of `twisted.internet.defer.returnValue`. Contributed by Colin Watson. ([\#18020](element-hq/synapse#18020))
- Refactor `get_profile` to no longer include fields with a value of `None`. ([\#18063](element-hq/synapse#18063))

* Bump anyhow from 1.0.93 to 1.0.95. ([\#18012](element-hq/synapse#18012), [\#18045](element-hq/synapse#18045))
* Bump authlib from 1.3.2 to 1.4.0. ([\#18048](element-hq/synapse#18048))
* Bump dawidd6/action-download-artifact from 6 to 7. ([\#17981](element-hq/synapse#17981))
* Bump http from 1.1.0 to 1.2.0. ([\#18013](element-hq/synapse#18013))
- Bump mypy from 1.11.2 to 1.12.1. ([\#17999](element-hq/synapse#17999))
* Bump mypy-zope from 1.0.8 to 1.0.9. ([\#18047](element-hq/synapse#18047))
* Bump pillow from 10.4.0 to 11.0.0. ([\#18015](element-hq/synapse#18015))
* Bump pydantic from 2.9.2 to 2.10.3. ([\#18014](element-hq/synapse#18014))
* Bump pyicu from 2.13.1 to 2.14. ([\#18060](element-hq/synapse#18060))
* Bump pyo3 from 0.23.2 to 0.23.3. ([\#18001](element-hq/synapse#18001))
* Bump python-multipart from 0.0.16 to 0.0.18. ([\#17985](element-hq/synapse#17985))
* Bump sentry-sdk from 2.17.0 to 2.19.2. ([\#18061](element-hq/synapse#18061))
* Bump serde from 1.0.215 to 1.0.217. ([\#18031](element-hq/synapse#18031), [\#18059](element-hq/synapse#18059))
* Bump serde_json from 1.0.133 to 1.0.134. ([\#18044](element-hq/synapse#18044))
* Bump twine from 5.1.1 to 6.0.1. ([\#18049](element-hq/synapse#18049))

**Changelogs for older versions can be found [here](docs/changelogs/).**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants