Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] adapter,controller: mz_cluster_replica_metrics source #29238

Closed

Conversation

teskje
Copy link
Contributor

@teskje teskje commented Aug 27, 2024

This PR converts mz_cluster_replica_metrics from a "retained metrics" builtin table to an append-only builtin source. The coordinator no longer needs to be involved in writing metrics data to storage, this happens now in the controller.

TODO

  • partial truncation
  • console adjustments

Motivation

  • This PR adds a known-desirable feature.

Part of https://github.com/MaterializeInc/database-issues/issues/8403

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

teskje added 3 commits August 27, 2024 15:56
This commit converts `mz_cluster_replica_metrics` from a "retained
metrics" builtin table to an append-only builtin source. The coordinator
no longer needs to be involved in writing metrics data to storage, this
can now happen in the controller.

This commit does not include any code to partially truncate
`mz_cluster_replica_metrics`, which is something we want to ensure it
doesn't grow unboundedly. A subsequent commit will deal with that.
This commit removes support for background table updates, as it is now
unused. This simplifies the code a bit.
This commit adapts the tests to the changes made to the
`mz_cluster_replica_metrics` system relation:

 * It's no longer a table but a source.
 * It has an extra column with the event time.
 * It is a log rather than a snapshot of the most recent state.

The `statement-logging` cluster test runs into #29237 when querying the
new `mz_cluster_replica_metrics` source, so that query has been disabled
until the bug is fixed.
@teskje teskje force-pushed the mz_cluster_replica_metrics-log branch from 351982c to 9f5827c Compare August 27, 2024 14:01
@teskje
Copy link
Contributor Author

teskje commented Aug 28, 2024

Closing in favor of #29254.

@teskje teskje closed this Aug 28, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Aug 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant