adapter,controller: add replica metrics history #29254
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 tasks
teskje
force-pushed
the
mz_cluster_replica_metrics_history
branch
2 times, most recently
from
8efe34a to
eab0341
Compare
teskje
changed the title
teskje
force-pushed
the
mz_cluster_replica_metrics_history
branch
2 times, most recently
from
dd399ae to
5f544d7
Compare
5 tasks
teskje
force-pushed
the
mz_cluster_replica_metrics_history
branch
from
5f544d7 to
ec750bf
Compare
This commit adds a new builtin source,
`mz_cluster_replica_metrics_history`, and a corresponding view
`mz_cluster_replica_utilization_history` on top of it. They contain the
data already present in `mz_cluster_replica_{metrics,utilization}`, but
as (pseudo-) append-only collections, rather than retained-metrics
collections.
This plan is to remove the old retained-metrics collections in favor of
the new append-only ones, but we need to wait until the new ones had
time to backfill the historical data present in the old ones.
This commit does not include any code to partially truncate
`mz_cluster_replica_metrics_history`, which is something we want to
ensure it doesn't grow unboundedly. A subsequent commit will deal with
that.
This commit adds truncation of the replica metrics history collection during bootstrap. In contrast to the existing status history collections, this collection is truncated based on the age of events instead of the number of events per ID. The implementation of the truncation method is kept simple. In the future it can be refined to avoid reading the entire collection into memory, if it turns out that the size of a metrics collection is problematic.
teskje
force-pushed
the
mz_cluster_replica_metrics_history
branch
from
ec750bf to
deeaca1
Compare
teskje
commented
Sep 1, 2024
| ScalarType::TimestampTz { precision: None }.nullable(false), | ||
| ) | ||
| .finish() | ||
| }); |
There was a problem hiding this comment.
This doesn't really fit into healthcheck.rs, but we already have the statement execution history stuff here.
aljoscha
approved these changes
Sep 3, 2024
|
TFTRs! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new builtin source
mz_cluster_replica_metrics_history, and a corresponding viewmz_cluster_replica_utilization_historyon top of it. They contain the data already present inmz_cluster_replica_{metrics,utilization}, but as (pseudo-) append-only collections, rather than retained-metrics collections.This plan is to remove the old retained-metrics collections in favor of the new append-only ones, but we need to wait until the new ones had time to backfill the historical data present in the old ones.
Motivation
Part of https://github.com/MaterializeInc/database-issues/issues/8403
Tips for reviewer
Some of our user docs (Troubleshooting, Grafana, Datadog) refer to
mz_cluster_replica_utilization. I'm not touching these here, but we'll need to change them when/before we remove that relation.Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.