Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(host_metrics source): Add additional cgroup memory metrics #22153

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nionata
Copy link
Contributor

@nionata nionata commented Jan 10, 2025

Summary

Add active and inactive metrics for anon and file memory to the cgroup collector. These additional metrics allow you to better understand the existing cgroup memory metrics. The new metrics follow reverse domain name notation. I opted to do this over keep the ordering in the memory.stat file to increase querying ergonomics.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

cargo fmt

cargo clippy --all-targets
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.87s

cargo test
test result: ok. 1628 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 21.40s

     Running tests/e2e/mod.rs (target/debug/deps/e2e-9fbb4963c350fe5f)

./scripts/check_changelog_fragments.sh
validating 'add_active_and_inactive_cgroup_memory_metrics.feature.md'
changelog additions are valid.
make environment CONTAINER_TOOL="podman"

/git/vectordotdev/vector# mkdir /etc/vector && cat << EOF > /etc/vector/vector.yaml
> sources:
>   in:
>     type: "host_metrics"
>     collectors:
>       - "cgroups"
>     cgroups:
>       base: ""
> 
> sinks:
>   out:
>     inputs:
>       - "in"
>     type: "console"
>     encoding:
>       codec: "native_json"
> EOF

/git/vectordotdev/vector# cargo run | jq '.metric.name, .metric.gauge.value'
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/vector`
2025-01-10T01:54:02.356851Z  INFO vector::app: Log level is enabled. level="info"
2025-01-10T01:54:02.359020Z  INFO vector::app: Loading configs. paths=["/etc/vector/vector.yaml"]
2025-01-10T01:54:02.364879Z  INFO source{component_kind="source" component_id=in component_type=host_metrics}: vector::sources::host_metrics: PROCFS_ROOT is unset. Using default '/proc' for procfs root.
2025-01-10T01:54:02.364942Z  INFO source{component_kind="source" component_id=in component_type=host_metrics}: vector::sources::host_metrics: SYSFS_ROOT is unset. Using default '/sys' for sysfs root.
2025-01-10T01:54:02.365956Z  INFO vector::topology::running: Running healthchecks.
2025-01-10T01:54:02.366216Z  INFO vector::topology::builder: Healthcheck passed.
2025-01-10T01:54:02.366282Z  INFO vector: Vector has started. debug="true" version="0.44.0" arch="x86_64" revision=""
2025-01-10T01:54:02.366326Z  INFO vector::app: API is disabled, enable by setting `api.enabled` to `true` and use commands like `vector top`.
"cgroup_cpu_usage_seconds_total"
null
"cgroup_cpu_user_seconds_total"
null
"cgroup_cpu_system_seconds_total"
null
"cgroup_memory_current_bytes"
11895951360
"cgroup_memory_anon_bytes"
25882624
"cgroup_memory_file_bytes"
11654889472
"cgroup_memory_anon_active_bytes"
20480
"cgroup_memory_anon_inactive_bytes"
25440256
"cgroup_memory_file_active_bytes"
542707712
"cgroup_memory_file_inactive_bytes"
11112181760

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

  • Please read our Vector contributor resources.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

https://docs.kernel.org/admin-guide/cgroup-v2.html

@nionata nionata requested a review from a team as a code owner January 10, 2025 00:56
@nionata nionata changed the title feat(host_metrics source): Add additional cgroup memory stats feat(host_metrics source): Add additional cgroup memory metrics Jan 10, 2025
@github-actions github-actions bot added the domain: sources Anything related to the Vector's sources label Jan 10, 2025
@nionata nionata force-pushed the nji/cgroup-mem-stat branch 2 times, most recently from ccb0a46 to 457ff2e Compare January 10, 2025 01:31
@nionata nionata requested review from a team as code owners January 10, 2025 01:31
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Jan 10, 2025
@nionata nionata force-pushed the nji/cgroup-mem-stat branch from 457ff2e to 4d40376 Compare January 10, 2025 01:33
@nionata nionata force-pushed the nji/cgroup-mem-stat branch from 4d40376 to d961b21 Compare January 10, 2025 01:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: external docs Anything related to Vector's external, public documentation domain: sources Anything related to the Vector's sources
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant