Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] main from open-telemetry:main #29

Merged
merged 34 commits into from
Oct 27, 2023

Conversation

pull[bot]
Copy link

@pull pull bot commented Oct 26, 2023

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

TylerHelmuth and others added 8 commits October 25, 2023 14:48
mulitmod -> multimod

Signed-off-by: Alex Boten <[email protected]>
We see a lot of issues opened that are related to collector usability.
As shared in #27546, documentation can often be added to help clear up
confusion, or provide relevant information. We can use the
`documentation` label for this going forward.

Resolves #27456

---------

Co-authored-by: bryan-aguilar <[email protected]>
It looks like this change was missed in a couple of releases.

Signed-off-by: Alex Boten <[email protected]>
…mplete the transition to container.cpu.utilization (#27795)

**Description:** <Describe what has changed.>

Following up #24183, this PR removes the deprecated
`container.cpu.percent` metric as explained in README's [deprecation
section](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/dockerstatsreceiver#transition-to-cpu-utilization-metric-name-aligned-with-opentelemetry-specification).

**Link to tracking Issue:** #21807

**Testing:** <Describe what testing was performed and which tests were
added.>

```
❯ go test -race -timeout 300s -parallel 4 --tags="" ./... -count 1
ok      github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver  1.480s
ok      github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver/internal/metadata        1.525s
```

**Documentation:** the documentation has been updated using `mdatagen`

---------

Co-authored-by: Pablo Baeyens <[email protected]>
**Description:** <Describe what has changed.>

Add an integration tests to datadog exporter that:
1. Use the mock server in datadog exporter testutil as the backend
2. Starts a collector with otlp receiver, datadog connector, tail
sampling processor, datadog exporter. Have datadog exporter send data to
the mock server
3. Generate and send 10 spans to collector, 5 of which are expected to
be sampled by tail sampling processor
4. Verify the mock server receives 5 traces (sampled) + APM stats on all
10 traces pre-sampled

**Link to tracking Issue:** <Issue number if applicable>

**Testing:** <Describe what testing was performed and which tests were
added.>

**Documentation:** <Describe the documentation added.>
**Description:**

Since each `tailSamplingSpanProcessor`'s instance is not concurrently
called by the ticker worker (it's a 1-to-1 relationship) we can safely
reuse a slice for the tag mutators used in `makeDecision`. Additionally
the tag mutators themselves were causing a lot of allocations and since
they are static, we created constants for them preventing allocations on
each execution of `makeDecision`.

This improved the `makeDecision` benchmark by ~31%.

```
benchstat old.txt new.txt
name         old time/op  new time/op  delta
Sampling-10  51.8µs ± 1%  35.7µs ± 1%  -30.94%  (p=0.008 n=5+5)
```

**Testing:** Unit tests unchanged; added a benchmark

**Documentation:** Perf improvement so no documentation changes needed.

This was all based on production profiling data at Polar Signals running
the collector. Here is a snapshot of the original profiling data we
started with: https://pprof.me/52a7fab/

Judging by the production profiling data, a 31% improvement on the
`makeDecision` codepath, should translate roughly into a 6% baseline CPU
improvement our production deployment of the opentelemetry collector.

The profiling data after improving: https://pprof.me/58c0e84/

This improvement was done as part of the Let's Profile Livestream where
we optimize popular open-source projects live:
https://www.youtube.com/watch?v=vkMQRjiNTHM

---------

Co-authored-by: Jiekun <[email protected]>
@pull pull bot added the ⤵️ pull label Oct 26, 2023
jpkrohling and others added 21 commits October 26, 2023 11:08
Fixes #21045

Signed-off-by: Juraci Paixão Kröhling <[email protected]>

---------

Signed-off-by: Juraci Paixão Kröhling <[email protected]>
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
The Documentation surrounding Azure Event Hub Metric creation is
out-of-date or not accurate.

**Link to tracking Issue:** <Issue number if applicable>

#28622

**Testing:** <Describe what testing was performed and which tests were
added.>
No code was modified, only documentation.

**Documentation:** <Describe the documentation added.>
Updated the README.md in reciever/azureeventhub to be more accurate and
improved the Metric mapping table to be consistent with the Log section.
**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
Allow the Consumer Group to be set in the Configuration.

**Link to tracking Issue:** <Issue number if applicable>
[#28633]

**Testing:** <Describe what testing was performed and which tests were
added.>
Locally tested and verified. No unit testing added.

**Documentation:** <Describe the documentation added.>
New configuration option added to README.
#28637)

Fixed a typo in README.md, in the example:

**Before:**
traces/in:
      receivers: [_**oltp**_]
      exporters: [routing]

**After:**
traces/in:
      receivers: [_**otlp**_]
      exporters: [routing]
See [design
document](https://docs.google.com/document/d/1KtH5atZQUs9Achbce6LiOaJxLbksNJenvgvyKLsJrkc/edit#heading=h.ioikt02qpy5f).

Depends on:

- [Implement ability for extensions to be notified about effective
configuration opentelemetry-collector#6596](open-telemetry/opentelemetry-collector#6596)
- [Make service.instance.id and other telemetry attributes available to
extensions opentelemetry-collector#6599](open-telemetry/opentelemetry-collector#6599)

Closes
#16618

---------

Signed-off-by: Sean Porter <[email protected]>
Co-authored-by: Daniel Jaglowski <[email protected]>
Co-authored-by: Tigran Najaryan <[email protected]>
Do not export function WithTracesMarshalers, WithMetricsMarshalers,
WithLogsMarshalers and pass checkapi

#26304

**Testing:** 
go run cmd/checkapi/main.go .
make chlog-validate
go test for kafkaexporter


Co-authored-by: Alex Boten <[email protected]>
…r readme (#28643)

The `compression` metric option appears more than once in the readme for
the googlecloudexporter exporter, so I've deleted one of the copies. The
first copy has more detail, so I preserved that one.
…emeritus (#28645)

Removed myself as code owner for bearertokenauth and added myself as
emeritus cc: @jpkrohling @frzifus @atoulme atoulme
…tely (#28650)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
When decoding from Azure Resource Log format to OTel if a numeric field
is represented without quotes the data will not be decoded and will fail
silently.

**Link to tracking Issue:** <Issue number if applicable>
#28648 

**Testing:** <Describe what testing was performed and which tests were
added.>
Invalid data captured from Azure Diagnostic Setting and used to create a
unit test. Then data was reprocessed with the Event Hubs Receiver.

**Documentation:** <Describe the documentation added.>
None required as this was a bug within the code.
…27898)

The current behavior consists in exiting early on the first difference
found.

This approach continues to look at all changes, and only exits after
outputting a git diff of all the changes.
…ts (#23637)

**Description:**
Add a test suite to the opentelemetry-contrib repository that checks the
behavior of the HEC exporter against a Splunk Enterprise instance.
**Description:**
We would double-encode JSON objects as strings. We typically have
encoded strings as JSON strings as static responses to issues
encountered during data processing. Along the way, we adopted JSON
objects to match the Splunk Enterprise API. However, we didn't stop
encoding as JSON strings, resulting in double encoded JSON strings.

**Link to tracking Issue:**
Fixes #27604 

**Testing:**
Unit test changes.
## Description: <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
Implementation of exporter to OpenSearch using opensearch-go library. As
of now, this PR was heavily inspired by
https://github.com/dbason/opentelemetry-collector-contrib/tree/opensearch-exporter/exporter/opensearchexporter.

By default, requests sent adhere to the OpenSearch Catalog [schema for
logs](https://github.com/opensearch-project/opensearch-catalog/tree/main/schema/observability/logs),
but allows users to export using the Elastic Common Schema as well.

This PR also:
- enables users to define the `bulk_action` between `create` and `index`
- enables users to define the logs index without necessarily adhering to
the new [index naming
conventions](opensearch-project/observability#1405)
through the `LogsIndex` config.

## Tracking Issue: 


[23611](#23611)

## Testing: <Describe what testing was performed and which tests were
added.>

### Integration
- Successful round-trip to HTTP endpoint,
- Permanent error during round-trip,
- Retryable error response for first request, followed by successful
response on retry,
- Two retriable error responses, followed by successful response on
second retry.

### Manual
- Authentication using `configtls.TLSSetting` (`ca_file`, `cert_file`,
`key_file`)
- Tested in EKS and K3s clusters running
[opni](https://github.com/rancher/opni).

---------

Signed-off-by: João Henri <[email protected]>
Signed-off-by: João Henri <[email protected]>
…in EventXML (Windows) (#28587)

**Description:**
The [XML schema for Windows events supports `Data` elements without the
`Name`
attribute](https://learn.microsoft.com/en-us/windows/win32/wes/eventschema-datafieldtype-complextype),
however, the current implementation doesn't capture `Data` elements
without the `Name` attribute.

Capturing such elements is specially important for events for which the
publisher metadata is invalid. These elements contain the data that will
give a user a much better chance of actually understanding the event,
see
[here](#21491 (comment))
for an example.

I'm adding also the optional `Binary` element. Although this element
typically requires knowledge of the actual data type it is representing
sometimes it can be useful together with the data elements.

I consider this to be a breaking change because it modifies the layout
of the event generated by the package. It isn't an addition, the old
representation is changed, please refer to the changes in tests to see
the difference.

**Link to tracking Issue:**
This is the last pending item to fix #24493, #21491 ([item
5](#21491)).

**Testing:**
- Local run of the affected receiver and package
- "Run Windows" on my fork

**Documentation:**
N/A

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
…via regular expression (#28654)

**Description:**
Allow to compare metrics resource attributes or metric attribute values
by matching on a portion of the dimension value with a regular
expression.

**Link to tracking Issue:**
Fixes #27690

**Testing:**
Unit tests.
**Description:**
Move the golden package to pkg/golden so it can be available to
downstream projects.

**Link to tracking Issue:**
Fixes #28594

---------

Co-authored-by: Daniel Jaglowski <[email protected]>
- Change `knownFiles` to `[]*reader.Metadata`. No files are held here.
- Introduce new `previousPollReaders` list, which holds open files from
the previous poll interval. If for any reason a file is closed, it
should immediately be removed from the list and its metadata should be
added to `knownFiles`.
- Introduce notion that each `reader.Metadata` should be treated as a
singleton. When a reader is closed, it pops out its `Metadata` so it can
be appended to `knownFiles`.
Updating core to bring in the new builder, which no longer uses a
replace in its template
(open-telemetry/opentelemetry-collector#8763)

---------

Signed-off-by: Alex Boten <[email protected]>
atoulme and others added 5 commits October 27, 2023 08:56
git grep points out that this script is not used anywhere.
Introduce OTLP encoding extension.
For more context, see
#27564
There are a number of links in the resource detection processor's README
that are pointing to the old spec repo. This was causing `check-links`
to fail.
@cparkins cparkins merged commit 511b04b into AmadeusITGroup:main Oct 27, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.