[otlp grpc] Handle retry in case of DEADLINE_EXCEEDED response code

[vishweshbankwar]

Spec suggests that DEADLINE_EXCEEDED is a retryable status code.

The current retry logic for otlp exporters would not retry any errors that would extend the export duration beyond the configured timeout period. This timeout is used as the deadline for a single grpc request that is sent to the server by the export client. In other words, the timeout period would be exhauted by a single grpc request and the request would not be retried (Not following the spec).

Opening this issue to further investigate how to handle DEADLINE_EXCEEDED response code in grpc.

[github-actions]

This issue was marked stale due to lack of activity and will be closed in 7 days. Commenting will instruct the bot to automatically remove the label. This bot runs once per day.

[Kielek]

Adding to Future as it seems to be not following spec.

[rajkumar-rangaraj]

After reviewing the implementation, I can confirm that the DEADLINE_EXCEEDED error is already handled in the code. The relevant handling can be found in the following section:
OtlpRetry.cs#L212.