-
Notifications
You must be signed in to change notification settings - Fork 399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Add retries to the internal httpx.Client
used by the SDK
#5386
Conversation
49484dc
to
b5acc70
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielmbmb should we define upon which failures we retry. I think it only makes sense for the 500 series and perhaps on request timeout or anything. https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses
if payload is invalid or something, we don't want to retry.
self.api_url = self.api_url | ||
self.api_key = self.api_key | ||
self.timeout = self.timeout or 60 | ||
timeout: int = 60 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielmbmb I think we can add this to the constants.py file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gabrielmbmb The refactor of timeout
and retries
looks good to me. It's more consistent and avoids background passing with **client_args
.
@davidberenstein1957 A few PRs have used constants in context lately. I think we should tackle this in a dedicated PR because using a single constant file is not universally good.
@gabrielmbmb Implementation looks good. What do you think about expanding the testing for this? I thought of these two but open to your suggestions:
|
@frascuchon @davidberenstein1957 What's your view of this PR? I've added a test and the remaining comments are: |
I think there are several error that can occur and will never be fixed by retries, like authentication or wrong payload, so for me it doesn't make sense to retry during those cases. W.r.t. variables vs shared constants, I don't have a strict opinion besides having everything else in |
Co-authored-by: Paco Aranda <[email protected]>
Description
This PR adds a new argument
retries
that can be used to specify the number of times that an HTTP request performed by the internalhttpx.Client
used by the SDK should be retried before raising an exception.This is useful as sometimes while using the
dataset.records.log
you can receive aConnectionError
or 5xx from the server and if you retry a few seconds later everything is fine.Type of change
How Has This Been Tested
Checklist