Default to a 1 second backoff when hitting 429s #142
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pipt
reviewed
Oct 17, 2024
pipt
approved these changes
Oct 17, 2024
When importing a large amount of new entries, or deleting them we're hitting against the general API limit. Using a pool improves our performance but we still seem to see the odd failure where the 10 routines are fighting one another enough that we fail after X retries (tried up to 10 and still saw issues). The big fix that worked here was to ensure we backoff for a minimum of 1 second at least (anything smaller, or even negative means we still do 1 now). This results in the rate we submit results peaking at several hundred requests per second until we hit our per min API rate limit, then we are at the mercy of the token bucket refill - essentially doing max we can as our rate limit refills. This slows down a little, but it's in theory the best we can do per the rate limit and we're a better citizen talking to the API.
rliddler
force-pushed
the
rob/handle-backoff-retries-better
branch
from
b5f9801 to
e91b80a
Compare
zetxek
reviewed
Oct 17, 2024
| @@ -14,7 +14,7 @@ import ( | |||
| "github.com/pkg/errors" | |||
| ) | |||
|
|
|||
| const maxRetries = 3 | |||
There was a problem hiding this comment.
could it be an idea to be able to customize this via command line arguments? 💡
There was a problem hiding this comment.
Ah yeah good shout! I just whacked it up for now as it didn't seem harmful to just try a bit more 😄
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When importing a large amount of new entries, or deleting them we're hitting against the general API limit.
Using a pool improves our performance but we still seem to see the odd failure where the 10 routines are fighting one another enough that we fail after X retries (tried up to 10 and still saw issues).
The big fix that worked here was to ensure we backoff for a minimum of 1 second at least (anything smaller, or even negative means we still do 1 now).
This results in the rate we submit results peaking at several hundred requests per second until we hit our per min API rate limit, then we are at the mercy of the token bucket refill - essentially doing max we can as our rate limit refills.
This slows down a little, but it's in theory the best we can do per the rate limit and we're a better citizen talking to the API.