You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I should be possible that the HTTPOperator retries some of the request without failing the whole task, since when it is using pagination, some pages might fail.
Use case/motivation
I'm requesting all the pages from an API endpoint (TeamTailor JobPosts) and that is a total of 438 pages.
Sometimes while going through these pages the server returns error 500.
If that page in particular would be retried with exponential backoff the whole task would probably succeed, but retrying the whole task might reproduce the problem.
Description
I should be possible that the HTTPOperator retries some of the request without failing the whole task, since when it is using pagination, some pages might fail.
Use case/motivation
I'm requesting all the pages from an API endpoint (TeamTailor JobPosts) and that is a total of 438 pages.
Sometimes while going through these pages the server returns error 500.
If that page in particular would be retried with exponential backoff the whole task would probably succeed, but retrying the whole task might reproduce the problem.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: