feat: add rate limiting, retry with backoff, and max count cap
- Add configurable request delay between paginated API calls (default 1.5s) - Add retry with exponential backoff on HTTP 429 and Twitter error code 88 - Add hard max count cap (default 200, absolute ceiling 500) - Add rateLimit config section with requestDelay, maxRetries, retryBaseDelay, maxCount - Add normalization tests for rateLimit config
This commit is contained in:
@@ -13,3 +13,9 @@ filter:
|
||||
replies: 2.0
|
||||
bookmarks: 5.0
|
||||
views_log: 0.5
|
||||
|
||||
rateLimit:
|
||||
requestDelay: 1.5 # seconds between paginated requests
|
||||
maxRetries: 3 # retry count on 429 / rate-limit errors
|
||||
retryBaseDelay: 5.0 # base delay for exponential backoff (seconds)
|
||||
maxCount: 200 # hard cap for single fetch
|
||||
|
||||
Reference in New Issue
Block a user