55c48b077b
- Add configurable request delay between paginated API calls (default 1.5s) - Add retry with exponential backoff on HTTP 429 and Twitter error code 88 - Add hard max count cap (default 200, absolute ceiling 500) - Add rateLimit config section with requestDelay, maxRetries, retryBaseDelay, maxCount - Add normalization tests for rateLimit config
22 lines
461 B
YAML
22 lines
461 B
YAML
fetch:
|
|
count: 50
|
|
|
|
filter:
|
|
mode: "topN"
|
|
topN: 20
|
|
minScore: 50
|
|
lang: []
|
|
excludeRetweets: false
|
|
weights:
|
|
likes: 1.0
|
|
retweets: 3.0
|
|
replies: 2.0
|
|
bookmarks: 5.0
|
|
views_log: 0.5
|
|
|
|
rateLimit:
|
|
requestDelay: 1.5 # seconds between paginated requests
|
|
maxRetries: 3 # retry count on 429 / rate-limit errors
|
|
retryBaseDelay: 5.0 # base delay for exponential backoff (seconds)
|
|
maxCount: 200 # hard cap for single fetch
|