Commit Graph

47 Commits

Author SHA1 Message Date
kshift
7c634e0d39 Add list cursor pagination (#56) 2026-05-07 23:53:22 +08:00
nemo
1f3a9ee535 fix: adapt to Twitter API schema changes (April 2026) (#51)
- Migrate user profile fields from `legacy{}` to `core{}` / `avatar{}` / `location{}` (with legacy fallback for older response shapes). Affects `user`, `whoami`, `status`, `followers`, `following`.
- Fix `user-posts` empty results: add missing `includePromotedContent` variable and update `instructions` path from `timeline_v2` to `timeline`.
- Switch `followers` / `following` to POST (Twitter changed the requirement).
- Refresh 16 stale `FALLBACK_QUERY_IDS` from fa0311/twitter-openapi.
- Drop `legacy{}` early-return in `parse_user_result`; key existence on `rest_id` so followers/following keep working when Twitter fully drops legacy.
- Add unit tests for the new core/avatar/location fallback chain and rest_id/typename guards.
2026-04-29 21:20:39 +08:00
Lucius
7816f8d813 Feed cursor pagination (#49)
* Expose promoted tweets in feed output

* Add cursor-based feed pagination output
2026-04-10 01:20:18 +08:00
jackwener
199a1490f9 fix: SearchTimeline POST + ondemand URL guard + refresh queryIds
Twitter migrated SearchTimeline from GET to POST; update fetch_search
to route through _graphql_post via a new use_post flag on _fetch_timeline,
avoiding duplicating the pagination logic (closes #39, refs #40, #42).

Also guard against get_ondemand_file_url returning None so the error
message is clear instead of crashing on NoneType.split (closes #43-adjacent).

Refresh all FALLBACK_QUERY_IDS from live JS bundles — 18 operations updated.
2026-03-20 18:21:29 +08:00
benny b
ffd2a42f7c feat: add bookmark folders support (#30)
Add `twitter bookmarks folders` command to list bookmark folders
and `twitter bookmarks folders <id>` to fetch tweets from a specific
folder, with --since date filtering and pagination support.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 17:59:08 +08:00
jackwener
e0f38cbbb1 fix(ci): restore FALLBACK_QUERY_IDS import, remove unused patch import 2026-03-14 13:32:28 +08:00
jackwener
ec4589c2d1 fix: P0 Windows Edge path, add time localization, show --output, cleanup tech debt
- Fix auth.py subprocess script Windows Edge cookie path inconsistency
- Add timeutil.py for UTC→local time and relative time conversion
- Integrate time localization into formatter.py and serialization.py
- Add --output/-o option to show command for saving tweet detail as JSON
- Remove constants.py legacy aliases (USER_AGENT, SEC_CH_UA)
- Remove client.py backward-compat delegation methods and re-exports
- Update test imports to use parser module directly
2026-03-14 13:26:36 +08:00
jackwener
69cb85a1c2 feat: add image upload support for post/reply/quote commands
- Add upload_media() method to TwitterClient (INIT/APPEND/FINALIZE flow
  via upload.twitter.com, supports JPEG/PNG/GIF/WebP up to 5MB)
- Extend create_tweet() and quote_tweet() with optional media_ids param
- Add --image/-i option to post, reply, and quote CLI commands (max 4)
- Add MediaUploadError exception for upload-specific error handling
- Add 6 unit tests covering upload flow, validation, and media_ids

Co-Authored-By: Catafal <67582323+Catafal@users.noreply.github.com>
2026-03-13 01:59:42 +08:00
jackwener
7d1b519c85 fix: harden search validation and release v0.7.1 2026-03-13 01:04:29 +08:00
jackwener
00d18686dd fix: remove unused client import 2026-03-12 14:48:35 +08:00
jakevin
79eadd2579 feat: add twitter article markdown command (#16) 2026-03-12 14:47:49 +08:00
jackwener
88a9f4ce97 fix: tighten pagination and platform-specific auth 2026-03-11 20:32:51 +08:00
jackwener
1de88ea2ed fix: update Likes queryId and response path (fixes #8)
- Update Likes fallback queryId: aeJWz7GtGNHHO2Z3GrjCWg -> dv5-II7_Bup_PHish7p6fw
- Fix response path: data.user.result.timeline.timeline.instructions
  (Twitter renamed timeline_v2 to timeline; code now tries both)
- Verified via Playwright: API returns 200 with correct tweet entries

Fixes #8
2026-03-11 17:08:14 +08:00
aidiff-kobe
f01ce77f20 refactor: Use specific ImportError instead of broad Exception (#10)
Changed the exception handling in _best_chrome_target() to catch
specific ImportError instead of broad Exception. This improves
code clarity and avoids masking unexpected errors.

Also added a debug log message to help diagnose when curl_cffi
is not available.

Co-authored-by: Security Bot <agent@example.com>
2026-03-11 12:36:13 +08:00
jackwener
1f267008ad fix: update stale Followers/Following queryIds and retry on 422
Twitter now returns HTTP 422 GRAPHQL_VALIDATION_FAILED (not just 404)
when a queryId goes stale. Updated fallback IDs and added 422 to the
stale-queryId retry logic in both _graphql_get and _graphql_post.
2026-03-11 00:45:13 +08:00
jackwener
5c1015f1fd fix: handle changed /account/multi/list.json response format
Twitter changed the response format from a list with nested 'user'
objects to {"users": [{user_id, name, screen_name, ...}]} with
minimal fields. Now extracts screen_name from the new format and
fetches the full profile via GraphQL UserByScreenName endpoint.
2026-03-11 00:40:07 +08:00
jackwener
f125d0fe7f refactor: replace all RuntimeError with specific exception types
- 5x RuntimeError → TwitterAPIError (create_tweet, fetch_me, quote_tweet, follow/unfollow)
- 2x RuntimeError(str(exc)) → bare raise (preserve original TwitterAPIError in _graphql_get/post)
2026-03-10 23:32:10 +08:00
jackwener
c2b9be4669 refactor: split client.py into graphql.py + parser.py modules
Split the monolithic client.py (1341 lines) into three focused modules:

- graphql.py (~200 lines): queryId resolution, URL building, JS bundle
  scanning, feature flag management
- parser.py (~270 lines): Tweet/User/Media/Article parsing, utility functions
  (_deep_get, _parse_int, _extract_cursor, _extract_media)
- client.py (~700 lines): TwitterClient class with HTTP engine, anti-detection,
  session management, and all public API methods

Backward compatibility: client.py re-exports all previously public symbols
so existing test imports work without modification. 88/88 tests pass.
2026-03-10 23:18:59 +08:00
jackwener
4afc4fc246 refactor: add exceptions.py module with structured exception hierarchy
- Create exceptions.py with 7 exception types: TwitterError, AuthenticationError,
  RateLimitError, NotFoundError, NetworkError, QueryIdError, TwitterAPIError
- Remove inline TwitterAPIError from client.py, import from exceptions module
- Replace RuntimeError('Cannot resolve queryId') with QueryIdError
- Replace RuntimeError('User not found') with NotFoundError
- Update test assertion for new TwitterAPIError message format
2026-03-10 23:05:05 +08:00
jackwener
9cf74abd56 feat: add integration smoke tests
CLI-level smoke tests using --yaml output against real Twitter API.
Default skipped via @pytest.mark.smoke marker + pyproject.toml addopts.
Run locally with: uv run pytest -m smoke -v
2026-03-10 22:26:46 +08:00
jackwener
642ffe84a8 feat: unify agent status schema 2026-03-10 21:02:08 +08:00
jackwener
32d074dc9f feat: anti-detection hardening, transaction cache, article parsing, structured write output
Anti-detection:
- Add 6 sec-ch-ua-* Client Hints headers (arch, bitness, full-version, etc.)
- POST requests now send Referer: x.com/compose/post + Priority: u=1, i
- follow/unfollow REST adds include_profile_interstitial_type param

Performance:
- Transaction ID cache with 1h TTL (~/.twitter-cli/transaction_cache.json)
- resolve_user_id: auto-detect screen_name vs numeric user_id

Features:
- Twitter Article parsing: extract long-form content as Markdown
- Write operations emit structured JSON/YAML when piped or OUTPUT env set
  ActionResult: {success, action, id, url, ...}

84 tests passing
2026-03-10 20:48:42 +08:00
jackwener
49d3e237c4 feat: add whoami, reply, quote, follow/unfollow commands and --compact mode
- whoami: fetch current authenticated user profile
- reply <id> <text>: standalone reply command
- quote <id> <text>: quote-tweet command
- follow/unfollow <handle>: follow/unfollow users
- --compact/-c: global flag for LLM-friendly minimal JSON output
- client.py: add fetch_me, quote_tweet, follow_user, unfollow_user
- serialization.py: add tweet_to_compact_dict, tweets_to_compact_json
- 7 new tests (82 total, all passing)
2026-03-10 20:09:08 +08:00
jackwener
d71ad45a0a fix: harden pagination auth and runtime headers 2026-03-10 12:33:04 +08:00
jackwener
8313a7012f refactor: dynamic UA matching, session reuse, score Optional, --output on all commands
- constants.py: sync_chrome_version() aligns UA/sec-ch-ua with impersonate target
- auth.py: reuse shared cffi session instead of creating duplicate
- filter.py: eliminate double weight building in score_tweet
- models.py: Tweet.score → Optional[float] for accurate display
- cli.py: add --output to search/likes/user-posts for consistency
2026-03-09 21:15:28 +08:00
jackwener
fda9b1c3dc fix: 431 Request Too Large — reduce FEATURES to 15 essential keys, dynamic update only updates existing keys 2026-03-09 20:59:16 +08:00
jackwener
d20c5699fd fix: 414 URI Too Long — omit False-valued features from GET URL, add regression tests 2026-03-09 20:50:45 +08:00
jackwener
12f425abea feat: write operation delays, dynamic FEATURES update, 30+ client.py tests, fix README proxy wording 2026-03-09 20:45:51 +08:00
jackwener
b9c226b804 fix: dynamic Chrome impersonation + in-process macOS Keychain cookie extraction (closes #4) 2026-03-09 20:38:47 +08:00
jackwener
59b5df7f71 fix: ClawHub publish (acceptLicenseTerms), add LICENSE, fix score display bug, remove unused var 2026-03-09 20:32:45 +08:00
jackwener
94e21fba9a fix: remove unused requests import (CI lint), fix README wording, drop requests dep 2026-03-09 20:25:24 +08:00
jackwener
a6ad246071 fix: improve error handling for write operation rate limits (retweet/like) 2026-03-09 20:22:46 +08:00
jackwener
731151e62e fix: use curl_cffi for ClientTransaction init to prevent TLS fingerprint leak 2026-03-09 19:14:33 +08:00
jackwener
b83abadb73 feat: full cookie forwarding from browser and TWITTER_PROXY support 2026-03-09 19:12:06 +08:00
jackwener
27d73efee5 feat: anti-detection hardening with curl_cffi TLS impersonation and request jitter 2026-03-09 17:11:59 +08:00
jackwener
b2603eb753 feat: update Chrome UA to 133 and add sec-ch-ua/Sec-Fetch headers for anti-detection 2026-03-09 12:04:26 +08:00
jackwener
d2d971c865 refactor: deep review fixes round 3
- client.py:
  - Remove dead _extract_cursor second branch (unreachable code)
  - Cache SSL context as module-level _SSL_CTX (avoid re-reading CA certs)
  - Add 404 stale-fallback retry to _graphql_post (parity with GET)
  - Remove dead core.get('name')/core.get('screen_name') in fetch_user
  - Set Content-Type: application/json only for POST requests
  - Rename _to_int → _parse_int for clarity vs config._as_int
  - Add 'not thread-safe' note on module-level caches

- cli.py:
  - _fetch_and_display now accepts optional config param (fix double load)
  - Refactor user_posts to use _fetch_and_display
  - Pass config to all _fetch_and_display callers

- pyproject.toml:
  - Move xclienttransaction/requests to optional [transaction] deps
  - Add beautifulsoup4 to [transaction] optional deps

- README.md:
  - Add rateLimit config section with comments
  - Add constants.py to project structure tree
2026-03-08 13:58:06 +08:00
jackwener
625181b76c refactor: fix remaining code review issues on kabi-use
- _get_client: remove useless try/except that re-raised same error
- verify_cookies: increase timeout from 3s to 5s
- fetch_user: use _deep_get for URL extraction (consistent with
  _parse_user_result)
- formatter: remove no-op tweets_to_json wrapper and unused import
- _as_int/_as_float: filter.py now imports from config.py (dedup)
- CLI read commands: extract _fetch_and_display() to dedup
  favorite/search/likes/list_timeline
- _write_action: move load_config inside try block
- auth.py: add PEP 8 blank line after logger
2026-03-07 21:49:12 +08:00
jackwener
df39a15d00 refactor: code review fixes
Bug fixes:
- _extract_cursor: only extract Bottom cursors, preventing Top cursor
  from corrupting pagination state
- _api_request: merge _api_get/_api_post into unified method — POST
  now has rate-limit code 88 retry (was missing)
- fetch_user_likes: add override_base_variables=True

Code quality:
- Extract BEARER_TOKEN and USER_AGENT into constants.py (was duped
  in auth.py and client.py)
- Add user_profile_to_dict/users_to_json for proper UserProfile
  serialization (followers/following JSON output was ad-hoc)
- Refactor 6 CLI write commands via _write_action helper
- Extract _extract_media and _extract_author from _parse_tweet_result
- Update CLI module docstring with all 18 commands
2026-03-07 20:30:59 +08:00
jackwener
6c73a9f0b6 fix: add fieldToggles support for TweetDetail
- TweetDetail requires fieldToggles (withArticleRichContentState: true)
  to populate tweet_results in entries — without it, server returns {}
- Add fieldToggles parameter throughout: _build_graphql_url, _graphql_get,
  _fetch_timeline
- Tested: tweet detail, followers, following, like — all working
2026-03-07 20:20:59 +08:00
jackwener
80499384c9 feat: add all remaining read/write endpoints
Read commands:
- twitter tweet <id>: view tweet detail + replies
- twitter list <id>: fetch list timeline
- twitter followers <name>: list user followers
- twitter following <name>: list user following

Write commands:
- twitter post <text>: create tweet (with --reply-to)
- twitter delete <id>: delete tweet
- twitter like/unlike <id>: manage likes
- twitter rt/unrt <id>: manage retweets
- twitter bookmark-add/bookmark-rm <id>: manage bookmarks

Infrastructure:
- _graphql_post + _api_post for write operations
- _fetch_user_list + _parse_user_result for user lists
- _deep_get now supports list index access
- _build_headers supports POST method for transaction ID
2026-03-07 20:07:10 +08:00
jackwener
767a466667 fix: add x-client-transaction-id header for search API
- Root cause: Twitter's SearchTimeline endpoint requires x-client-transaction-id
  header that HomeTimeline/Bookmarks don't enforce
- Integrate XClientTransaction library for header generation
- Add requests + beautifulsoup4 dependencies
- Fix SearchTimeline variables (override base timeline vars)
- Update FEATURES dict and fallback queryIds to match live x.com values
2026-03-07 19:53:55 +08:00
jackwener
b0866ed8d7 feat: add search and likes commands
- Add 'twitter search' command with --type (Top/Latest/Photos/Videos), --max, --json, --filter
- Add 'twitter likes' command to view tweets liked by a user
- Add SearchTimeline and Likes GraphQL operations with fallback queryIds
- Update README with new command examples (EN + CN)
2026-03-07 19:15:37 +08:00
jackwener
55c48b077b feat: add rate limiting, retry with backoff, and max count cap
- Add configurable request delay between paginated API calls (default 1.5s)
- Add retry with exponential backoff on HTTP 429 and Twitter error code 88
- Add hard max count cap (default 200, absolute ceiling 500)
- Add rateLimit config section with requestDelay, maxRetries, retryBaseDelay, maxCount
- Add normalization tests for rateLimit config
2026-03-07 19:02:49 +08:00
jackwener
4c08d09304 refactor: harden CLI/client/config and centralize serialization 2026-03-05 16:13:54 +08:00
jackwener
7238b932ab feat: add user commands, auto-detect browser, optimize performance
- Add user/user-posts/followers/following commands
- Add UserProfile model and GraphQL API methods
- Add print_user_profile and print_user_table formatters
- Auto-detect browser for cookies (Chrome → Edge → Firefox → Brave)
- Remove --browser option from all commands
- Remove cookie verification (v1.1 endpoints are gone)
- Use hardcoded fallback query IDs first (skip slow JS bundle scan)
- Update FEATURES from latest twitter-openapi config
- Fix user-posts: add required withVoice variable
- Add tweet URL links in feed output
- Add error handling to all user commands
2026-03-05 00:41:26 +08:00
jackwener
16752c3115 Initial commit: twitter-cli v0.1.0 2026-03-04 17:56:42 +08:00