Commit Graph

52 Commits

Author SHA1 Message Date
jackwener
1313eb0be1 feat: add full-text option for tweet tables 2026-03-11 20:58:12 +08:00
jackwener
88a9f4ce97 fix: tighten pagination and platform-specific auth 2026-03-11 20:32:51 +08:00
jackwener
74f06638ee fix: resolve lint errors (E741 ambiguous var, F401 unused import) 2026-03-11 17:33:36 +08:00
jackwener
1de88ea2ed fix: update Likes queryId and response path (fixes #8)
- Update Likes fallback queryId: aeJWz7GtGNHHO2Z3GrjCWg -> dv5-II7_Bup_PHish7p6fw
- Fix response path: data.user.result.timeline.timeline.instructions
  (Twitter renamed timeline_v2 to timeline; code now tries both)
- Verified via Playwright: API returns 200 with correct tweet entries

Fixes #8
2026-03-11 17:08:14 +08:00
jackwener
60e1e7c580 feat: improve cookie extraction diagnostics and add doctor command
- Add _diagnose_keychain_issues() for macOS Keychain/SSH detection
- Extraction functions now return (cookies, diagnostics) tuples
- Error messages include actionable Keychain hints (e.g. unlock-keychain)
- Add 'twitter doctor' diagnostic command for troubleshooting
- Enhance bug_report.yml with browser/access method/diagnostics fields
- Expand README troubleshooting (EN+CN) with Keychain/SSH solutions
- Add 5 new tests for Keychain diagnostics

Closes #11
2026-03-11 16:53:06 +08:00
jackwener
47be88e62d fix: use importlib.metadata for __version__ (was hardcoded stale) 2026-03-11 14:11:07 +08:00
jackwener
53a700ec60 feat: support Chrome multi-profile cookie extraction
Auto-iterates all Chrome/Arc/Edge/Brave profiles (Default, Profile 1,
Profile 2, ...) to find Twitter cookies. Falls back to the default
browser_cookie3 behavior when no profile dirs are found.

Set TWITTER_CHROME_PROFILE env var to specify a profile explicitly:
  TWITTER_CHROME_PROFILE='Profile 2' twitter feed

Closes #6
2026-03-11 12:53:25 +08:00
jackwener
93840ba99b fix: add logger definition to cli.py (fix CI lint F821) 2026-03-11 12:37:28 +08:00
aidiff-kobe
f01ce77f20 refactor: Use specific ImportError instead of broad Exception (#10)
Changed the exception handling in _best_chrome_target() to catch
specific ImportError instead of broad Exception. This improves
code clarity and avoids masking unexpected errors.

Also added a debug log message to help diagnose when curl_cffi
is not available.

Co-authored-by: Security Bot <agent@example.com>
2026-03-11 12:36:13 +08:00
jackwener
e9efa11fa5 fix: warn when querying other users' likes (private since Jun 2024)
Twitter/X made all likes private since June 2024. The likes command now:
- Detects if the target user differs from the authenticated user
- Shows a clear warning that only your own likes are visible
- Updated SKILL.md and README.md with likes privacy limitation

Closes #8
2026-03-11 12:34:17 +08:00
jackwener
1f267008ad fix: update stale Followers/Following queryIds and retry on 422
Twitter now returns HTTP 422 GRAPHQL_VALIDATION_FAILED (not just 404)
when a queryId goes stale. Updated fallback IDs and added 422 to the
stale-queryId retry logic in both _graphql_get and _graphql_post.
2026-03-11 00:45:13 +08:00
jackwener
5c1015f1fd fix: handle changed /account/multi/list.json response format
Twitter changed the response format from a list with nested 'user'
objects to {"users": [{user_id, name, screen_name, ...}]} with
minimal fields. Now extracts screen_name from the new format and
fetches the full profile via GraphQL UserByScreenName endpoint.
2026-03-11 00:40:07 +08:00
jackwener
f125d0fe7f refactor: replace all RuntimeError with specific exception types
- 5x RuntimeError → TwitterAPIError (create_tweet, fetch_me, quote_tweet, follow/unfollow)
- 2x RuntimeError(str(exc)) → bare raise (preserve original TwitterAPIError in _graphql_get/post)
2026-03-10 23:32:10 +08:00
jackwener
c2b9be4669 refactor: split client.py into graphql.py + parser.py modules
Split the monolithic client.py (1341 lines) into three focused modules:

- graphql.py (~200 lines): queryId resolution, URL building, JS bundle
  scanning, feature flag management
- parser.py (~270 lines): Tweet/User/Media/Article parsing, utility functions
  (_deep_get, _parse_int, _extract_cursor, _extract_media)
- client.py (~700 lines): TwitterClient class with HTTP engine, anti-detection,
  session management, and all public API methods

Backward compatibility: client.py re-exports all previously public symbols
so existing test imports work without modification. 88/88 tests pass.
2026-03-10 23:18:59 +08:00
jackwener
4afc4fc246 refactor: add exceptions.py module with structured exception hierarchy
- Create exceptions.py with 7 exception types: TwitterError, AuthenticationError,
  RateLimitError, NotFoundError, NetworkError, QueryIdError, TwitterAPIError
- Remove inline TwitterAPIError from client.py, import from exceptions module
- Replace RuntimeError('Cannot resolve queryId') with QueryIdError
- Replace RuntimeError('User not found') with NotFoundError
- Update test assertion for new TwitterAPIError message format
2026-03-10 23:05:05 +08:00
jackwener
9cf74abd56 feat: add integration smoke tests
CLI-level smoke tests using --yaml output against real Twitter API.
Default skipped via @pytest.mark.smoke marker + pyproject.toml addopts.
Run locally with: uv run pytest -m smoke -v
2026-03-10 22:26:46 +08:00
jackwener
4c2c02efd5 feat: unify structured error output 2026-03-10 21:18:38 +08:00
jackwener
9b7bdf3b06 feat: wrap structured outputs in agent schema 2026-03-10 21:10:48 +08:00
jackwener
642ffe84a8 feat: unify agent status schema 2026-03-10 21:02:08 +08:00
jackwener
32d074dc9f feat: anti-detection hardening, transaction cache, article parsing, structured write output
Anti-detection:
- Add 6 sec-ch-ua-* Client Hints headers (arch, bitness, full-version, etc.)
- POST requests now send Referer: x.com/compose/post + Priority: u=1, i
- follow/unfollow REST adds include_profile_interstitial_type param

Performance:
- Transaction ID cache with 1h TTL (~/.twitter-cli/transaction_cache.json)
- resolve_user_id: auto-detect screen_name vs numeric user_id

Features:
- Twitter Article parsing: extract long-form content as Markdown
- Write operations emit structured JSON/YAML when piped or OUTPUT env set
  ActionResult: {success, action, id, url, ...}

84 tests passing
2026-03-10 20:48:42 +08:00
jackwener
49d3e237c4 feat: add whoami, reply, quote, follow/unfollow commands and --compact mode
- whoami: fetch current authenticated user profile
- reply <id> <text>: standalone reply command
- quote <id> <text>: quote-tweet command
- follow/unfollow <handle>: follow/unfollow users
- --compact/-c: global flag for LLM-friendly minimal JSON output
- client.py: add fetch_me, quote_tweet, follow_user, unfollow_user
- serialization.py: add tweet_to_compact_dict, tweets_to_compact_json
- 7 new tests (82 total, all passing)
2026-03-10 20:09:08 +08:00
jackwener
d71ad45a0a fix: harden pagination auth and runtime headers 2026-03-10 12:33:04 +08:00
jackwener
19ab11d6a4 fix: harden auth flow and sync browser support docs 2026-03-10 11:02:34 +08:00
Ali Aldhamen
5eb0b1a8f4 feat: add arc browser support 2026-03-09 21:04:48 +03:00
jackwener
7fb02ba18c feat: cookie file cache with TTL + user --json
- Cookie cache: save to ~/.cache/twitter-cli/cookies.json (24h TTL)
- On 401/403 auth failure: auto-invalidate cache, re-extract from browser
- Cache uses 0600 permissions for security
- Add --json option to twitter user command for scripting
- Priority: env vars → cache file → browser extraction
2026-03-09 21:47:23 +08:00
jackwener
bbe0bc8de3 docs: rename rt/unrt → retweet/unretweet, update SKILL.md + README.md
- Rename rt/unrt commands to retweet/unretweet for clarity
- SKILL.md: fix Python version, add --output/--filter examples, safety notes
- README.md: dynamic Chrome fingerprint, write delays, --output examples
- Sync Chinese section with English changes
2026-03-09 21:20:05 +08:00
jackwener
8313a7012f refactor: dynamic UA matching, session reuse, score Optional, --output on all commands
- constants.py: sync_chrome_version() aligns UA/sec-ch-ua with impersonate target
- auth.py: reuse shared cffi session instead of creating duplicate
- filter.py: eliminate double weight building in score_tweet
- models.py: Tweet.score → Optional[float] for accurate display
- cli.py: add --output to search/likes/user-posts for consistency
2026-03-09 21:15:28 +08:00
jackwener
fda9b1c3dc fix: 431 Request Too Large — reduce FEATURES to 15 essential keys, dynamic update only updates existing keys 2026-03-09 20:59:16 +08:00
jackwener
d20c5699fd fix: 414 URI Too Long — omit False-valued features from GET URL, add regression tests 2026-03-09 20:50:45 +08:00
jackwener
12f425abea feat: write operation delays, dynamic FEATURES update, 30+ client.py tests, fix README proxy wording 2026-03-09 20:45:51 +08:00
jackwener
b9c226b804 fix: dynamic Chrome impersonation + in-process macOS Keychain cookie extraction (closes #4) 2026-03-09 20:38:47 +08:00
jackwener
59b5df7f71 fix: ClawHub publish (acceptLicenseTerms), add LICENSE, fix score display bug, remove unused var 2026-03-09 20:32:45 +08:00
jackwener
94e21fba9a fix: remove unused requests import (CI lint), fix README wording, drop requests dep 2026-03-09 20:25:24 +08:00
jackwener
a6ad246071 fix: improve error handling for write operation rate limits (retweet/like) 2026-03-09 20:22:46 +08:00
jackwener
731151e62e fix: use curl_cffi for ClientTransaction init to prevent TLS fingerprint leak 2026-03-09 19:14:33 +08:00
jackwener
b83abadb73 feat: full cookie forwarding from browser and TWITTER_PROXY support 2026-03-09 19:12:06 +08:00
jackwener
27d73efee5 feat: anti-detection hardening with curl_cffi TLS impersonation and request jitter 2026-03-09 17:11:59 +08:00
jackwener
b2603eb753 feat: update Chrome UA to 133 and add sec-ch-ua/Sec-Fetch headers for anti-detection 2026-03-09 12:04:26 +08:00
jackwener
0a7b6a6b78 fix: console output to stderr for clean --json output
- Change Console() to Console(stderr=True) so all status/progress
  messages go to stderr, keeping stdout pure JSON when --json is used
- Add missing exception handling in likes command for fetch_user
- Fix SKILL.md: favorite -> favorites (correct command name)
2026-03-08 22:37:00 +08:00
jackwener
2559549a84 refactor: rename CLI commands for consistency
- user-posts → tweets (remove only hyphenated command)
- rt → retweet (more readable)
- unrt → unretweet (more readable)

All commands now use full English words.
2026-03-08 14:05:36 +08:00
jackwener
215337ebec refactor: rename bookmark commands to favorite/unfavorite/favorites
- favorite <id>     → bookmark a tweet (was bookmark-add)
- unfavorite <id>    → remove bookmark (was bookmark-rm)
- favorites          → list all bookmarks (was favorite)

Consistent with like/unlike/likes naming pattern.
2026-03-08 14:00:38 +08:00
jackwener
d2d971c865 refactor: deep review fixes round 3
- client.py:
  - Remove dead _extract_cursor second branch (unreachable code)
  - Cache SSL context as module-level _SSL_CTX (avoid re-reading CA certs)
  - Add 404 stale-fallback retry to _graphql_post (parity with GET)
  - Remove dead core.get('name')/core.get('screen_name') in fetch_user
  - Set Content-Type: application/json only for POST requests
  - Rename _to_int → _parse_int for clarity vs config._as_int
  - Add 'not thread-safe' note on module-level caches

- cli.py:
  - _fetch_and_display now accepts optional config param (fix double load)
  - Refactor user_posts to use _fetch_and_display
  - Pass config to all _fetch_and_display callers

- pyproject.toml:
  - Move xclienttransaction/requests to optional [transaction] deps
  - Add beautifulsoup4 to [transaction] optional deps

- README.md:
  - Add rateLimit config section with comments
  - Add constants.py to project structure tree
2026-03-08 13:58:06 +08:00
jackwener
625181b76c refactor: fix remaining code review issues on kabi-use
- _get_client: remove useless try/except that re-raised same error
- verify_cookies: increase timeout from 3s to 5s
- fetch_user: use _deep_get for URL extraction (consistent with
  _parse_user_result)
- formatter: remove no-op tweets_to_json wrapper and unused import
- _as_int/_as_float: filter.py now imports from config.py (dedup)
- CLI read commands: extract _fetch_and_display() to dedup
  favorite/search/likes/list_timeline
- _write_action: move load_config inside try block
- auth.py: add PEP 8 blank line after logger
2026-03-07 21:49:12 +08:00
jackwener
df39a15d00 refactor: code review fixes
Bug fixes:
- _extract_cursor: only extract Bottom cursors, preventing Top cursor
  from corrupting pagination state
- _api_request: merge _api_get/_api_post into unified method — POST
  now has rate-limit code 88 retry (was missing)
- fetch_user_likes: add override_base_variables=True

Code quality:
- Extract BEARER_TOKEN and USER_AGENT into constants.py (was duped
  in auth.py and client.py)
- Add user_profile_to_dict/users_to_json for proper UserProfile
  serialization (followers/following JSON output was ad-hoc)
- Refactor 6 CLI write commands via _write_action helper
- Extract _extract_media and _extract_author from _parse_tweet_result
- Update CLI module docstring with all 18 commands
2026-03-07 20:30:59 +08:00
jackwener
6c73a9f0b6 fix: add fieldToggles support for TweetDetail
- TweetDetail requires fieldToggles (withArticleRichContentState: true)
  to populate tweet_results in entries — without it, server returns {}
- Add fieldToggles parameter throughout: _build_graphql_url, _graphql_get,
  _fetch_timeline
- Tested: tweet detail, followers, following, like — all working
2026-03-07 20:20:59 +08:00
jackwener
80499384c9 feat: add all remaining read/write endpoints
Read commands:
- twitter tweet <id>: view tweet detail + replies
- twitter list <id>: fetch list timeline
- twitter followers <name>: list user followers
- twitter following <name>: list user following

Write commands:
- twitter post <text>: create tweet (with --reply-to)
- twitter delete <id>: delete tweet
- twitter like/unlike <id>: manage likes
- twitter rt/unrt <id>: manage retweets
- twitter bookmark-add/bookmark-rm <id>: manage bookmarks

Infrastructure:
- _graphql_post + _api_post for write operations
- _fetch_user_list + _parse_user_result for user lists
- _deep_get now supports list index access
- _build_headers supports POST method for transaction ID
2026-03-07 20:07:10 +08:00
jackwener
767a466667 fix: add x-client-transaction-id header for search API
- Root cause: Twitter's SearchTimeline endpoint requires x-client-transaction-id
  header that HomeTimeline/Bookmarks don't enforce
- Integrate XClientTransaction library for header generation
- Add requests + beautifulsoup4 dependencies
- Fix SearchTimeline variables (override base timeline vars)
- Update FEATURES dict and fallback queryIds to match live x.com values
2026-03-07 19:53:55 +08:00
jackwener
b0866ed8d7 feat: add search and likes commands
- Add 'twitter search' command with --type (Top/Latest/Photos/Videos), --max, --json, --filter
- Add 'twitter likes' command to view tweets liked by a user
- Add SearchTimeline and Likes GraphQL operations with fallback queryIds
- Update README with new command examples (EN + CN)
2026-03-07 19:15:37 +08:00
jackwener
55c48b077b feat: add rate limiting, retry with backoff, and max count cap
- Add configurable request delay between paginated API calls (default 1.5s)
- Add retry with exponential backoff on HTTP 429 and Twitter error code 88
- Add hard max count cap (default 200, absolute ceiling 500)
- Add rateLimit config section with requestDelay, maxRetries, retryBaseDelay, maxCount
- Add normalization tests for rateLimit config
2026-03-07 19:02:49 +08:00
jackwener
4c08d09304 refactor: harden CLI/client/config and centralize serialization 2026-03-05 16:13:54 +08:00