Commit Graph

115 Commits

Author SHA1 Message Date
jackwener
8505428264 docs: update cross-references (xhs-cli → xiaohongshu-cli) 2026-03-11 00:33:14 +08:00
jackwener
34d2491ba2 chore: bump version to 0.5.0
- refactor: split client.py into graphql.py + parser.py
- refactor: structured exception hierarchy (exceptions.py)
- feat: integration smoke tests
- docs: updated project structure
2026-03-11 00:30:19 +08:00
jackwener
21b4c28d63 docs: update project structure in README (add graphql.py, parser.py, exceptions.py, output.py) 2026-03-10 23:45:14 +08:00
jackwener
f125d0fe7f refactor: replace all RuntimeError with specific exception types
- 5x RuntimeError → TwitterAPIError (create_tweet, fetch_me, quote_tweet, follow/unfollow)
- 2x RuntimeError(str(exc)) → bare raise (preserve original TwitterAPIError in _graphql_get/post)
2026-03-10 23:32:10 +08:00
jackwener
c2b9be4669 refactor: split client.py into graphql.py + parser.py modules
Split the monolithic client.py (1341 lines) into three focused modules:

- graphql.py (~200 lines): queryId resolution, URL building, JS bundle
  scanning, feature flag management
- parser.py (~270 lines): Tweet/User/Media/Article parsing, utility functions
  (_deep_get, _parse_int, _extract_cursor, _extract_media)
- client.py (~700 lines): TwitterClient class with HTTP engine, anti-detection,
  session management, and all public API methods

Backward compatibility: client.py re-exports all previously public symbols
so existing test imports work without modification. 88/88 tests pass.
2026-03-10 23:18:59 +08:00
jackwener
4afc4fc246 refactor: add exceptions.py module with structured exception hierarchy
- Create exceptions.py with 7 exception types: TwitterError, AuthenticationError,
  RateLimitError, NotFoundError, NetworkError, QueryIdError, TwitterAPIError
- Remove inline TwitterAPIError from client.py, import from exceptions module
- Replace RuntimeError('Cannot resolve queryId') with QueryIdError
- Replace RuntimeError('User not found') with NotFoundError
- Update test assertion for new TwitterAPIError message format
2026-03-10 23:05:05 +08:00
jackwener
9cf74abd56 feat: add integration smoke tests
CLI-level smoke tests using --yaml output against real Twitter API.
Default skipped via @pytest.mark.smoke marker + pyproject.toml addopts.
Run locally with: uv run pytest -m smoke -v
2026-03-10 22:26:46 +08:00
jackwener
fa6255f2ee docs: document structured output schema 2026-03-10 21:22:35 +08:00
jackwener
4c2c02efd5 feat: unify structured error output 2026-03-10 21:18:38 +08:00
jackwener
9b7bdf3b06 feat: wrap structured outputs in agent schema 2026-03-10 21:10:48 +08:00
jackwener
642ffe84a8 feat: unify agent status schema 2026-03-10 21:02:08 +08:00
jackwener
32d074dc9f feat: anti-detection hardening, transaction cache, article parsing, structured write output
Anti-detection:
- Add 6 sec-ch-ua-* Client Hints headers (arch, bitness, full-version, etc.)
- POST requests now send Referer: x.com/compose/post + Priority: u=1, i
- follow/unfollow REST adds include_profile_interstitial_type param

Performance:
- Transaction ID cache with 1h TTL (~/.twitter-cli/transaction_cache.json)
- resolve_user_id: auto-detect screen_name vs numeric user_id

Features:
- Twitter Article parsing: extract long-form content as Markdown
- Write operations emit structured JSON/YAML when piped or OUTPUT env set
  ActionResult: {success, action, id, url, ...}

84 tests passing
2026-03-10 20:48:42 +08:00
jackwener
97708889c9 docs: enhance SKILL.md with auth guide, agent workflows, compact mode, error ref
- Add step-by-step authentication guidance for agents (3 methods)
- Add 10+ agent workflow examples (post/reply/quote/thread/search+jq)
- Document compact mode (-c) for LLM token efficiency
- Add new commands: whoami, reply, quote, follow/unfollow
- Add complete error reference table
- Add limitations section
2026-03-10 20:13:47 +08:00
jackwener
49d3e237c4 feat: add whoami, reply, quote, follow/unfollow commands and --compact mode
- whoami: fetch current authenticated user profile
- reply <id> <text>: standalone reply command
- quote <id> <text>: quote-tweet command
- follow/unfollow <handle>: follow/unfollow users
- --compact/-c: global flag for LLM-friendly minimal JSON output
- client.py: add fetch_me, quote_tweet, follow_user, unfollow_user
- serialization.py: add tweet_to_compact_dict, tweets_to_compact_json
- 7 new tests (82 total, all passing)
2026-03-10 20:09:08 +08:00
jackwener
250fca46f0 docs: add structured output guidance for AI agents 2026-03-10 15:47:06 +08:00
jackwener
d22a729fb9 docs: add related CLI project links 2026-03-10 14:15:35 +08:00
jackwener
d71ad45a0a fix: harden pagination auth and runtime headers 2026-03-10 12:33:04 +08:00
jackwener
4f144d1591 test: track parser fixture files 2026-03-10 12:08:47 +08:00
jackwener
19ab11d6a4 fix: harden auth flow and sync browser support docs 2026-03-10 11:02:34 +08:00
jakevin
db7d7e8874 Merge pull request #5 from y0u-0/arc-browser-support
feat: add Arc browser support
2026-03-10 11:02:08 +08:00
Ali Aldhamen
5eb0b1a8f4 feat: add arc browser support 2026-03-09 21:04:48 +03:00
jackwener
7fb02ba18c feat: cookie file cache with TTL + user --json
- Cookie cache: save to ~/.cache/twitter-cli/cookies.json (24h TTL)
- On 401/403 auth failure: auto-invalidate cache, re-extract from browser
- Cache uses 0600 permissions for security
- Add --json option to twitter user command for scripting
- Priority: env vars → cache file → browser extraction
2026-03-09 21:47:23 +08:00
jackwener
bbe0bc8de3 docs: rename rt/unrt → retweet/unretweet, update SKILL.md + README.md
- Rename rt/unrt commands to retweet/unretweet for clarity
- SKILL.md: fix Python version, add --output/--filter examples, safety notes
- README.md: dynamic Chrome fingerprint, write delays, --output examples
- Sync Chinese section with English changes
2026-03-09 21:20:05 +08:00
jackwener
8313a7012f refactor: dynamic UA matching, session reuse, score Optional, --output on all commands
- constants.py: sync_chrome_version() aligns UA/sec-ch-ua with impersonate target
- auth.py: reuse shared cffi session instead of creating duplicate
- filter.py: eliminate double weight building in score_tweet
- models.py: Tweet.score → Optional[float] for accurate display
- cli.py: add --output to search/likes/user-posts for consistency
2026-03-09 21:15:28 +08:00
jackwener
fda9b1c3dc fix: 431 Request Too Large — reduce FEATURES to 15 essential keys, dynamic update only updates existing keys 2026-03-09 20:59:16 +08:00
jackwener
d20c5699fd fix: 414 URI Too Long — omit False-valued features from GET URL, add regression tests 2026-03-09 20:50:45 +08:00
jackwener
12f425abea feat: write operation delays, dynamic FEATURES update, 30+ client.py tests, fix README proxy wording 2026-03-09 20:45:51 +08:00
jackwener
b9c226b804 fix: dynamic Chrome impersonation + in-process macOS Keychain cookie extraction (closes #4) 2026-03-09 20:38:47 +08:00
jackwener
59b5df7f71 fix: ClawHub publish (acceptLicenseTerms), add LICENSE, fix score display bug, remove unused var 2026-03-09 20:32:45 +08:00
jackwener
94e21fba9a fix: remove unused requests import (CI lint), fix README wording, drop requests dep 2026-03-09 20:25:24 +08:00
jackwener
a6ad246071 fix: improve error handling for write operation rate limits (retweet/like) 2026-03-09 20:22:46 +08:00
jackwener
27b02578c8 docs: update README with anti-detection features, proxy support, and ban prevention tips 2026-03-09 19:17:01 +08:00
jackwener
731151e62e fix: use curl_cffi for ClientTransaction init to prevent TLS fingerprint leak 2026-03-09 19:14:33 +08:00
jackwener
b83abadb73 feat: full cookie forwarding from browser and TWITTER_PROXY support 2026-03-09 19:12:06 +08:00
jackwener
27d73efee5 feat: anti-detection hardening with curl_cffi TLS impersonation and request jitter 2026-03-09 17:11:59 +08:00
jackwener
3e13ed3749 chore: bump version to 0.1.1 2026-03-09 12:28:45 +08:00
jackwener
b2603eb753 feat: update Chrome UA to 133 and add sec-ch-ua/Sec-Fetch headers for anti-detection 2026-03-09 12:04:26 +08:00
jackwener
df9b58ed0a docs: comprehensive SKILL.md rewrite — add install, all commands, patterns, error handling, safety notes 2026-03-09 01:38:51 +08:00
jackwener
0a7b6a6b78 fix: console output to stderr for clean --json output
- Change Console() to Console(stderr=True) so all status/progress
  messages go to stderr, keeping stdout pure JSON when --json is used
- Add missing exception handling in likes command for fetch_user
- Fix SKILL.md: favorite -> favorites (correct command name)
2026-03-08 22:37:00 +08:00
jackwener
2559549a84 refactor: rename CLI commands for consistency
- user-posts → tweets (remove only hyphenated command)
- rt → retweet (more readable)
- unrt → unretweet (more readable)

All commands now use full English words.
2026-03-08 14:05:36 +08:00
jackwener
215337ebec refactor: rename bookmark commands to favorite/unfavorite/favorites
- favorite <id>     → bookmark a tweet (was bookmark-add)
- unfavorite <id>    → remove bookmark (was bookmark-rm)
- favorites          → list all bookmarks (was favorite)

Consistent with like/unlike/likes naming pattern.
2026-03-08 14:00:38 +08:00
jackwener
d2d971c865 refactor: deep review fixes round 3
- client.py:
  - Remove dead _extract_cursor second branch (unreachable code)
  - Cache SSL context as module-level _SSL_CTX (avoid re-reading CA certs)
  - Add 404 stale-fallback retry to _graphql_post (parity with GET)
  - Remove dead core.get('name')/core.get('screen_name') in fetch_user
  - Set Content-Type: application/json only for POST requests
  - Rename _to_int → _parse_int for clarity vs config._as_int
  - Add 'not thread-safe' note on module-level caches

- cli.py:
  - _fetch_and_display now accepts optional config param (fix double load)
  - Refactor user_posts to use _fetch_and_display
  - Pass config to all _fetch_and_display callers

- pyproject.toml:
  - Move xclienttransaction/requests to optional [transaction] deps
  - Add beautifulsoup4 to [transaction] optional deps

- README.md:
  - Add rateLimit config section with comments
  - Add constants.py to project structure tree
2026-03-08 13:58:06 +08:00
jackwener
625181b76c refactor: fix remaining code review issues on kabi-use
- _get_client: remove useless try/except that re-raised same error
- verify_cookies: increase timeout from 3s to 5s
- fetch_user: use _deep_get for URL extraction (consistent with
  _parse_user_result)
- formatter: remove no-op tweets_to_json wrapper and unused import
- _as_int/_as_float: filter.py now imports from config.py (dedup)
- CLI read commands: extract _fetch_and_display() to dedup
  favorite/search/likes/list_timeline
- _write_action: move load_config inside try block
- auth.py: add PEP 8 blank line after logger
2026-03-07 21:49:12 +08:00
jackwener
df39a15d00 refactor: code review fixes
Bug fixes:
- _extract_cursor: only extract Bottom cursors, preventing Top cursor
  from corrupting pagination state
- _api_request: merge _api_get/_api_post into unified method — POST
  now has rate-limit code 88 retry (was missing)
- fetch_user_likes: add override_base_variables=True

Code quality:
- Extract BEARER_TOKEN and USER_AGENT into constants.py (was duped
  in auth.py and client.py)
- Add user_profile_to_dict/users_to_json for proper UserProfile
  serialization (followers/following JSON output was ad-hoc)
- Refactor 6 CLI write commands via _write_action helper
- Extract _extract_media and _extract_author from _parse_tweet_result
- Update CLI module docstring with all 18 commands
2026-03-07 20:30:59 +08:00
jackwener
6c73a9f0b6 fix: add fieldToggles support for TweetDetail
- TweetDetail requires fieldToggles (withArticleRichContentState: true)
  to populate tweet_results in entries — without it, server returns {}
- Add fieldToggles parameter throughout: _build_graphql_url, _graphql_get,
  _fetch_timeline
- Tested: tweet detail, followers, following, like — all working
2026-03-07 20:20:59 +08:00
jackwener
80499384c9 feat: add all remaining read/write endpoints
Read commands:
- twitter tweet <id>: view tweet detail + replies
- twitter list <id>: fetch list timeline
- twitter followers <name>: list user followers
- twitter following <name>: list user following

Write commands:
- twitter post <text>: create tweet (with --reply-to)
- twitter delete <id>: delete tweet
- twitter like/unlike <id>: manage likes
- twitter rt/unrt <id>: manage retweets
- twitter bookmark-add/bookmark-rm <id>: manage bookmarks

Infrastructure:
- _graphql_post + _api_post for write operations
- _fetch_user_list + _parse_user_result for user lists
- _deep_get now supports list index access
- _build_headers supports POST method for transaction ID
2026-03-07 20:07:10 +08:00
jackwener
767a466667 fix: add x-client-transaction-id header for search API
- Root cause: Twitter's SearchTimeline endpoint requires x-client-transaction-id
  header that HomeTimeline/Bookmarks don't enforce
- Integrate XClientTransaction library for header generation
- Add requests + beautifulsoup4 dependencies
- Fix SearchTimeline variables (override base timeline vars)
- Update FEATURES dict and fallback queryIds to match live x.com values
2026-03-07 19:53:55 +08:00
jackwener
b0866ed8d7 feat: add search and likes commands
- Add 'twitter search' command with --type (Top/Latest/Photos/Videos), --max, --json, --filter
- Add 'twitter likes' command to view tweets liked by a user
- Add SearchTimeline and Likes GraphQL operations with fallback queryIds
- Update README with new command examples (EN + CN)
2026-03-07 19:15:37 +08:00
jackwener
55c48b077b feat: add rate limiting, retry with backoff, and max count cap
- Add configurable request delay between paginated API calls (default 1.5s)
- Add retry with exponential backoff on HTTP 429 and Twitter error code 88
- Add hard max count cap (default 200, absolute ceiling 500)
- Add rateLimit config section with requestDelay, maxRetries, retryBaseDelay, maxCount
- Add normalization tests for rateLimit config
2026-03-07 19:02:49 +08:00
jackwener
0f26e20abb Revert npx skills install examples in README 2026-03-06 21:23:32 +08:00