Add AGENTS playbook and prioritize automation backlog

AGENTS.md

@@ -0,0 +1,48 @@
+# AGENTS
+
+## Purpose
+This file defines how agents and contributors should work in this repository.
+It is a codebase playbook, not a product roadmap.
+
+## Repository Map
+- `server/app.py`: FastAPI routes, auth checks, response envelope, exception handling.
+- `server/services.py`: screenshot/OCR/input/window/exec behavior and safety enforcement.
+- `server/models.py`: request schemas and validation rules.
+- `server/config.py`: environment loading and runtime settings.
+- `tests/`: unit and API contract tests (monkeypatched where host behavior is nondeterministic).
+- `docs/API.md`: public API request/response reference.
+- `skill/SKILL.md`: operational method for agent usage of the API.
+
+## Local Workflow
+- Install dependencies: `pip install -r requirements.txt`
+- Run server: `python -m server.app`
+- Run tests: `pytest -q`
+- Basic health check: `GET /health` with `x-clickthrough-token` when token auth is enabled.
+
+## Non-Negotiable Contracts
+- Keep the response envelope shape stable:
+  - `ok`, `request_id`, `time_ms`, `data`, `error`
+- Preserve one-action-per-request semantics in `/interact`.
+- Keep coordinate behavior in global desktop coordinates unless an explicit versioned change is introduced.
+- All new request fields must be represented in `server/models.py` with explicit validation constraints.
+
+## Safety and Security Rules
+- Do not weaken `x-clickthrough-token` validation.
+- Do not weaken `/exec` secret validation (`x-clickthrough-exec-secret`).
+- Preserve `dry_run` behavior for non-destructive execution paths.
+- Preserve allowed-region enforcement for pointer-target actions.
+- Keep `/exec` constraints explicit:
+  - shell allowlist
+  - timeout limits
+  - output truncation limits
+
+## Testing Expectations
+- Add tests for each new behavior in services/models/routes.
+- Cover success and failure paths, especially validation and ambiguity branches.
+- Prefer deterministic tests with monkeypatches over host/UI-dependent flakiness.
+- If API behavior changes, update tests in the same change.
+
+## Documentation Policy
+- When API behavior changes, update `docs/API.md` in the same change.
+- Keep examples aligned with current schema in `docs/API.md` and `examples/quickstart.py`.
+- Keep `skill/SKILL.md` aligned with current safe usage flow.

TODO.md

@@ -27,7 +27,9 @@
 - [x] Add top-level skill section for instance setup + mini API docs
 - [x] Clarify user-owned setup responsibilities vs agent responsibilities in skill docs
 
-## Deferred Backlog
-- [ ] Higher-level task macros composed from `see` + `interact` + `interact/verify` primitives
-- [ ] Additional verify primitives beyond `ocr_text_near_point` (image-diff region, window title/process state, color/pixel checks)
-- [ ] Broader API simplification pass to reduce payload overlap and consolidate shared OCR options
+## Deferred Backlog (Prioritized)
+1. [ ] Higher-level automation macros composed from `see` + `interact` + `interact/verify`
+2. [ ] Reusable workflow templates (for example: find text -> zoom fallback -> click -> verify)
+3. [ ] Batch-safe orchestration primitives with explicit per-step results and auditability
+4. [ ] Additional verify primitives beyond `ocr_text_near_point` (image diff region, window title/process state, color/pixel checks)
+5. [ ] Broader API simplification pass to reduce payload overlap and consolidate shared OCR options