space/clickthrough
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

Document control UI
Some checks failed
CI / test (push) Failing after 11s
Details

Browse Source
This commit is contained in:
Luna
2026-04-05 19:37:39 +02:00
parent 5fa516f7e7
commit 48ac9f5d7d
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@@ -56,8 +56,14 @@ The `tests/` suite covers grid construction, the FastAPI surface, and the skill/
- Runs `ruff check` over the Python packages.
- Executes `pytest` to keep coverage high.
## Control UI
- `/ui/` serves a small control panel where you can bootstrap a grid from a base64 screenshot, ask the planner for a preview, execute clicks, refresh the screenshot, and watch the summary/history.
- Most traffic is HTTP: `/grid/init`, `/grid/{id}/plan`, `/grid/{id}/action`, `/grid/{id}/refresh`, `/grid/{id}/summary`, and `/grid/{id}/history`. Only the `/stream/screenshots` websocket pushes updates after a refresh so the overlay redraws.
- The FastAPI root now redirects to `/ui/` when the client assets are present, making the UI a lightweight entry point for demos or manual command-and-control work.
## Next steps
- Add OCR or UI heuristics so grid cells have meaningful labels before the agent reasons about them.
- Persist grids and histories in a lightweight store so long-running sessions survive restarts.
- Expose a websocket/watch endpoint that streams updated screenshots and invalidates cached `grid_id`s when the scene changes.
- Expand the UI to preview actions visually (perhaps overlaying cells on top of rendered screenshots).