diff --git a/README.md b/README.md index c7519e9..a77e716 100644 --- a/README.md +++ b/README.md @@ -56,8 +56,14 @@ The `tests/` suite covers grid construction, the FastAPI surface, and the skill/ - Runs `ruff check` over the Python packages. - Executes `pytest` to keep coverage high. +## Control UI + +- `/ui/` serves a small control panel where you can bootstrap a grid from a base64 screenshot, ask the planner for a preview, execute clicks, refresh the screenshot, and watch the summary/history. +- Most traffic is HTTP: `/grid/init`, `/grid/{id}/plan`, `/grid/{id}/action`, `/grid/{id}/refresh`, `/grid/{id}/summary`, and `/grid/{id}/history`. Only the `/stream/screenshots` websocket pushes updates after a refresh so the overlay redraws. +- The FastAPI root now redirects to `/ui/` when the client assets are present, making the UI a lightweight entry point for demos or manual command-and-control work. + ## Next steps - Add OCR or UI heuristics so grid cells have meaningful labels before the agent reasons about them. - Persist grids and histories in a lightweight store so long-running sessions survive restarts. -- Expose a websocket/watch endpoint that streams updated screenshots and invalidates cached `grid_id`s when the scene changes. +- Expand the UI to preview actions visually (perhaps overlaying cells on top of rendered screenshots).