feat(verify): add compound action+verify flows
All checks were successful
python-syntax / syntax-check (push) Successful in 9s
All checks were successful
python-syntax / syntax-check (push) Successful in 9s
This commit is contained in:
@@ -59,6 +59,7 @@ Say what you actually have: screenshots, OCR output, and fresh verification capt
|
||||
- `POST /ocr` → text extraction with bounding boxes from full screen, region, or provided image bytes
|
||||
- `POST /ocr/find?screen=0` → search OCR output for matching text candidates
|
||||
- `POST /action?screen=0` → single interaction (`move`, `click`, `scroll`, `type`, `hotkey`, ...)
|
||||
- `POST /action/verify?screen=0` → execute one action plus structured success verification
|
||||
- `POST /batch?screen=0` → sequential action list
|
||||
- `POST /exec` → PowerShell/Bash/CMD command execution (requires configured exec secret + header)
|
||||
|
||||
@@ -93,7 +94,7 @@ Good pattern:
|
||||
3. ask a precise question about the visible UI
|
||||
4. convert the answer into a concrete Clickthrough target
|
||||
5. act once
|
||||
6. recapture and verify again
|
||||
6. recapture and verify again, or use `POST /action/verify` when the action+postcondition loop is simple enough to bundle cleanly
|
||||
|
||||
Ask narrow questions.
|
||||
Good:
|
||||
@@ -187,7 +188,7 @@ Avoid using `/exec` for routine in-app clicks, menu navigation, or text entry wh
|
||||
4. **Before any click**, verify target identity (OCR text/icon/location consistency).
|
||||
5. If OCR is insufficient, inspect the screenshot explicitly with the OpenClaw `image` tool instead of pretending you can already see enough.
|
||||
6. Execute one minimal action via `POST /action`.
|
||||
7. Re-capture with `GET /screen` or use `POST /wait` to verify the expected state change.
|
||||
7. Re-capture with `GET /screen` or use `POST /wait`, `POST /vision/diff`, `POST /vision/stability`, or `POST /action/verify` to verify the expected state change.
|
||||
8. Repeat until objective is complete.
|
||||
|
||||
## Verify-before-click rules
|
||||
|
||||
Reference in New Issue
Block a user