clickthrough/docs/API.md

API Reference (v2)

Base URL: http://127.0.0.1:8123

If CLICKTHROUGH_TOKEN is set, include:

x-clickthrough-token: <token>

Endpoints

• POST /v2/observe
• POST /v2/localize
• POST /v2/act
• POST /v2/act-verify
• GET /health
• GET /displays
• GET /windows
• POST /windows/action
• POST /launch
• POST /exec

No v1 endpoints are supported.

POST /v2/observe

{
  "mode": "region",
  "region_x": 800,
  "region_y": 420,
  "region_width": 700,
  "region_height": 420,
  "include_image": true,
  "image_format": "jpeg",
  "jpeg_quality": 75,
  "ocr_mode": "region",
  "language_hint": "eng",
  "min_confidence": 0.45,
  "max_ocr_area_px": 1500000,
  "group_lines": true
}

Returns observation metadata, optional image, OCR blocks/lines, and timing fields.

POST /v2/localize

Text localization:

{
  "observation_id": "...",
  "text_query": "Save",
  "text_match": "exact",
  "candidate_index": 0
}

Image-tool point localization:

{
  "observation_id": "...",
  "image_tool_point": {"x": 312, "y": 188}
}

Returns resolved_target_id, global pixel, and localization_confidence.

POST /v2/act

{
  "action": {
    "action": "click",
    "target": {"resolved_target_id": "..."},
    "button": "left",
    "clicks": 1
  }
}

POST /v2/act-verify

{
  "action": {
    "action": "click",
    "target": {"resolved_target_id": "..."}
  },
  "condition": {
    "kind": "text",
    "mode": "region",
    "text": "Saved",
    "match": "contains",
    "present": true,
    "region_x": 820,
    "region_y": 420,
    "region_width": 500,
    "region_height": 140,
    "min_confidence": 0.4
  },
  "risk_level": "low"
}

Risk defaults:

• low: retries 0, timeout 2500ms
• high: retries 1, timeout 6000ms

Response envelope

Success:

{
  "ok": true,
  "request_id": "...",
  "time_ms": 1710000000000,
  "data": { },
  "error": null
}

Error:

{
  "ok": false,
  "request_id": "...",
  "time_ms": 1710000000000,
  "data": null,
  "error": {
    "code": "http_error",
    "message": "...",
    "details": {}
  }
}