Support multi-display screen selection
All checks were successful
python-syntax / syntax-check (push) Successful in 1m33s

This commit is contained in:
Space-Banane
2026-04-29 21:52:01 +02:00
parent a8f2e01bb9
commit 775c188732
6 changed files with 170 additions and 33 deletions

View File

@@ -12,19 +12,39 @@ x-clickthrough-token: <token>
Returns status and runtime safety flags, including `exec` capability config.
## `GET /displays`
Returns detected displays in API screen order.
```json
{
"ok": true,
"default_screen": 0,
"displays": [
{"screen": 0, "mss_index": 1, "primary": true, "x": 0, "y": 0, "width": 1920, "height": 1080},
{"screen": 1, "mss_index": 2, "primary": false, "x": 1920, "y": 0, "width": 1920, "height": 1080}
]
}
```
`screen` is zero-based. `screen=0` is the primary display when detectable, falling back to the first monitor reported by the capture backend.
Invalid `screen` values fall back to `0`.
## `GET /screen`
Query params:
- `screen` (int, default `0`) — zero-based display selector; invalid values fall back to `0`
- `with_grid` (bool, default `true`)
- `grid_rows` (int, default env or `12`)
- `grid_cols` (int, default env or `12`)
- `include_labels` (bool, default `true`)
- `image_format` (`png`|`jpeg`, default `png`)
- `jpeg_quality` (1-100, default `85`)
- `asImage` (bool, default `false`) if `true`, return raw image bytes only (`image/png` or `image/jpeg`)
- `asImage` (bool, default `false`) - if `true`, return raw image bytes only (`image/png` or `image/jpeg`)
Default response includes base64 image and metadata (`meta.region`, optional `meta.grid`).
Default response includes base64 image and metadata (`meta.region`, `meta.screen`, `meta.displays`, optional `meta.grid`).
`meta.region` uses global desktop coordinates.
## `POST /zoom`
@@ -47,14 +67,21 @@ Body:
Query params:
- `asImage` (bool, default `false`) — if `true`, return raw image bytes only (`image/png` or `image/jpeg`)
- `screen` (int, default `0`) - zero-based display selector; invalid values fall back to `0`
- `asImage` (bool, default `false`) - if `true`, return raw image bytes only (`image/png` or `image/jpeg`)
Default response returns cropped image + region metadata in global pixel coordinates.
Default response returns cropped image + region metadata in global pixel coordinates. `center_x` and `center_y` are also global coordinates; use the selected display's `meta.region` from `/screen?screen=X` as the coordinate base.
## `POST /action`
Body: one action.
Query params:
- `screen` (int, default `0`) - zero-based display selector included in the response metadata; invalid values fall back to `0`
Pointer coordinates remain global desktop coordinates. For multi-display actions, first capture `/screen?screen=X` and use that response's `meta.region` or grid metadata to compute the target.
### Pointer target modes
#### Pixel target
@@ -147,6 +174,10 @@ Hotkey:
Extract visible text from either a full screenshot, a region crop, or caller-provided image bytes.
Query params:
- `screen` (int, default `0`) - zero-based display selector for `mode=screen` and `mode=region`; invalid values fall back to `0`
Body:
```json
@@ -158,7 +189,7 @@ Body:
```
Modes:
- `screen` (default): OCR over full captured monitor
- `screen` (default): OCR over full selected monitor
- `region`: OCR over explicit region (`region_x`, `region_y`, `region_width`, `region_height`)
- `image`: OCR over provided `image_base64` (supports plain base64 or data URL)
@@ -246,6 +277,10 @@ Response includes `stdout`, `stderr`, `exit_code`, timeout state, and execution
Runs multiple `action` payloads sequentially.
Query params:
- `screen` (int, default `0`) - zero-based display selector applied to each action response; invalid values fall back to `0`
```json
{
"actions": [