feat. Revision of Agent Lopp & major clean-up

README.md

@@ -12,11 +12,12 @@ It lets an LLM use controlled local tools (screen, mouse, keyboard, clipboard, s
 - Returns structured agent output as:
   - `return`: human-readable completion message
   - `data`: structured payload (for example command output)
+- Cleans up old runs and history on startup (default retention: 7 days)
 
 ## Core Features
 
 - Hybrid control model: screenshot grounding plus Windows-native window, dialog, and UI-element helpers when available
-- Tool-based agent loop (`execute_command`, `see_screen`, `enhance`, `list_windows`, `find_window`, `focus_window`, `close_window`, `wait_for_window`, `wait_for_focus_change`, `detect_dialog`, `dialog_action`, `dialog_set_filename`, `wait_for_dialog_close`, `list_ui_elements`, `invoke_ui_element`, `set_ui_element_value`, `select_ui_element`, `wait_for_ui_element`, `click`, `scroll`, `drag`, `move_mouse`, `type`, `press_key`, `clipboard_get`, `clipboard_set`, `get_cursor_position`, `get_active_window`, `sleep`, `task_complete`)
+- Tool-based agent loop (`execute_command`, `enhance`, `list_windows`, `find_window`, `focus_window`, `close_window`, `wait_for_window`, `wait_for_focus_change`, `detect_dialog`, `dialog_action`, `dialog_set_filename`, `wait_for_dialog_close`, `list_ui_elements`, `invoke_ui_element`, `set_ui_element_value`, `select_ui_element`, `wait_for_ui_element`, `click`, `scroll`, `drag`, `move_mouse`, `type`, `press_key`, `clipboard_get`, `clipboard_set`, `get_cursor_position`, `sleep`, `task_complete`)
 - Safety pre-check with override support
 - Per-job tool disable list
 - Live/final usage and cost estimates
@@ -73,6 +74,7 @@ SCREENJOB_SAFETY_MODEL=gpt-5.4-mini
 SCREENJOB_HOST=127.0.0.1
 SCREENJOB_PORT=8787
 DISABLE_UI=false
+SCREENJOB_RETENTION_DAYS=7
 ```
 
 ## Usage
@@ -83,16 +85,11 @@ DISABLE_UI=false
 python main.py run "Open amazon.de and go to my orders"
 ```
 
-CLI JSON output includes both legacy and structured fields:
+CLI JSON output:
 
 ```json
 {
   "completed": true,
-  "result": "Task completed successfully",
-  "response": {
-    "return": "Task completed successfully",
-    "data": "file1.txt\nfile2.txt"
-  },
   "return": "Task completed successfully",
   "data": "file1.txt\nfile2.txt"
 }
@@ -147,43 +144,6 @@ If you need to start the backend manually, run:
 .\start_backend.ps1
 ```
 
-The legacy Windows service host remains in the tree for reference, but it is not the recommended path for GUI tasks.
-
-### System Tray Icon (Windows)
-
-Start tray icon now:
-
-```powershell
-powershell -NoProfile -ExecutionPolicy Bypass -STA -File .\screenjob_tray.ps1
-```
-
-Install startup shortcut (current user):
-
-```powershell
-.\install_tray_startup_shortcut.ps1
-```
-
-Install startup shortcut for all users:
-
-```powershell
-.\install_tray_startup_shortcut.ps1 -AllUsers
-```
-
-Remove startup shortcut:
-
-```powershell
-.\install_tray_startup_shortcut.ps1 -Remove
-```
-
-Tray menu actions:
-
-- The service controls are for the legacy Windows service host.
-- Refresh service status
-- Start/Stop/Restart service (prompts for admin/UAC)
-- Open dashboard URL from `.env` `SCREENJOB_HOST` / `SCREENJOB_PORT`
-- Open service logs folder
-- Exit tray icon process
-
 Auth for all API routes:
 
 - `Authorization: Bearer <SCREENJOB_TOKEN>`
@@ -225,10 +185,8 @@ Response:
 
 Each job payload includes:
 
-- `result` (compat string)
-- `response.return`
-- `response.data`
-- top-level `return` and `data` aliases
+- `return`
+- `data`
 
 ### Monitoring UI
 
@@ -236,20 +194,20 @@ Each job payload includes:
 - Read-only dashboard (no run controls)
 - Requires token input
 - Live updates via `/ws`
-- Analytics dashboards for success rate by objective category and daily averages
+- Analytics dashboard for success by objective category
 - Set `DISABLE_UI=true` to disable UI
 
 ### Analytics API
 
 - `GET /api/analytics`
-- Returns objective-category success rates plus average steps/cost over time
+- Returns objective-category success rates plus average steps/cost by category
 
 ## Agent Instructions (Practical)
 
 - Prefer `execute_command` for deterministic actions (opening URLs, filesystem checks).
 - First classify the current Windows surface, then choose the control channel.
 - Prefer native window/dialog/element tools for focus changes, file pickers, modal confirmations, and browser-owned dialogs when available.
-- Use `see_screen` before UI interaction.
+- Use `enhance` for screen observation. Call it with no coordinate for a full-screen grid capture.
 - Use `enhance` before clicking small/ambiguous targets; prefer `region="small"` for compact controls.
 - Use `enhance` `mode="text"` for tiny labels/text, or `mode="ui"` for general UI.
 - Optionally set `enhance` `scale` (2-6) for tighter zoom control.
@@ -261,11 +219,11 @@ Each job payload includes:
 - Use `click` offsets via `offset_up/down/left/right`; set `button` and `click_count` there instead of inventing one-off click tools.
 - Use `move_mouse` when you need hover-only behavior and `drag` for slider, selection, or window moves.
 - Use `scroll` for vertical navigation; positive amounts scroll up and negative amounts scroll down.
-- Use `clipboard_get` / `clipboard_set` for copy-paste workflows, `get_cursor_position` for cursor inspection, and `get_active_window` before interacting with uncertain focus.
+- Use `clipboard_get` / `clipboard_set` for copy-paste workflows, `get_cursor_position` for cursor inspection, and rely on the injected foreground-window context before interacting with uncertain focus.
 - If native automation is unavailable or disabled, ScreenJob falls back to screenshots plus mouse/keyboard control and emits fallback events.
 - When done, call:
   - `task_complete(return="...", data=...)`
-- Before `task_complete`, verify expected on-screen content with `see_screen` (and `enhance` if needed), and include an `observed_result` summary in `data`.
+- Before `task_complete`, verify expected on-screen content with the latest retained screen and `enhance` if needed, and include an `observed_result` summary in `data`.
 
 Per-job `disabled_tools` must match the built-in tool allowlist. `task_complete` cannot be disabled.