Compare commits

...

9 Commits

Author SHA1 Message Date
space 75a2600e4d feat. Revision of Agent Lopp & major clean-up
CI / test (push) Successful in 8s
2026-06-04 18:00:26 +02:00
space 1f606475c5 Fix Windows tool behavior in tests
CI / test (push) Successful in 7s
2026-06-04 15:20:38 +02:00
space 643320a9de feat. Finishing sound & agent loop fixes
CI / test (push) Failing after 33s
2026-06-04 15:16:31 +02:00
Space-Banane 97641b354e feat: add CI & pytest fix to the TODO list
CI / test (push) Failing after 11s
2026-05-31 21:47:22 +02:00
Space-Banane 2495b6d62f refactor: remove redundant native control checks from window and UI element tools
CI / test (push) Failing after 9s
2026-05-31 21:46:37 +02:00
Space-Banane f0058d1057 feat: add support for Windows-only tools and enhance platform checks
CI / test (push) Failing after 10s
2026-05-31 21:02:56 +02:00
Space-Banane d514fe161c docs: update context compaction prompt with observe-decide-act-verify loop
CI / test (push) Failing after 8s
2026-05-31 20:52:49 +02:00
Space-Banane 4123765aba Commit remaining workspace updates
CI / test (push) Failing after 8s
2026-05-31 20:43:36 +02:00
Space-Banane 79c9e98842 Switch backend startup to interactive session 2026-05-31 20:43:36 +02:00
31 changed files with 5416 additions and 1367 deletions
+23
View File
@@ -0,0 +1,23 @@
---
name: death-is-soon-wrap-up
description: Use when the user says "death is soon, wrap up" or wants the conversation closed out with a final note, concise summary, goodbye, and instruction to reset the chat.
---
# Death Is Soon Wrap Up
## Purpose
Use this skill to close a conversation cleanly and intentionally.
## Wrap-Up Flow
1. Note down anything that has a place to live.
1. If there are notes, tasks, summaries, or memory slots available, capture the important details there before ending.
1. Give the user one last short summary of the situation, decisions, and any next steps.
1. Say goodbye plainly.
1. Tell the user to reset the chat.
## Output Style
Keep the closing message short and calm.
Do not reopen the discussion after the goodbye unless the user asks for more.
+43 -65
View File
@@ -1,7 +1,7 @@
# ScreenJob
ScreenJob is an autonomous desktop-and-terminal execution service.
It lets an LLM use controlled local tools (screen, click, type, shell) to complete GUI-heavy tasks on a real computer.
It lets an LLM use controlled local tools (screen, mouse, keyboard, clipboard, shell) to complete GUI-heavy tasks on a real computer.
## What It Solves
@@ -12,10 +12,12 @@ It lets an LLM use controlled local tools (screen, click, type, shell) to comple
- Returns structured agent output as:
- `return`: human-readable completion message
- `data`: structured payload (for example command output)
- Cleans up old runs and history on startup (default retention: 7 days)
## Core Features
- Tool-based agent loop (`execute_command`, `see_screen`, `enhance`, `click`, `type`, `press_key`, `sleep`, `task_complete`)
- Hybrid control model: screenshot grounding plus Windows-native window, dialog, and UI-element helpers when available
- Tool-based agent loop (`execute_command`, `enhance`, `list_windows`, `find_window`, `focus_window`, `close_window`, `wait_for_window`, `wait_for_focus_change`, `detect_dialog`, `dialog_action`, `dialog_set_filename`, `wait_for_dialog_close`, `list_ui_elements`, `invoke_ui_element`, `set_ui_element_value`, `select_ui_element`, `wait_for_ui_element`, `click`, `scroll`, `drag`, `move_mouse`, `type`, `press_key`, `clipboard_get`, `clipboard_set`, `get_cursor_position`, `sleep`, `task_complete`)
- Safety pre-check with override support
- Per-job tool disable list
- Live/final usage and cost estimates
@@ -72,6 +74,7 @@ SCREENJOB_SAFETY_MODEL=gpt-5.4-mini
SCREENJOB_HOST=127.0.0.1
SCREENJOB_PORT=8787
DISABLE_UI=false
SCREENJOB_RETENTION_DAYS=7
```
## Usage
@@ -82,16 +85,11 @@ DISABLE_UI=false
python main.py run "Open amazon.de and go to my orders"
```
CLI JSON output includes both legacy and structured fields:
CLI JSON output:
```json
{
"completed": true,
"result": "Task completed successfully",
"response": {
"return": "Task completed successfully",
"data": "file1.txt\nfile2.txt"
},
"return": "Task completed successfully",
"data": "file1.txt\nfile2.txt"
}
@@ -109,77 +107,43 @@ Or use the PowerShell launcher:
.\start_backend.ps1
```
### Windows Service
### Backend Startup
Run these from an elevated PowerShell session (Run as Administrator):
Requires .NET SDK 10+ (installer publishes a native service host executable).
For screenshot-driven automation, start the backend in the logged-in user session.
That gives `pyautogui` access to the interactive desktop, which Windows services do not.
If you previously installed the legacy service, remove it once from an elevated PowerShell session with `.\uninstall_backend_service.ps1`.
Install and start at boot:
Install a sign-in launcher for the current user:
```powershell
.\install_backend_service.ps1 -ForceReinstall -StartAfterInstall -DelayedAutoStart
.\install_backend_service.ps1
```
Check status:
Install it for all users:
```powershell
Get-Service -Name ScreenJobBackend
.\install_backend_service.ps1 -AllUsers
```
Stop/start manually:
Start it immediately after installing:
```powershell
Stop-Service -Name ScreenJobBackend
Start-Service -Name ScreenJobBackend
.\install_backend_service.ps1 -StartNow
```
Uninstall:
Remove the launcher:
```powershell
.\uninstall_backend_service.ps1
```
Service logs are written to:
```text
screenjob_runs/service/backend-service.stdout.log
screenjob_runs/service/backend-service.stderr.log
```
### System Tray Icon (Windows)
Start tray icon now:
The launcher runs `start_backend.ps1` hidden via `start_backend_hidden.vbs`.
If you need to start the backend manually, run:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -STA -File .\screenjob_tray.ps1
.\start_backend.ps1
```
Install startup shortcut (current user):
```powershell
.\install_tray_startup_shortcut.ps1
```
Install startup shortcut for all users:
```powershell
.\install_tray_startup_shortcut.ps1 -AllUsers
```
Remove startup shortcut:
```powershell
.\install_tray_startup_shortcut.ps1 -Remove
```
Tray menu actions:
- Refresh service status
- Start/Stop/Restart service (prompts for admin/UAC)
- Open dashboard URL from `.env` `SCREENJOB_HOST` / `SCREENJOB_PORT`
- Open service logs folder
- Exit tray icon process
Auth for all API routes:
- `Authorization: Bearer <SCREENJOB_TOKEN>`
@@ -194,6 +158,11 @@ Auth for all API routes:
{
"job": "run \"ls -a\" in C:/Users/username/Documents and return output",
"model": "gpt-5.4-mini",
"native_automation_mode": "prefer",
"dialog_timeout_seconds": 12,
"focus_timeout_seconds": 8,
"ui_element_timeout_seconds": 8,
"max_retries_per_surface": 3,
"disabled_tools": [],
"safety_override": false
}
@@ -216,10 +185,8 @@ Response:
Each job payload includes:
- `result` (compat string)
- `response.return`
- `response.data`
- top-level `return` and `data` aliases
- `return`
- `data`
### Monitoring UI
@@ -227,27 +194,38 @@ Each job payload includes:
- Read-only dashboard (no run controls)
- Requires token input
- Live updates via `/ws`
- Analytics dashboards for success rate by objective category and daily averages
- Analytics dashboard for success by objective category
- Set `DISABLE_UI=true` to disable UI
### Analytics API
- `GET /api/analytics`
- Returns objective-category success rates plus average steps/cost over time
- Returns objective-category success rates plus average steps/cost by category
## Agent Instructions (Practical)
- Prefer `execute_command` for deterministic actions (opening URLs, filesystem checks).
- Use `see_screen` before UI interaction.
- First classify the current Windows surface, then choose the control channel.
- Prefer native window/dialog/element tools for focus changes, file pickers, modal confirmations, and browser-owned dialogs when available.
- Use `enhance` for screen observation. Call it with no coordinate for a full-screen grid capture.
- Use `enhance` before clicking small/ambiguous targets; prefer `region="small"` for compact controls.
- Use `enhance` `mode="text"` for tiny labels/text, or `mode="ui"` for general UI.
- Optionally set `enhance` `scale` (2-6) for tighter zoom control.
- Use `list_windows`, `find_window`, `focus_window`, and `wait_for_focus_change` instead of blind Alt+Tab retries.
- Use `detect_dialog`, `dialog_set_filename`, `dialog_action`, and `wait_for_dialog_close` for native open/save/confirm flows.
- Use `list_ui_elements`, `invoke_ui_element`, `set_ui_element_value`, `select_ui_element`, and `wait_for_ui_element` when controls are exposed natively.
- Use `press_key` for non-text keys (Enter, Tab, arrows, Escape).
- For shortcuts, use one `press_key` call with combo syntax (example: `win+r`).
- Use `click` offsets via `offset_up/down/left/right` and optional `sleep_after_seconds`.
- Use `click` offsets via `offset_up/down/left/right`; set `button` and `click_count` there instead of inventing one-off click tools.
- Use `move_mouse` when you need hover-only behavior and `drag` for slider, selection, or window moves.
- Use `scroll` for vertical navigation; positive amounts scroll up and negative amounts scroll down.
- Use `clipboard_get` / `clipboard_set` for copy-paste workflows, `get_cursor_position` for cursor inspection, and rely on the injected foreground-window context before interacting with uncertain focus.
- If native automation is unavailable or disabled, ScreenJob falls back to screenshots plus mouse/keyboard control and emits fallback events.
- When done, call:
- `task_complete(return="...", data=...)`
- Before `task_complete`, verify expected on-screen content with `see_screen` (and `enhance` if needed), and include an `observed_result` summary in `data`.
- Before `task_complete`, verify expected on-screen content with the latest retained screen and `enhance` if needed, and include an `observed_result` summary in `data`.
Per-job `disabled_tools` must match the built-in tool allowlist. `task_complete` cannot be disabled.
`data` should contain useful structured output for the requester (text, object, list, etc.).
+8
View File
@@ -6,8 +6,10 @@ ScreenJob lets an agent execute tasks that require a real desktop UI plus termin
## Main Features
- Hybrid control model: screenshot grounding plus Windows-native window/dialog/element helpers when available
- Screen perception (`see_screen`, `enhance`)
- Mouse/keyboard control (`click`, `type`, `press_key`)
- Native window/dialog control (`list_windows`, `find_window`, `focus_window`, `detect_dialog`, `dialog_action`, `dialog_set_filename`, `list_ui_elements`)
- Terminal execution (`execute_command`, `sleep`)
- Structured completion payload (`task_complete(return=..., data=...)`)
- Safety gate, auth, history, and live monitoring
@@ -45,6 +47,12 @@ Enhance-first click rule:
- Optional zoom control: set `scale` from `2` to `6` (defaults are tuned by region).
- After checking the enhanced image, click using the same target coordinate (or a small directional offset if needed).
Windows-native routing rule:
- First classify whether the current surface is a normal app window, browser window, `#32770` dialog, Explorer file picker, or another system surface.
- Prefer native window/dialog/element tools for focus changes, save/open dialogs, modal confirmations, and exposed controls.
- Fall back to screenshots plus mouse/keyboard only when native automation is unavailable or the UI is custom-drawn.
Verification rule:
- Before `task_complete`, verify actual on-screen content matches the expected outcome.
+59 -100
View File
@@ -1,125 +1,84 @@
[CmdletBinding(SupportsShouldProcess = $true)]
param(
[string]$ServiceName = "ScreenJobBackend",
[string]$DisplayName = "ScreenJob Backend",
[string]$Description = "Runs the ScreenJob backend (start_backend.ps1) as a Windows service.",
[ValidateSet("Automatic", "Manual", "Disabled")]
[string]$StartupType = "Automatic",
[switch]$DelayedAutoStart,
[switch]$ForceReinstall,
[switch]$StartAfterInstall
[switch]$Remove,
[switch]$AllUsers,
[switch]$StartNow
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
$scriptDir = Split-Path -Parent $PSCommandPath
$backendScript = Join-Path $scriptDir "start_backend.ps1"
$vbsLauncher = Join-Path $scriptDir "start_backend_hidden.vbs"
$shortcutName = "ScreenJob Backend.lnk"
if (-not (Test-Path -LiteralPath $backendScript)) {
throw "Backend launcher script not found: $backendScript"
}
if (-not (Test-Path -LiteralPath $vbsLauncher)) {
throw "Hidden backend launcher file not found: $vbsLauncher"
}
function Test-IsAdministrator {
$identity = [Security.Principal.WindowsIdentity]::GetCurrent()
$principal = New-Object Security.Principal.WindowsPrincipal($identity)
return $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
}
if (-not (Test-IsAdministrator)) {
throw "Run this script from an elevated PowerShell session (Run as Administrator)."
}
$scriptDir = Split-Path -Parent $PSCommandPath
$backendScript = Join-Path $scriptDir "start_backend.ps1"
if (-not (Test-Path -LiteralPath $backendScript)) {
throw "Backend launcher script not found: $backendScript"
}
$projectFile = Join-Path $scriptDir "service_host\ScreenJob.WindowsServiceHost\ScreenJob.WindowsServiceHost.csproj"
if (-not (Test-Path -LiteralPath $projectFile)) {
throw "Windows service host project not found: $projectFile"
}
$dotnetCmd = Get-Command dotnet -ErrorAction SilentlyContinue
if ($null -eq $dotnetCmd) {
throw "dotnet SDK was not found in PATH. Install .NET SDK 10+ and retry."
}
$publishDir = Join-Path $scriptDir "service_host\publish"
$serviceExe = Join-Path $publishDir "ScreenJob.WindowsServiceHost.exe"
$logDir = Join-Path $scriptDir "screenjob_runs\service"
$existingService = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($null -ne $existingService) {
if (-not $ForceReinstall) {
throw "Service '$ServiceName' already exists. Re-run with -ForceReinstall to replace it."
}
if ($PSCmdlet.ShouldProcess($ServiceName, "Remove existing service")) {
if ($existingService.Status -ne "Stopped") {
Stop-Service -Name $ServiceName -Force -ErrorAction Stop
}
& sc.exe delete $ServiceName | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to delete existing service '$ServiceName' (sc.exe exit code $LASTEXITCODE)."
}
$deadline = (Get-Date).AddSeconds(15)
while ((Get-Date) -lt $deadline) {
$stillThere = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($null -eq $stillThere) {
break
$legacyService = Get-Service -Name "ScreenJobBackend" -ErrorAction SilentlyContinue
if ($null -ne $legacyService) {
if (Test-IsAdministrator) {
if ($PSCmdlet.ShouldProcess("ScreenJobBackend", "Remove legacy Windows service")) {
if ($legacyService.Status -ne "Stopped") {
Stop-Service -Name "ScreenJobBackend" -Force -ErrorAction Stop
}
Start-Sleep -Milliseconds 300
& sc.exe delete ScreenJobBackend | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to delete legacy service 'ScreenJobBackend' (sc.exe exit code $LASTEXITCODE)."
}
Write-Host "Removed legacy Windows service: ScreenJobBackend"
}
} else {
Write-Warning "Legacy Windows service 'ScreenJobBackend' is still installed. Run uninstall_backend_service.ps1 from an elevated PowerShell session once to remove it."
}
}
if ($PSCmdlet.ShouldProcess($projectFile, "Publish Windows service host")) {
if (Test-Path -LiteralPath $serviceExe) {
Remove-Item -LiteralPath $serviceExe -Force -ErrorAction SilentlyContinue
}
& $dotnetCmd.Source publish `
$projectFile `
-c Release `
-r win-x64 `
--self-contained false `
-p:PublishSingleFile=true `
-o $publishDir
if ($LASTEXITCODE -ne 0) {
throw "dotnet publish failed with exit code $LASTEXITCODE."
}
$startupFolder = if ($AllUsers) {
[Environment]::GetFolderPath("CommonStartup")
} else {
[Environment]::GetFolderPath("Startup")
}
if (-not (Test-Path -LiteralPath $serviceExe)) {
throw "Published service executable not found: $serviceExe"
}
$shortcutPath = Join-Path $startupFolder $shortcutName
$binaryPath = "`"$serviceExe`" --backend-script `"$backendScript`" --working-dir `"$scriptDir`" --log-dir `"$logDir`""
if ($PSCmdlet.ShouldProcess($ServiceName, "Create service")) {
New-Service `
-Name $ServiceName `
-BinaryPathName $binaryPath `
-DisplayName $DisplayName `
-Description $Description `
-StartupType $StartupType
if ($StartupType -eq "Automatic" -and $DelayedAutoStart) {
& sc.exe config $ServiceName start= delayed-auto | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to enable delayed auto-start for '$ServiceName' (sc.exe exit code $LASTEXITCODE)."
if ($Remove) {
if (Test-Path -LiteralPath $shortcutPath) {
if ($PSCmdlet.ShouldProcess($shortcutPath, "Remove backend startup shortcut")) {
Remove-Item -LiteralPath $shortcutPath -Force
Write-Host "Removed backend startup shortcut: $shortcutPath"
}
} else {
Write-Host "No backend startup shortcut found at: $shortcutPath"
}
# Restart on first/second/subsequent failure after 5 seconds.
& sc.exe failure $ServiceName reset= 86400 actions= restart/5000/restart/5000/restart/5000 | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to configure failure actions for '$ServiceName' (sc.exe exit code $LASTEXITCODE)."
}
if ($StartAfterInstall) {
Start-Service -Name $ServiceName -ErrorAction Stop
}
return
}
Write-Host "Service '$ServiceName' installed successfully." -ForegroundColor Green
Write-Host "Check status with: Get-Service -Name $ServiceName"
Write-Host "View logs in: $logDir"
if ($PSCmdlet.ShouldProcess($shortcutPath, "Create backend startup shortcut")) {
$shell = New-Object -ComObject WScript.Shell
$shortcut = $shell.CreateShortcut($shortcutPath)
$shortcut.TargetPath = "$env:SystemRoot\System32\wscript.exe"
$shortcut.Arguments = '"' + $vbsLauncher + '"'
$shortcut.WorkingDirectory = $scriptDir
$shortcut.Description = "Launch ScreenJob backend at sign-in in the current user session."
$shortcut.Save()
Write-Host "Created backend startup shortcut: $shortcutPath"
}
if ($StartNow) {
Start-Process -FilePath "$env:SystemRoot\System32\wscript.exe" -ArgumentList @($vbsLauncher) -WorkingDirectory $scriptDir | Out-Null
Write-Host "Started backend launcher now."
}
-47
View File
@@ -1,47 +0,0 @@
[CmdletBinding(SupportsShouldProcess = $true)]
param(
[switch]$Remove,
[switch]$AllUsers
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
$scriptDir = Split-Path -Parent $PSCommandPath
$vbsLauncher = Join-Path $scriptDir "start_screenjob_tray_hidden.vbs"
$shortcutName = "ScreenJob Tray.lnk"
if (-not (Test-Path -LiteralPath $vbsLauncher)) {
throw "Launcher file not found: $vbsLauncher"
}
$startupFolder = if ($AllUsers) {
[Environment]::GetFolderPath("CommonStartup")
} else {
[Environment]::GetFolderPath("Startup")
}
$shortcutPath = Join-Path $startupFolder $shortcutName
if ($Remove) {
if (Test-Path -LiteralPath $shortcutPath) {
if ($PSCmdlet.ShouldProcess($shortcutPath, "Remove startup shortcut")) {
Remove-Item -LiteralPath $shortcutPath -Force
Write-Host "Removed startup shortcut: $shortcutPath"
}
} else {
Write-Host "No startup shortcut found at: $shortcutPath"
}
return
}
if ($PSCmdlet.ShouldProcess($shortcutPath, "Create startup shortcut")) {
$shell = New-Object -ComObject WScript.Shell
$shortcut = $shell.CreateShortcut($shortcutPath)
$shortcut.TargetPath = "$env:SystemRoot\System32\wscript.exe"
$shortcut.Arguments = '"' + $vbsLauncher + '"'
$shortcut.WorkingDirectory = $scriptDir
$shortcut.Description = "Launch ScreenJob tray icon at sign-in."
$shortcut.Save()
Write-Host "Created startup shortcut: $shortcutPath"
}
-307
View File
@@ -1,307 +0,0 @@
param(
[string]$ServiceName = "ScreenJobBackend"
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
$controlScript = Join-Path $scriptDir "tray_service_control.ps1"
$logsDir = Join-Path $scriptDir "screenjob_runs\service"
$defaultHost = "127.0.0.1"
$defaultPort = "8787"
function Read-EnvConfig {
param([string]$EnvFilePath)
$result = @{}
if (-not (Test-Path -LiteralPath $EnvFilePath)) {
return $result
}
foreach ($line in Get-Content -Path $EnvFilePath) {
$trimmed = $line.Trim()
if ($trimmed.Length -eq 0 -or $trimmed.StartsWith("#")) {
continue
}
$parts = $trimmed.Split("=", 2)
if ($parts.Count -eq 2) {
$key = $parts[0].Trim()
$value = $parts[1].Trim()
if (($value.StartsWith('"') -and $value.EndsWith('"')) -or ($value.StartsWith("'") -and $value.EndsWith("'"))) {
$value = $value.Substring(1, $value.Length - 2)
}
$result[$key] = $value
}
}
return $result
}
function Get-ServiceStatusSafe {
param([string]$Name)
try {
$svc = Get-Service -Name $Name -ErrorAction Stop
return $svc.Status.ToString()
} catch {
return "NotInstalled"
}
}
function Invoke-ServiceActionElevated {
param(
[Parameter(Mandatory = $true)][string]$Action,
[Parameter(Mandatory = $true)][string]$Name
)
if (-not (Test-Path -LiteralPath $controlScript)) {
[System.Windows.Forms.MessageBox]::Show(
"Missing control script: $controlScript",
"ScreenJob Tray",
[System.Windows.Forms.MessageBoxButtons]::OK,
[System.Windows.Forms.MessageBoxIcon]::Error
) | Out-Null
return
}
$argList = @(
"-NoProfile",
"-ExecutionPolicy", "Bypass",
"-File", "`"$controlScript`"",
"-Action", $Action,
"-ServiceName", $Name
)
try {
Start-Process -FilePath "powershell.exe" -ArgumentList $argList -Verb RunAs -WindowStyle Hidden | Out-Null
} catch {
# User canceled UAC prompt or launch failed.
}
}
function Get-DashboardUrl {
$envFile = Join-Path $scriptDir ".env"
$envVars = Read-EnvConfig -EnvFilePath $envFile
$dashboardHost = $defaultHost
$dashboardPort = $defaultPort
if ($envVars.ContainsKey("SCREENJOB_HOST") -and -not [string]::IsNullOrWhiteSpace($envVars["SCREENJOB_HOST"])) {
$dashboardHost = $envVars["SCREENJOB_HOST"]
}
if ($envVars.ContainsKey("SCREENJOB_PORT") -and -not [string]::IsNullOrWhiteSpace($envVars["SCREENJOB_PORT"])) {
$dashboardPort = $envVars["SCREENJOB_PORT"]
}
$connectHost = Resolve-ConnectHost -ConfiguredHost $dashboardHost
return "http://{0}:{1}/" -f $connectHost, $dashboardPort
}
function Resolve-ConnectHost {
param([string]$ConfiguredHost)
if ([string]::IsNullOrWhiteSpace($ConfiguredHost)) {
return "127.0.0.1"
}
switch ($ConfiguredHost.Trim().ToLowerInvariant()) {
"0.0.0.0" { return "127.0.0.1" }
"::" { return "127.0.0.1" }
"*" { return "127.0.0.1" }
default { return $ConfiguredHost }
}
}
function Get-HealthCheckHosts {
param([string]$ConfiguredHost)
if ([string]::IsNullOrWhiteSpace($ConfiguredHost)) {
return @("127.0.0.1", "localhost")
}
$normalized = $ConfiguredHost.Trim().ToLowerInvariant()
switch ($normalized) {
"0.0.0.0" { return @("127.0.0.1", "localhost", "::1") }
"::" { return @("127.0.0.1", "localhost", "::1") }
"*" { return @("127.0.0.1", "localhost", "::1") }
default { return @($ConfiguredHost) }
}
}
function Test-TcpEndpoint {
param(
[Parameter(Mandatory = $true)][string]$HostName,
[Parameter(Mandatory = $true)][int]$Port,
[int]$TimeoutMs = 1200
)
$client = New-Object System.Net.Sockets.TcpClient
try {
$async = $client.BeginConnect($HostName, $Port, $null, $null)
$connected = $async.AsyncWaitHandle.WaitOne($TimeoutMs, $false)
if (-not $connected) {
return $false
}
$client.EndConnect($async) | Out-Null
return $true
} catch {
return $false
} finally {
$client.Dispose()
}
}
function Get-BackendReachability {
$envFile = Join-Path $scriptDir ".env"
$envVars = Read-EnvConfig -EnvFilePath $envFile
$configuredHost = $defaultHost
$configuredPort = $defaultPort
if ($envVars.ContainsKey("SCREENJOB_HOST") -and -not [string]::IsNullOrWhiteSpace($envVars["SCREENJOB_HOST"])) {
$configuredHost = $envVars["SCREENJOB_HOST"]
}
if ($envVars.ContainsKey("SCREENJOB_PORT") -and -not [string]::IsNullOrWhiteSpace($envVars["SCREENJOB_PORT"])) {
$configuredPort = $envVars["SCREENJOB_PORT"]
}
$portNumber = 8787
[void][int]::TryParse([string]$configuredPort, [ref]$portNumber)
$hostsToTry = Get-HealthCheckHosts -ConfiguredHost $configuredHost
foreach ($candidateHost in $hostsToTry) {
if (Test-TcpEndpoint -HostName $candidateHost -Port $portNumber) {
return $true
}
}
return $false
}
function Update-TrayState {
param(
[System.Windows.Forms.NotifyIcon]$NotifyIcon,
[System.Windows.Forms.ToolStripMenuItem]$StatusItem,
[string]$Name
)
$status = Get-ServiceStatusSafe -Name $Name
$isBackendReachable = Get-BackendReachability
$displayStatus = $status
if ($status -eq "Running" -and -not $isBackendReachable) {
$displayStatus = "Running (Backend Down)"
} elseif ($status -eq "Stopped" -and $isBackendReachable) {
$displayStatus = "Stopped (Backend Up)"
} elseif ($status -eq "NotInstalled" -and $isBackendReachable) {
$displayStatus = "NotInstalled (Backend Up)"
}
$StatusItem.Text = "Status: $displayStatus"
switch ($displayStatus) {
"Running" {
$NotifyIcon.Icon = [System.Drawing.SystemIcons]::Information
}
"Stopped" {
$NotifyIcon.Icon = [System.Drawing.SystemIcons]::Warning
}
default {
$NotifyIcon.Icon = [System.Drawing.SystemIcons]::Error
}
}
$tooltip = "ScreenJob Backend: $displayStatus"
if ($tooltip.Length -gt 63) {
$tooltip = $tooltip.Substring(0, 63)
}
$NotifyIcon.Text = $tooltip
}
$appContext = New-Object System.Windows.Forms.ApplicationContext
$notifyIcon = New-Object System.Windows.Forms.NotifyIcon
$notifyIcon.Visible = $false
$menu = New-Object System.Windows.Forms.ContextMenuStrip
$statusItem = New-Object System.Windows.Forms.ToolStripMenuItem "Status: Unknown"
$statusItem.Enabled = $false
$refreshItem = New-Object System.Windows.Forms.ToolStripMenuItem "Refresh Status"
$refreshItem.Add_Click({
Update-TrayState -NotifyIcon $notifyIcon -StatusItem $statusItem -Name $ServiceName
})
$startItem = New-Object System.Windows.Forms.ToolStripMenuItem "Start Service (Admin)"
$startItem.Add_Click({
Invoke-ServiceActionElevated -Action "start" -Name $ServiceName
})
$stopItem = New-Object System.Windows.Forms.ToolStripMenuItem "Stop Service (Admin)"
$stopItem.Add_Click({
Invoke-ServiceActionElevated -Action "stop" -Name $ServiceName
})
$restartItem = New-Object System.Windows.Forms.ToolStripMenuItem "Restart Service (Admin)"
$restartItem.Add_Click({
Invoke-ServiceActionElevated -Action "restart" -Name $ServiceName
})
$dashboardItem = New-Object System.Windows.Forms.ToolStripMenuItem "Open Dashboard"
$dashboardItem.Add_Click({
$url = Get-DashboardUrl
Start-Process $url | Out-Null
})
$logsItem = New-Object System.Windows.Forms.ToolStripMenuItem "Open Service Logs"
$logsItem.Add_Click({
if (-not (Test-Path -LiteralPath $logsDir)) {
New-Item -ItemType Directory -Path $logsDir -Force | Out-Null
}
Start-Process explorer.exe $logsDir | Out-Null
})
$openFolderItem = New-Object System.Windows.Forms.ToolStripMenuItem "Open Project Folder"
$openFolderItem.Add_Click({
Start-Process explorer.exe $scriptDir | Out-Null
})
$exitItem = New-Object System.Windows.Forms.ToolStripMenuItem "Exit Tray"
$exitItem.Add_Click({
$refreshTimer.Stop()
$notifyIcon.Visible = $false
$notifyIcon.Dispose()
$menu.Dispose()
$appContext.ExitThread()
})
[void]$menu.Items.Add($statusItem)
[void]$menu.Items.Add($refreshItem)
[void]$menu.Items.Add((New-Object System.Windows.Forms.ToolStripSeparator))
[void]$menu.Items.Add($startItem)
[void]$menu.Items.Add($stopItem)
[void]$menu.Items.Add($restartItem)
[void]$menu.Items.Add((New-Object System.Windows.Forms.ToolStripSeparator))
[void]$menu.Items.Add($dashboardItem)
[void]$menu.Items.Add($logsItem)
[void]$menu.Items.Add($openFolderItem)
[void]$menu.Items.Add((New-Object System.Windows.Forms.ToolStripSeparator))
[void]$menu.Items.Add($exitItem)
$notifyIcon.ContextMenuStrip = $menu
$notifyIcon.Visible = $true
$notifyIcon.Add_DoubleClick({
$url = Get-DashboardUrl
Start-Process $url | Out-Null
})
$refreshTimer = New-Object System.Windows.Forms.Timer
$refreshTimer.Interval = 5000
$refreshTimer.Add_Tick({
Update-TrayState -NotifyIcon $notifyIcon -StatusItem $statusItem -Name $ServiceName
})
Update-TrayState -NotifyIcon $notifyIcon -StatusItem $statusItem -Name $ServiceName
$refreshTimer.Start()
[System.Windows.Forms.Application]::Run($appContext)
@@ -1,138 +0,0 @@
using System.Diagnostics;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace ScreenJob.WindowsServiceHost;
internal sealed class BackendProcessService : BackgroundService
{
private readonly ILogger<BackendProcessService> _logger;
private readonly ServiceOptions _options;
private readonly object _logLock = new();
private Process? _backendProcess;
private string _stdoutLogPath = string.Empty;
private string _stderrLogPath = string.Empty;
public BackendProcessService(ILogger<BackendProcessService> logger, ServiceOptions options)
{
_logger = logger;
_options = options;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
Directory.CreateDirectory(_options.LogDirectory);
_stdoutLogPath = Path.Combine(_options.LogDirectory, "backend-service.stdout.log");
_stderrLogPath = Path.Combine(_options.LogDirectory, "backend-service.stderr.log");
LogStdOut("Service host starting backend process.");
LogStdOut($"Script: {_options.BackendScriptPath}");
LogStdOut($"Working directory: {_options.WorkingDirectory}");
var powershellPath = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.Windows),
"System32",
"WindowsPowerShell",
"v1.0",
"powershell.exe");
var startInfo = new ProcessStartInfo
{
FileName = powershellPath,
Arguments = $"-NoProfile -ExecutionPolicy Bypass -File \"{_options.BackendScriptPath}\"",
WorkingDirectory = _options.WorkingDirectory,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
CreateNoWindow = true
};
_backendProcess = new Process { StartInfo = startInfo };
if (!_backendProcess.Start())
{
throw new InvalidOperationException("Failed to start backend process.");
}
LogStdOut($"Backend process started with PID {_backendProcess.Id}.");
_logger.LogInformation("Backend process started with PID {Pid}.", _backendProcess.Id);
var stdoutPump = PumpStreamAsync(_backendProcess.StandardOutput, LogStdOut, stoppingToken);
var stderrPump = PumpStreamAsync(_backendProcess.StandardError, LogStdErr, stoppingToken);
try
{
await _backendProcess.WaitForExitAsync(stoppingToken);
var exitCode = _backendProcess.ExitCode;
LogStdErr($"Backend process exited unexpectedly with code {exitCode}.");
_logger.LogError("Backend process exited unexpectedly with code {ExitCode}.", exitCode);
Environment.ExitCode = exitCode == 0 ? 1 : exitCode;
throw new InvalidOperationException(
$"Backend process ended unexpectedly. Service host exit code: {Environment.ExitCode}.");
}
catch (OperationCanceledException)
{
LogStdOut("Service stop requested.");
}
finally
{
await Task.WhenAll(stdoutPump, stderrPump);
}
}
public override async Task StopAsync(CancellationToken cancellationToken)
{
if (_backendProcess is { HasExited: false })
{
try
{
LogStdOut("Stopping backend process.");
_backendProcess.Kill(entireProcessTree: true);
}
catch (Exception ex)
{
LogStdErr($"Failed to stop backend process cleanly: {ex.Message}");
_logger.LogError(ex, "Failed to stop backend process cleanly.");
}
}
await base.StopAsync(cancellationToken);
}
private async Task PumpStreamAsync(
StreamReader reader,
Action<string> sink,
CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var line = await reader.ReadLineAsync();
if (line is null)
{
break;
}
sink(line);
}
}
private void LogStdOut(string message)
{
WriteLog(_stdoutLogPath, message);
}
private void LogStdErr(string message)
{
WriteLog(_stderrLogPath, message);
}
private void WriteLog(string path, string message)
{
var stamp = DateTimeOffset.Now.ToString("yyyy-MM-dd HH:mm:ss");
var line = $"[{stamp}] {message}{Environment.NewLine}";
lock (_logLock)
{
File.AppendAllText(path, line);
}
}
}
@@ -1,18 +0,0 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using ScreenJob.WindowsServiceHost;
var options = ServiceOptions.Parse(args);
Host.CreateDefaultBuilder(args)
.UseWindowsService(serviceOptions =>
{
serviceOptions.ServiceName = "ScreenJobBackend";
})
.ConfigureServices(services =>
{
services.AddSingleton(options);
services.AddHostedService<BackendProcessService>();
})
.Build()
.Run();
@@ -1,12 +0,0 @@
<Project Sdk="Microsoft.NET.Sdk.Worker">
<PropertyGroup>
<TargetFramework>net10.0-windows</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<OutputType>Exe</OutputType>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting.WindowsServices" Version="10.0.0" />
</ItemGroup>
</Project>
@@ -1,77 +0,0 @@
namespace ScreenJob.WindowsServiceHost;
internal sealed record ServiceOptions(
string BackendScriptPath,
string WorkingDirectory,
string LogDirectory)
{
public static ServiceOptions Parse(string[] args)
{
var map = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
for (var i = 0; i < args.Length; i++)
{
var raw = args[i];
if (!raw.StartsWith("--", StringComparison.Ordinal))
{
continue;
}
var key = raw[2..];
if (string.IsNullOrWhiteSpace(key))
{
continue;
}
if (i + 1 < args.Length && !args[i + 1].StartsWith("--", StringComparison.Ordinal))
{
map[key] = args[++i];
}
else
{
map[key] = "true";
}
}
if (!map.TryGetValue("backend-script", out var backendScript) || string.IsNullOrWhiteSpace(backendScript))
{
throw new ArgumentException("Missing required argument: --backend-script <absolute-path-to-start_backend.ps1>.");
}
if (!Path.IsPathRooted(backendScript))
{
throw new ArgumentException("The --backend-script value must be an absolute path.");
}
if (!File.Exists(backendScript))
{
throw new FileNotFoundException("Backend script not found.", backendScript);
}
if (!map.TryGetValue("working-dir", out var workingDir) || string.IsNullOrWhiteSpace(workingDir))
{
workingDir = Path.GetDirectoryName(backendScript)
?? throw new ArgumentException("Could not resolve working directory from backend script path.");
}
if (!Path.IsPathRooted(workingDir))
{
throw new ArgumentException("The --working-dir value must be an absolute path.");
}
if (!map.TryGetValue("log-dir", out var logDir) || string.IsNullOrWhiteSpace(logDir))
{
logDir = Path.Combine(workingDir, "screenjob_runs", "service");
}
if (!Path.IsPathRooted(logDir))
{
throw new ArgumentException("The --log-dir value must be an absolute path.");
}
return new ServiceOptions(
Path.GetFullPath(backendScript),
Path.GetFullPath(workingDir),
Path.GetFullPath(logDir));
}
}
+3606 -161
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -30,6 +30,8 @@ def main(argv: list[str] | None = None) -> int:
print(" OPENAI_API_KEY=...")
print(" SCREENJOB_TOKEN=...")
print(" DISABLE_UI=true|false (optional)")
print(" SCREENJOB_PROHIBITED_KEY_COMBOS=ctrl+shift+s,alt+f4 (optional)")
print(" SCREENJOB_RETENTION_DAYS=7 (optional)")
return 0
server.main()
return 0
+121 -28
View File
@@ -2,13 +2,18 @@ from __future__ import annotations
import argparse
import json
import signal
import sys
import threading
from pathlib import Path
from .agent import normalize_disabled_tools
from .config import load_app_config
from .models import RuntimeOptions
from .runtime import create_openai_client, run_job
from .safety import assess_task_safety
from .storage import HistoryDB
from .utils import cleanup_old_run_artifacts
def build_parser() -> argparse.ArgumentParser:
@@ -40,8 +45,56 @@ def build_parser() -> argparse.ArgumentParser:
default=4,
help="Compact model context every N steps to decay old screenshots (0 disables).",
)
parser.add_argument(
"--max-visual-context-images",
type=int,
default=None,
help="Maximum recent screen images retained in model-visible context during rebases. Defaults to SCREENJOB_MAX_VISUAL_CONTEXT_IMAGES or 3.",
)
parser.add_argument(
"--native-automation-mode",
choices=["off", "prefer"],
default="prefer",
help="How strongly the agent should prefer Windows-native automation helpers over pixel fallback.",
)
parser.add_argument(
"--dialog-timeout-seconds",
type=float,
default=12.0,
help="Timeout for dialog-oriented waits and retries.",
)
parser.add_argument(
"--focus-timeout-seconds",
type=float,
default=8.0,
help="Timeout for focus-change waits and verification.",
)
parser.add_argument(
"--ui-element-timeout-seconds",
type=float,
default=8.0,
help="Timeout for native UI element lookup waits.",
)
parser.add_argument(
"--max-retries-per-surface",
type=int,
default=3,
help="Maximum repeated retries on the same classified window/dialog surface before the agent must pivot.",
)
parser.add_argument(
"--pretty-logs",
action=argparse.BooleanOptionalAction,
default=False,
help="Emit expanded multi-line tool call/result logs for easier debugging.",
)
parser.add_argument("--disable-tool", action="append", default=[], help="Disable a tool by name.")
parser.add_argument("--skip-safety-check", action="store_true", help="Bypass pre-flight safety check.")
parser.add_argument(
"--skip-safety-check",
"--skip-safety-chec",
dest="skip_safety_check",
action="store_true",
help="Bypass pre-flight safety check.",
)
parser.add_argument("--no-failsafe", action="store_true", help="Disable PyAutoGUI fail-safe.")
return parser
@@ -56,8 +109,14 @@ def main(argv: list[str] | None = None) -> int:
print("ERROR: Missing OPENAI_API_KEY in environment/.env", file=sys.stderr)
return 2
HistoryDB(config.db_path).prune_older_than(config.retention_days)
cleanup_old_run_artifacts(config.runs_dir, config.retention_days)
model = args.model or config.default_model
disabled_tools = sorted({str(x).strip() for x in args.disable_tool if str(x).strip()})
try:
disabled_tools = normalize_disabled_tools(args.disable_tool)
except ValueError as exc:
parser.error(str(exc))
if not args.skip_safety_check:
safety_client = create_openai_client(config.openai_api_key)
@@ -72,8 +131,6 @@ def main(argv: list[str] | None = None) -> int:
json.dumps(
{
"completed": False,
"result": f"Blocked by safety check: {reason}",
"response": {"return": f"Blocked by safety check: {reason}", "data": parsed},
"return": f"Blocked by safety check: {reason}",
"data": parsed,
"safety": parsed,
@@ -92,34 +149,70 @@ def main(argv: list[str] | None = None) -> int:
click_pause=args.click_pause,
reasoning_effort=args.reasoning_effort,
screen_context_decay_steps=max(0, int(args.screen_context_decay_steps)),
disable_tools=set(disabled_tools),
)
try:
result, artifacts = run_job(
api_key=config.openai_api_key,
objective=args.job,
options=options,
runs_base=config.runs_dir,
no_failsafe=args.no_failsafe,
)
except KeyboardInterrupt:
print(json.dumps({"completed": False, "result": "Interrupted by user."}, ensure_ascii=False, indent=2))
return 130
except Exception as exc: # noqa: BLE001
print(
json.dumps(
{"completed": False, "result": f"Fatal error: {type(exc).__name__}: {exc}"},
ensure_ascii=False,
indent=2,
max_visual_context_images=max(
0,
int(
config.max_visual_context_images_default
if args.max_visual_context_images is None
else args.max_visual_context_images
),
file=sys.stderr,
)
return 1
),
native_automation_mode=args.native_automation_mode,
dialog_timeout_seconds=max(0.5, float(args.dialog_timeout_seconds)),
focus_timeout_seconds=max(0.5, float(args.focus_timeout_seconds)),
ui_element_timeout_seconds=max(0.5, float(args.ui_element_timeout_seconds)),
max_retries_per_surface=max(1, int(args.max_retries_per_surface)),
pretty_logs=bool(args.pretty_logs),
disable_tools=set(disabled_tools),
prohibited_key_combos=set(config.prohibited_key_combos),
)
cancel_event = threading.Event()
interrupt_state = {"count": 0}
previous_sigint_handler = signal.getsignal(signal.SIGINT)
def _handle_sigint(_signum, _frame):
interrupt_state["count"] += 1
cancel_event.set()
if interrupt_state["count"] == 1:
print("\nInterrupt received, cancelling run... Press Ctrl+C again to abort immediately.", file=sys.stderr)
return
raise KeyboardInterrupt
signal.signal(signal.SIGINT, _handle_sigint)
try:
try:
result, artifacts = run_job(
api_key=config.openai_api_key,
objective=args.job,
options=options,
runs_base=config.runs_dir,
no_failsafe=args.no_failsafe,
cancel_event=cancel_event,
)
except KeyboardInterrupt:
print(json.dumps({"completed": False, "return": "Interrupted by user.", "data": None}, ensure_ascii=False, indent=2))
return 130
except Exception as exc: # noqa: BLE001
print(
json.dumps(
{"completed": False, "return": f"Fatal error: {type(exc).__name__}: {exc}", "data": None},
ensure_ascii=False,
indent=2,
),
file=sys.stderr,
)
return 1
finally:
signal.signal(signal.SIGINT, previous_sigint_handler)
if result.cancelled:
usage = result.usage.to_dict()
estimated_cost = usage.get("estimated_cost_usd")
if estimated_cost is not None:
print(f"Estimated cost before cancel: ${float(estimated_cost):.6f}", file=sys.stderr)
payload = {
"completed": result.completed,
"result": result.return_message,
"response": {"return": result.return_message, "data": result.data},
"return": result.return_message,
"data": result.data,
"steps": result.steps,
+16 -1
View File
@@ -14,6 +14,13 @@ def _env_bool(name: str, default: bool = False) -> bool:
return raw.strip().lower() in {"1", "true", "yes", "on"}
def _env_csv(name: str) -> list[str]:
raw = os.getenv(name)
if raw is None:
return []
return [item.strip() for item in raw.split(",") if item.strip()]
@dataclass(frozen=True)
class AppConfig:
openai_api_key: str
@@ -25,6 +32,9 @@ class AppConfig:
port: int
runs_dir: Path
db_path: Path
max_visual_context_images_default: int = 3
prohibited_key_combos: tuple[str, ...] = ()
retention_days: int = 7
def load_app_config(cwd: Path) -> AppConfig:
@@ -38,6 +48,9 @@ def load_app_config(cwd: Path) -> AppConfig:
runs_dir = cwd / "screenjob_runs"
db_path = cwd / "screenjob.db"
disable_ui = _env_bool("DISABLE_UI", default=False)
max_visual_context_images_default = max(0, int(os.getenv("SCREENJOB_MAX_VISUAL_CONTEXT_IMAGES", "3").strip() or "3"))
prohibited_key_combos = tuple(_env_csv("SCREENJOB_PROHIBITED_KEY_COMBOS"))
retention_days = max(1, int(os.getenv("SCREENJOB_RETENTION_DAYS", "7").strip() or "7"))
return AppConfig(
openai_api_key=openai_api_key,
screenjob_token=screenjob_token,
@@ -48,5 +61,7 @@ def load_app_config(cwd: Path) -> AppConfig:
port=port,
runs_dir=runs_dir,
db_path=db_path,
max_visual_context_images_default=max_visual_context_images_default,
prohibited_key_combos=prohibited_key_combos,
retention_days=retention_days,
)
+8
View File
@@ -60,4 +60,12 @@ class RuntimeOptions:
click_pause: float = 0.10
reasoning_effort: str = "medium"
screen_context_decay_steps: int = 4
max_visual_context_images: int = 3
native_automation_mode: str = "prefer"
dialog_timeout_seconds: float = 12.0
focus_timeout_seconds: float = 8.0
ui_element_timeout_seconds: float = 8.0
max_retries_per_surface: int = 3
pretty_logs: bool = False
disable_tools: set[str] | None = None
prohibited_key_combos: set[str] | None = None
+24 -16
View File
@@ -20,6 +20,9 @@ else:
_PYAUTOGUI_IMPORT_ERROR = None
_DESKTOP_RUN_LOCK = threading.Lock()
def create_openai_client(api_key: str) -> OpenAI:
return OpenAI(api_key=api_key)
@@ -44,20 +47,25 @@ def run_job(
pyautogui.FAILSAFE = not no_failsafe
pyautogui.PAUSE = 0.05
artifacts = setup_artifacts(runs_base)
active_logger = logger or setup_logger(artifacts.log_file, verbose=True)
active_logger.info("ScreenJob booting. Artifacts: %s", str(artifacts.root_dir.resolve()))
active_logger.info("PyAutoGUI FAILSAFE=%s", pyautogui.FAILSAFE)
if not _DESKTOP_RUN_LOCK.acquire(blocking=False):
raise RuntimeError("Another ScreenJob run is already active on this desktop.")
try:
artifacts = setup_artifacts(runs_base)
active_logger = logger or setup_logger(artifacts.log_file, verbose=True)
active_logger.info("ScreenJob booting. Artifacts: %s", str(artifacts.root_dir.resolve()))
active_logger.info("PyAutoGUI FAILSAFE=%s", pyautogui.FAILSAFE)
client = create_openai_client(api_key)
agent = ScreenJobAgent(
client=client,
logger=active_logger,
artifacts=artifacts,
options=options,
cancel_event=cancel_event,
event_callback=event_callback,
)
result = agent.run(objective)
active_logger.info("Run finished. completed=%s elapsed=%.2fs", result.completed, result.ended_at - result.started_at)
return result, artifacts
client = create_openai_client(api_key)
agent = ScreenJobAgent(
client=client,
logger=active_logger,
artifacts=artifacts,
options=options,
cancel_event=cancel_event,
event_callback=event_callback,
)
result = agent.run(objective)
active_logger.info("Run finished. completed=%s elapsed=%.2fs", result.completed, result.ended_at - result.started_at)
return result, artifacts
finally:
_DESKTOP_RUN_LOCK.release()
+40 -15
View File
@@ -12,11 +12,12 @@ from fastapi.responses import FileResponse
from fastapi.responses import HTMLResponse, JSONResponse
from pydantic import BaseModel, Field
from .agent import normalize_disabled_tools
from .config import AppConfig, load_app_config
from .storage import HistoryDB
from .task_manager import JobManager
from .ui import monitoring_js_path, monitoring_page_html
from .utils import utc_now_iso
from .utils import cleanup_old_run_artifacts, utc_now_iso
class CreateJobRequest(BaseModel):
@@ -28,6 +29,13 @@ class CreateJobRequest(BaseModel):
click_pause: float = Field(0.10, ge=0.0, le=2.0)
reasoning_effort: str = Field("medium", pattern="^(low|medium|high)$")
screen_context_decay_steps: int = Field(4, ge=0, le=50)
max_visual_context_images: int | None = Field(default=None, ge=0, le=12)
native_automation_mode: str = Field("prefer", pattern="^(off|prefer)$")
dialog_timeout_seconds: float = Field(12.0, ge=0.5, le=120.0)
focus_timeout_seconds: float = Field(8.0, ge=0.5, le=120.0)
ui_element_timeout_seconds: float = Field(8.0, ge=0.5, le=120.0)
max_retries_per_surface: int = Field(3, ge=1, le=10)
pretty_logs: bool = False
disabled_tools: list[str] = Field(default_factory=list)
safety_override: bool = False
no_failsafe: bool = False
@@ -175,7 +183,7 @@ def _build_replay_payload(job_id: str, job: dict[str, Any], events: list[dict[st
width = _safe_int(size.get("width"))
height = _safe_int(size.get("height"))
is_fullscreen = (
str(payload.get("kind") or "") == "see_screen"
(bool(image_meta.get("full_screen")) or str(payload.get("kind") or "") == "see_screen")
and bool(image_meta.get("grid"))
and isinstance(width, int)
and isinstance(height, int)
@@ -254,6 +262,8 @@ def create_app(config: AppConfig | None = None) -> FastAPI:
raise RuntimeError("SCREENJOB_TOKEN is required in environment or .env.")
db = HistoryDB(app_config.db_path)
db.prune_older_than(app_config.retention_days)
cleanup_old_run_artifacts(app_config.runs_dir, app_config.retention_days)
ws_hub = _WebSocketHub()
manager = JobManager(config=app_config, db=db, broadcast=ws_hub.broadcast_from_thread)
@@ -297,19 +307,34 @@ def create_app(config: AppConfig | None = None) -> FastAPI:
@app.post("/api/jobs")
def create_job(payload: CreateJobRequest, _: None = Depends(require_token)) -> dict[str, str]:
job_id = manager.submit_job(
objective=payload.job,
model=payload.model,
max_steps=payload.max_steps,
command_timeout=payload.command_timeout,
type_interval=payload.type_interval,
click_pause=payload.click_pause,
reasoning_effort=payload.reasoning_effort,
screen_context_decay_steps=payload.screen_context_decay_steps,
disabled_tools=payload.disabled_tools,
safety_override=payload.safety_override,
no_failsafe=payload.no_failsafe,
)
try:
disabled_tools = normalize_disabled_tools(payload.disabled_tools)
job_id = manager.submit_job(
objective=payload.job,
model=payload.model,
max_steps=payload.max_steps,
command_timeout=payload.command_timeout,
type_interval=payload.type_interval,
click_pause=payload.click_pause,
reasoning_effort=payload.reasoning_effort,
screen_context_decay_steps=payload.screen_context_decay_steps,
max_visual_context_images=(
app_config.max_visual_context_images_default
if payload.max_visual_context_images is None
else payload.max_visual_context_images
),
native_automation_mode=payload.native_automation_mode,
dialog_timeout_seconds=payload.dialog_timeout_seconds,
focus_timeout_seconds=payload.focus_timeout_seconds,
ui_element_timeout_seconds=payload.ui_element_timeout_seconds,
max_retries_per_surface=payload.max_retries_per_surface,
pretty_logs=payload.pretty_logs,
disabled_tools=disabled_tools,
safety_override=payload.safety_override,
no_failsafe=payload.no_failsafe,
)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return {"job_id": job_id}
@app.get("/api/jobs")
+37 -52
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import json
import sqlite3
import threading
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any
@@ -217,6 +218,29 @@ class HistoryDB:
).fetchone()
return dict(totals) if totals else {}
def prune_older_than(self, retention_days: int) -> int:
if retention_days < 1:
return 0
cutoff = (datetime.now(timezone.utc) - timedelta(days=retention_days)).isoformat()
with self._lock, self._connect() as conn:
job_ids = [
row[0]
for row in conn.execute(
"""
SELECT job_id
FROM jobs
WHERE created_at < ? AND status IN ('completed', 'failed', 'cancelled')
""",
(cutoff,),
).fetchall()
]
if not job_ids:
return 0
conn.executemany("DELETE FROM job_events WHERE job_id = ?", ((job_id,) for job_id in job_ids))
conn.executemany("DELETE FROM jobs WHERE job_id = ?", ((job_id,) for job_id in job_ids))
conn.commit()
return len(job_ids)
def analytics(self) -> dict[str, Any]:
with self._connect() as conn:
rows = conn.execute(
@@ -227,17 +251,7 @@ class HistoryDB:
"""
).fetchall()
total_jobs = 0
finished_jobs = 0
completed_jobs = 0
failed_jobs = 0
cancelled_jobs = 0
steps_sum = 0
steps_count = 0
cost_sum = 0.0
cost_count = 0
by_category: dict[str, dict[str, Any]] = {}
by_day: dict[str, dict[str, Any]] = {}
def _bucket(target: dict[str, dict[str, Any]], key: str) -> dict[str, Any]:
bucket = target.setdefault(
@@ -258,57 +272,38 @@ class HistoryDB:
return bucket
for row in rows:
total_jobs += 1
status = str(row["status"] or "")
finished = status in _TERMINAL_STATUSES
completed = status == "completed"
objective = str(row["objective"] or "")
category = _objective_category(objective)
created_at = str(row["created_at"] or "")
day = created_at[:10] if len(created_at) >= 10 else created_at or "unknown"
category_bucket = _bucket(by_category, category)
day_bucket = _bucket(by_day, day)
for bucket in (category_bucket, day_bucket):
bucket["total_jobs"] += 1
category_bucket["total_jobs"] += 1
if not finished:
continue
finished_jobs += 1
if completed:
completed_jobs += 1
category_bucket["completed_jobs"] += 1
elif status == "failed":
failed_jobs += 1
category_bucket["failed_jobs"] += 1
elif status == "cancelled":
cancelled_jobs += 1
category_bucket["cancelled_jobs"] += 1
steps = row["steps"]
if steps is not None:
step_value = int(steps)
steps_sum += step_value
steps_count += 1
for bucket in (category_bucket, day_bucket):
bucket["steps_sum"] += step_value
bucket["steps_count"] += 1
category_bucket["steps_sum"] += step_value
category_bucket["steps_count"] += 1
estimated_cost = row["estimated_cost_usd"]
if estimated_cost is not None:
cost_value = float(estimated_cost)
cost_sum += cost_value
cost_count += 1
for bucket in (category_bucket, day_bucket):
bucket["cost_sum"] += cost_value
bucket["cost_count"] += 1
category_bucket["cost_sum"] += cost_value
category_bucket["cost_count"] += 1
for bucket in (category_bucket, day_bucket):
bucket["finished_jobs"] += 1
if completed:
bucket["completed_jobs"] += 1
elif status == "failed":
bucket["failed_jobs"] += 1
elif status == "cancelled":
bucket["cancelled_jobs"] += 1
category_bucket["finished_jobs"] += 1
def _finalize(bucket: dict[str, Any]) -> dict[str, Any]:
finished = bucket["finished_jobs"]
@@ -325,21 +320,10 @@ class HistoryDB:
}
category_rows = [_finalize(bucket) for bucket in by_category.values()]
category_rows.sort(key=lambda item: (-item["success_rate"], item["label"]))
day_rows = [_finalize(bucket) for bucket in by_day.values()]
day_rows.sort(key=lambda item: item["label"])
category_rows.sort(key=lambda item: item["label"])
return {
"total_jobs": total_jobs,
"finished_jobs": finished_jobs,
"completed_jobs": completed_jobs,
"failed_jobs": failed_jobs,
"cancelled_jobs": cancelled_jobs,
"success_rate": round((completed_jobs / finished_jobs) * 100, 2) if finished_jobs else 0.0,
"avg_steps": round(steps_sum / steps_count, 2) if steps_count else None,
"avg_cost_usd": round(cost_sum / cost_count, 6) if cost_count else None,
"by_category": category_rows,
"timeline": day_rows,
}
def _row_to_job(self, row: sqlite3.Row) -> dict[str, Any]:
@@ -348,6 +332,7 @@ class HistoryDB:
disabled_tools = json.loads(row["disabled_tools_json"]) if row["disabled_tools_json"] else []
except Exception:
disabled_tools = []
response = self._parse_response_payload(row["response_json"], row["result"])
return {
"job_id": row["job_id"],
"objective": row["objective"],
@@ -356,8 +341,8 @@ class HistoryDB:
"created_at": row["created_at"],
"started_at": row["started_at"],
"ended_at": row["ended_at"],
"result": row["result"],
"response": self._parse_response_payload(row["response_json"], row["result"]),
"return": response["return"],
"data": response["data"],
"error": row["error"],
"steps": row["steps"],
"cancelled": bool(row["cancelled"]),
+55 -9
View File
@@ -8,6 +8,7 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable
from .agent import normalize_disabled_tools
from .config import AppConfig
from .models import RuntimeOptions
from .runtime import create_openai_client, run_job
@@ -50,6 +51,13 @@ class JobManager:
click_pause: float = 0.10,
reasoning_effort: str = "medium",
screen_context_decay_steps: int = 4,
max_visual_context_images: int | None = None,
native_automation_mode: str = "prefer",
dialog_timeout_seconds: float = 12.0,
focus_timeout_seconds: float = 8.0,
ui_element_timeout_seconds: float = 8.0,
max_retries_per_surface: int = 3,
pretty_logs: bool = False,
disabled_tools: list[str] | None = None,
safety_override: bool = False,
no_failsafe: bool = False,
@@ -57,7 +65,13 @@ class JobManager:
job_id = f"job_{int(time.time())}_{uuid.uuid4().hex[:8]}"
created_at = utc_now_iso()
selected_model = (model or self.config.default_model).strip() or self.config.default_model
disabled = sorted({tool.strip() for tool in (disabled_tools or []) if tool.strip()})
disabled = normalize_disabled_tools(disabled_tools)
with self._lock:
stale = [key for key, job in self._running.items() if not job.thread.is_alive()]
for key in stale:
self._running.pop(key, None)
if self._running:
raise ValueError("Another ScreenJob run is already active on this desktop.")
self.db.create_job(
job_id=job_id,
objective=objective,
@@ -97,6 +111,17 @@ class JobManager:
"click_pause": click_pause,
"reasoning_effort": reasoning_effort,
"screen_context_decay_steps": screen_context_decay_steps,
"max_visual_context_images": (
self.config.max_visual_context_images_default
if max_visual_context_images is None
else max_visual_context_images
),
"native_automation_mode": native_automation_mode,
"dialog_timeout_seconds": dialog_timeout_seconds,
"focus_timeout_seconds": focus_timeout_seconds,
"ui_element_timeout_seconds": ui_element_timeout_seconds,
"max_retries_per_surface": max_retries_per_surface,
"pretty_logs": pretty_logs,
"no_failsafe": no_failsafe,
"cancel_event": cancel_event,
},
@@ -127,6 +152,13 @@ class JobManager:
click_pause: float,
reasoning_effort: str,
screen_context_decay_steps: int,
max_visual_context_images: int | None,
native_automation_mode: str,
dialog_timeout_seconds: float,
focus_timeout_seconds: float,
ui_element_timeout_seconds: float,
max_retries_per_surface: int,
pretty_logs: bool,
no_failsafe: bool,
cancel_event: threading.Event,
) -> None:
@@ -226,7 +258,22 @@ class JobManager:
click_pause=click_pause,
reasoning_effort=reasoning_effort,
screen_context_decay_steps=max(0, int(screen_context_decay_steps)),
max_visual_context_images=max(
0,
int(
self.config.max_visual_context_images_default
if max_visual_context_images is None
else max_visual_context_images
),
),
native_automation_mode=str(native_automation_mode or "prefer").strip().lower() or "prefer",
dialog_timeout_seconds=max(0.5, float(dialog_timeout_seconds)),
focus_timeout_seconds=max(0.5, float(focus_timeout_seconds)),
ui_element_timeout_seconds=max(0.5, float(ui_element_timeout_seconds)),
max_retries_per_surface=max(1, int(max_retries_per_surface)),
pretty_logs=bool(pretty_logs),
disable_tools=set(disabled_tools),
prohibited_key_combos=set(self.config.prohibited_key_combos),
)
try:
result, artifacts = run_job(
@@ -289,8 +336,8 @@ class JobManager:
"event_type": "job_finished",
"payload": {
"status": status,
"result": result.return_message,
"response": {"return": result.return_message, "data": result.data},
"return": result.return_message,
"data": result.data,
"error": result.error,
"cancelled": result.cancelled,
"usage": result.usage.to_dict(),
@@ -355,10 +402,9 @@ class JobManager:
return self.db.analytics()
def _normalize_job_payload(self, job: dict[str, Any]) -> dict[str, Any]:
response = job.get("response")
if not isinstance(response, dict):
response = {"return": str(job.get("result") or ""), "data": None}
job["response"] = response
job["return"] = str(response.get("return") or "")
job["data"] = response.get("data")
if "return" not in job:
job["return"] = str(job.get("result") or "")
job.setdefault("data", None)
job.pop("response", None)
job.pop("result", None)
return job
+5 -15
View File
@@ -26,22 +26,12 @@
<h2 class="font-semibold">Analytics</h2>
<div id="analyticsMeta" class="text-[11px] text-slate-400"></div>
</div>
<div id="analyticsSummary" class="grid grid-cols-2 md:grid-cols-4 gap-3"></div>
<div class="grid grid-cols-1 xl:grid-cols-2 gap-4">
<div class="bg-slate-900/70 border border-slate-800 rounded-xl p-4 space-y-3">
<div class="flex items-center justify-between gap-3">
<h3 class="font-semibold text-sm">Success by Objective Category</h3>
<div id="analyticsCategorySummary" class="text-[11px] text-slate-400"></div>
</div>
<div id="analyticsCategories" class="space-y-3"></div>
</div>
<div class="bg-slate-900/70 border border-slate-800 rounded-xl p-4 space-y-3">
<div class="flex items-center justify-between gap-3">
<h3 class="font-semibold text-sm">Avg Steps / Cost Over Time</h3>
<div id="analyticsTrendSummary" class="text-[11px] text-slate-400"></div>
</div>
<div id="analyticsTrends" class="space-y-4"></div>
<div class="bg-slate-900/70 border border-slate-800 rounded-xl p-4 space-y-3">
<div class="flex items-center justify-between gap-3">
<h3 class="font-semibold text-sm">Success by Objective Category</h3>
<div id="analyticsCategorySummary" class="text-[11px] text-slate-400"></div>
</div>
<div id="analyticsCategories" class="space-y-3"></div>
</div>
</section>
+1 -116
View File
@@ -18,11 +18,8 @@ const replayNextBtn = document.getElementById("replayNextBtn");
const replaySpeedEl = document.getElementById("replaySpeed");
const replaySeekEl = document.getElementById("replaySeek");
const analyticsMetaEl = document.getElementById("analyticsMeta");
const analyticsSummaryEl = document.getElementById("analyticsSummary");
const analyticsCategorySummaryEl = document.getElementById("analyticsCategorySummary");
const analyticsCategoriesEl = document.getElementById("analyticsCategories");
const analyticsTrendSummaryEl = document.getElementById("analyticsTrendSummary");
const analyticsTrendsEl = document.getElementById("analyticsTrends");
const state = {
token: localStorage.getItem("screenjob_token") || "",
@@ -98,104 +95,10 @@ function formatPercent(value) {
return Number.isFinite(num) ? `${num.toFixed(1)}%` : "—";
}
function formatDateLabel(value) {
const dt = new Date(value);
if (Number.isNaN(dt.getTime())) return String(value || "—");
return dt.toLocaleDateString(undefined, { month: "short", day: "numeric" });
}
function renderMetricCard(label, value) {
return `
<div class="bg-slate-950 border border-slate-800 rounded-xl p-3">
<div class="text-[11px] uppercase tracking-wide text-slate-400">${escapeHtml(label)}</div>
<div class="text-xl font-semibold mt-1">${escapeHtml(value)}</div>
</div>
`;
}
function renderLineChart(title, points, options = {}) {
const color = options.color || "#22d3ee";
const valueLabel = options.valueLabel || "";
const sourcePoints = Array.isArray(points)
? points.filter((point) => Number.isFinite(Number(point.value)))
: [];
if (!sourcePoints.length) {
return `
<div class="rounded-lg border border-slate-800 bg-slate-950/70 p-3">
<div class="flex items-center justify-between gap-3">
<div>
<div class="text-xs text-slate-400">${escapeHtml(title)}</div>
<div class="text-sm text-slate-200 font-semibold">No data yet</div>
</div>
</div>
</div>
`;
}
const width = 640;
const height = 220;
const margin = { top: 20, right: 18, bottom: 34, left: 44 };
const values = sourcePoints.map((point) => Number(point.value));
const minValue = Math.min(...values);
const maxValue = Math.max(...values);
const span = maxValue - minValue || 1;
const chartWidth = width - margin.left - margin.right;
const chartHeight = height - margin.top - margin.bottom;
const xStep = sourcePoints.length > 1 ? chartWidth / (sourcePoints.length - 1) : 0;
const coords = sourcePoints.map((point, index) => ({
x: margin.left + (index * xStep),
y: margin.top + ((maxValue - Number(point.value)) / span) * chartHeight,
}));
const linePath = coords.map((point, index) => `${index === 0 ? "M" : "L"} ${point.x} ${point.y}`).join(" ");
const baseline = height - margin.bottom;
const midIndex = Math.floor(sourcePoints.length / 2);
const xLabels = [
{ index: 0, label: sourcePoints[0].label },
{ index: midIndex, label: sourcePoints[midIndex].label },
{ index: sourcePoints.length - 1, label: sourcePoints[sourcePoints.length - 1].label },
].filter((item, index, array) => item.label && array.findIndex((candidate) => candidate.index === item.index) === index);
const minLabel = options.formatValue ? options.formatValue(minValue) : formatNumber(minValue, 2);
const maxLabel = options.formatValue ? options.formatValue(maxValue) : formatNumber(maxValue, 2);
const latest = sourcePoints[sourcePoints.length - 1];
const latestValue = options.formatValue ? options.formatValue(latest.value) : formatNumber(latest.value, 2);
return `
<div class="rounded-lg border border-slate-800 bg-slate-950/70 p-3 space-y-2">
<div class="flex items-center justify-between gap-3">
<div>
<div class="text-xs text-slate-400">${escapeHtml(title)}</div>
<div class="text-sm text-slate-200 font-semibold">${escapeHtml(latestValue)}${valueLabel ? ` <span class="text-slate-500 font-normal">${escapeHtml(valueLabel)}</span>` : ""}</div>
</div>
<div class="text-[11px] text-slate-400 text-right">
<div>${escapeHtml(sourcePoints.length)} points</div>
<div>${escapeHtml(minLabel)} - ${escapeHtml(maxLabel)}</div>
</div>
</div>
<svg viewBox="0 0 ${width} ${height}" class="w-full h-56">
${Array.from({ length: 4 }, (_, idx) => {
const y = margin.top + (chartHeight / 3) * idx;
return `<line x1="${margin.left}" y1="${y}" x2="${width - margin.right}" y2="${y}" stroke="rgba(51, 65, 85, 0.7)" stroke-width="1" />`;
}).join("")}
<line x1="${margin.left}" y1="${baseline}" x2="${width - margin.right}" y2="${baseline}" stroke="rgba(71, 85, 105, 0.8)" stroke-width="1.5" />
<path d="${linePath}" fill="none" stroke="${color}" stroke-width="3" stroke-linecap="round" stroke-linejoin="round" />
${coords.map((point) => `
<circle cx="${point.x}" cy="${point.y}" r="4.5" fill="${color}" />
`).join("")}
<text x="${margin.left - 8}" y="${margin.top + 4}" text-anchor="end" class="fill-slate-400 text-[10px]">${escapeHtml(maxLabel)}</text>
<text x="${margin.left - 8}" y="${baseline}" text-anchor="end" class="fill-slate-400 text-[10px]">${escapeHtml(minLabel)}</text>
${xLabels.map((item) => `
<text x="${coords[item.index].x}" y="${height - 10}" text-anchor="middle" class="fill-slate-500 text-[10px]">${escapeHtml(formatDateLabel(item.label))}</text>
`).join("")}
</svg>
</div>
`;
}
function renderAnalytics(payload) {
const analytics = payload || {};
const categories = Array.isArray(analytics.by_category) ? analytics.by_category : [];
const timeline = Array.isArray(analytics.timeline) ? analytics.timeline : [];
const finishedCategories = categories.filter((row) => Number(row.finished_jobs || 0) > 0);
if (analyticsMetaEl) {
@@ -204,13 +107,6 @@ function renderAnalytics(payload) {
: "Historical snapshot";
}
analyticsSummaryEl.innerHTML = [
renderMetricCard("Finished Jobs", analytics.finished_jobs || 0),
renderMetricCard("Success Rate", formatPercent(analytics.success_rate)),
renderMetricCard("Avg Steps", formatNumber(analytics.avg_steps, 1)),
renderMetricCard("Avg Cost", formatCurrency(analytics.avg_cost_usd)),
].join("");
analyticsCategorySummaryEl.textContent = finishedCategories.length
? `${finishedCategories.length} categories`
: "No finished jobs yet";
@@ -220,7 +116,6 @@ function renderAnalytics(payload) {
const successRate = Number(row.success_rate || 0);
const completed = Number(row.completed_jobs || 0);
const finished = Number(row.finished_jobs || 0);
const total = Number(row.total_jobs || 0);
const avgSteps = row.avg_steps == null ? "—" : formatNumber(row.avg_steps, 1);
const avgCost = row.avg_cost_usd == null ? "—" : formatCurrency(row.avg_cost_usd);
return `
@@ -228,7 +123,7 @@ function renderAnalytics(payload) {
<div class="flex items-start justify-between gap-3">
<div>
<div class="font-medium">${escapeHtml(row.label || "Other")}</div>
<div class="text-[11px] text-slate-400">${finished} finished · ${completed} completed · ${total} total</div>
<div class="text-[11px] text-slate-400">${finished} finished · ${completed} completed</div>
</div>
<div class="text-right">
<div class="text-base font-semibold">${formatPercent(successRate)}</div>
@@ -252,16 +147,6 @@ function renderAnalytics(payload) {
</div>
`;
}
analyticsTrendSummaryEl.textContent = timeline.length ? `${timeline.length} days` : "No daily data yet";
analyticsTrendsEl.innerHTML = [
renderLineChart("Average steps per day", timeline.map((row) => ({ label: row.label, value: row.avg_steps })), { color: "#38bdf8" }),
renderLineChart("Average cost per day", timeline.map((row) => ({ label: row.label, value: row.avg_cost_usd })), {
color: "#34d399",
valueLabel: "USD",
formatValue: (value) => formatCurrency(value),
}),
].join("");
}
function renderJobs() {
+38 -5
View File
@@ -3,8 +3,9 @@ from __future__ import annotations
import base64
import io
import logging
import shutil
import sys
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from pathlib import Path
from PIL import Image, ImageDraw
@@ -85,23 +86,55 @@ def setup_artifacts(base_dir: Path) -> RunArtifacts:
)
def cleanup_old_run_artifacts(base_dir: Path, retention_days: int) -> int:
if retention_days < 1 or not base_dir.exists() or not base_dir.is_dir():
return 0
cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
removed = 0
for child in base_dir.iterdir():
try:
modified_at = datetime.fromtimestamp(child.stat().st_mtime, tz=timezone.utc)
except FileNotFoundError:
continue
if modified_at >= cutoff:
continue
if child.is_dir():
shutil.rmtree(child, ignore_errors=True)
removed += 1
elif child.is_file():
try:
child.unlink()
removed += 1
except FileNotFoundError:
pass
return removed
def setup_logger(log_file: Path, verbose: bool = True) -> logging.Logger:
logger = logging.getLogger("screenjob")
logger_name = f"screenjob.{log_file.parent.parent.name}"
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
logger.handlers.clear()
logger.propagate = False
if logger.handlers:
return logger
stream_level = logging.INFO if verbose else logging.WARNING
stream_handler = logging.StreamHandler(sys.stdout)
stream_handler.setLevel(stream_level)
stream_handler.setFormatter(
logging.Formatter("%(asctime)s | %(levelname)-8s | %(message)s")
logging.Formatter(
"%(asctime)s.%(msecs)03d | %(message)s",
datefmt="%H:%M:%S",
)
)
file_handler = logging.FileHandler(log_file, encoding="utf-8")
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(
logging.Formatter(
"%(asctime)s | %(levelname)-8s | %(name)s | %(filename)s:%(lineno)d | %(message)s"
"%(asctime)s.%(msecs)03d | %(levelname)-8s | %(filename)s:%(lineno)d | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
)
@@ -5,7 +5,7 @@ Set shell = CreateObject("WScript.Shell")
Set fso = CreateObject("Scripting.FileSystemObject")
scriptDir = fso.GetParentFolderName(WScript.ScriptFullName)
psScript = """" & fso.BuildPath(scriptDir, "screenjob_tray.ps1") & """"
psScript = """" & fso.BuildPath(scriptDir, "start_backend.ps1") & """"
command = "powershell.exe -NoProfile -ExecutionPolicy Bypass -WindowStyle Hidden -STA -File " & psScript
shell.Run command, 0, False
File diff suppressed because it is too large Load Diff
+9 -2
View File
@@ -20,6 +20,7 @@ def test_cli_emits_structured_return_and_data(monkeypatch: Any, capsys, tmp_path
port=8787,
runs_dir=tmp_path / "runs",
db_path=tmp_path / "screenjob.db",
prohibited_key_combos=("ctrl+shift+s",),
)
config.runs_dir.mkdir(parents=True, exist_ok=True)
@@ -65,9 +66,15 @@ def test_cli_emits_structured_return_and_data(monkeypatch: Any, capsys, tmp_path
out = capsys.readouterr().out
payload = json.loads(out)
assert payload["response"]["return"] == "Task completed successfully"
assert payload["response"]["data"] == "file1.txt\nfile2.txt"
assert payload["return"] == "Task completed successfully"
assert payload["data"] == "file1.txt\nfile2.txt"
assert captured_kwargs["options"].reasoning_effort == "medium"
assert captured_kwargs["options"].screen_context_decay_steps == 4
assert captured_kwargs["options"].max_visual_context_images == 3
assert captured_kwargs["options"].native_automation_mode == "prefer"
assert captured_kwargs["options"].dialog_timeout_seconds == 12.0
assert captured_kwargs["options"].focus_timeout_seconds == 8.0
assert captured_kwargs["options"].ui_element_timeout_seconds == 8.0
assert captured_kwargs["options"].max_retries_per_surface == 3
assert captured_kwargs["options"].pretty_logs is False
assert captured_kwargs["options"].prohibited_key_combos == {"ctrl+shift+s"}
+67 -90
View File
@@ -7,26 +7,12 @@ from fastapi.testclient import TestClient
import src.server as server_module
from src.config import AppConfig
from src.storage import _objective_category
_TERMINAL_STATUSES = {"completed", "failed", "cancelled"}
def _objective_category(objective: str) -> str:
text = objective.lower()
if any(keyword in text for keyword in ("browser", "website", "amazon", "google", "login", "shopping", "checkout", "orders")):
return "Browser / web"
if any(keyword in text for keyword in ("file", "folder", "directory", "terminal", "shell", "command", "cli", "script", "git", "repo", "install", "pip", "npm")):
return "Files / terminal"
if any(keyword in text for keyword in ("write", "summary", "document", "docs", "report", "email", "message", "readme", "markdown")):
return "Writing / docs"
if any(keyword in text for keyword in ("data", "analysis", "csv", "spreadsheet", "sheet", "table", "chart", "dashboard", "metric", "sql")):
return "Data / analysis"
if any(keyword in text for keyword in ("code", "bug", "fix", "test", "debug", "api", "backend", "frontend", "database", "deploy", "docker", "service", "build")):
return "Development / ops"
return "Other"
class FakeJobManager:
def __init__(self, *, config: AppConfig, db: Any, broadcast: Any = None) -> None:
self.config = config
@@ -46,6 +32,13 @@ class FakeJobManager:
click_pause: float = 0.10,
reasoning_effort: str = "medium",
screen_context_decay_steps: int = 4,
max_visual_context_images: int = 3,
native_automation_mode: str = "prefer",
dialog_timeout_seconds: float = 12.0,
focus_timeout_seconds: float = 8.0,
ui_element_timeout_seconds: float = 8.0,
max_retries_per_surface: int = 3,
pretty_logs: bool = False,
disabled_tools: list[str] | None = None,
safety_override: bool = False,
no_failsafe: bool = False,
@@ -69,6 +62,13 @@ class FakeJobManager:
"click_pause": click_pause,
"reasoning_effort": reasoning_effort,
"screen_context_decay_steps": screen_context_decay_steps,
"max_visual_context_images": max_visual_context_images,
"native_automation_mode": native_automation_mode,
"dialog_timeout_seconds": dialog_timeout_seconds,
"focus_timeout_seconds": focus_timeout_seconds,
"ui_element_timeout_seconds": ui_element_timeout_seconds,
"max_retries_per_surface": max_retries_per_surface,
"pretty_logs": pretty_logs,
"no_failsafe": no_failsafe,
}
self._jobs[job_id] = {
@@ -80,8 +80,6 @@ class FakeJobManager:
"started_at": created_at,
"ended_at": None,
"steps": 1,
"result": "Running",
"response": {"return": "Running", "data": None},
"return": "Running",
"data": None,
"usage": {
@@ -174,7 +172,6 @@ class FakeJobManager:
def analytics(self) -> dict[str, Any]:
by_category: dict[str, dict[str, Any]] = {}
by_day: dict[str, dict[str, Any]] = {}
def bucket(target: dict[str, dict[str, Any]], key: str) -> dict[str, Any]:
return target.setdefault(
@@ -193,65 +190,28 @@ class FakeJobManager:
},
)
total_jobs = 0
finished_jobs = 0
completed_jobs = 0
failed_jobs = 0
cancelled_jobs = 0
steps_sum = 0
steps_count = 0
cost_sum = 0.0
cost_count = 0
for job in self._jobs.values():
total_jobs += 1
status = str(job.get("status") or "")
finished = status in _TERMINAL_STATUSES
category = _objective_category(str(job.get("objective") or ""))
day = str(job.get("created_at") or "")[:10] or "unknown"
category_bucket = bucket(by_category, category)
day_bucket = bucket(by_day, day)
for item in (category_bucket, day_bucket):
item["total_jobs"] += 1
category_bucket = bucket(by_category, _objective_category(str(job.get("objective") or "")))
category_bucket["total_jobs"] += 1
if not finished:
continue
finished_jobs += 1
category_bucket["finished_jobs"] += 1
if status == "completed":
completed_jobs += 1
category_bucket["completed_jobs"] += 1
elif status == "failed":
failed_jobs += 1
category_bucket["failed_jobs"] += 1
elif status == "cancelled":
cancelled_jobs += 1
category_bucket["cancelled_jobs"] += 1
steps_raw = job.get("steps")
if steps_raw is not None:
steps = int(steps_raw)
steps_sum += steps
steps_count += 1
for item in (category_bucket, day_bucket):
item["steps_sum"] += steps
item["steps_count"] += 1
category_bucket["steps_sum"] += int(steps_raw)
category_bucket["steps_count"] += 1
estimated_cost_raw = (job.get("usage") or {}).get("estimated_cost_usd")
if estimated_cost_raw is not None:
estimated_cost = float(estimated_cost_raw)
cost_sum += estimated_cost
cost_count += 1
for item in (category_bucket, day_bucket):
item["cost_sum"] += estimated_cost
item["cost_count"] += 1
for item in (category_bucket, day_bucket):
item["finished_jobs"] += 1
if status == "completed":
item["completed_jobs"] += 1
elif status == "failed":
item["failed_jobs"] += 1
elif status == "cancelled":
item["cancelled_jobs"] += 1
category_bucket["cost_sum"] += float(estimated_cost_raw)
category_bucket["cost_count"] += 1
def finalize(item: dict[str, Any]) -> dict[str, Any]:
finished = item["finished_jobs"]
@@ -267,18 +227,7 @@ class FakeJobManager:
"avg_cost_usd": round(item["cost_sum"] / item["cost_count"], 6) if item["cost_count"] else None,
}
return {
"total_jobs": total_jobs,
"finished_jobs": finished_jobs,
"completed_jobs": completed_jobs,
"failed_jobs": failed_jobs,
"cancelled_jobs": cancelled_jobs,
"success_rate": round((completed_jobs / finished_jobs) * 100, 2) if finished_jobs else 0.0,
"avg_steps": round(steps_sum / steps_count, 2) if steps_count else None,
"avg_cost_usd": round(cost_sum / cost_count, 6) if cost_count else None,
"by_category": sorted((finalize(item) for item in by_category.values()), key=lambda item: (-item["success_rate"], item["label"])),
"timeline": sorted((finalize(item) for item in by_day.values()), key=lambda item: item["label"]),
}
return {"by_category": sorted((finalize(item) for item in by_category.values()), key=lambda item: item["label"])}
def _build_app(tmp_path: Path, monkeypatch: Any, disable_ui: bool = False):
@@ -293,6 +242,7 @@ def _build_app(tmp_path: Path, monkeypatch: Any, disable_ui: bool = False):
port=8787,
runs_dir=tmp_path / "runs",
db_path=tmp_path / "screenjob_test.db",
prohibited_key_combos=("ctrl+shift+s",),
)
config.runs_dir.mkdir(parents=True, exist_ok=True)
app = server_module.create_app(config)
@@ -326,12 +276,49 @@ def test_create_job_returns_only_job_id_and_defaults_model(tmp_path: Path, monke
assert manager.last_submit_payload["disabled_tools"] == ["click"]
assert manager.last_submit_payload["reasoning_effort"] == "medium"
assert manager.last_submit_payload["screen_context_decay_steps"] == 4
assert manager.last_submit_payload["max_visual_context_images"] == 3
assert manager.last_submit_payload["native_automation_mode"] == "prefer"
assert manager.last_submit_payload["dialog_timeout_seconds"] == 12.0
assert manager.last_submit_payload["focus_timeout_seconds"] == 8.0
assert manager.last_submit_payload["ui_element_timeout_seconds"] == 8.0
assert manager.last_submit_payload["max_retries_per_surface"] == 3
assert manager.last_submit_payload["pretty_logs"] is False
status_res = client.get(f"/api/jobs/{job_id}/status", headers=headers)
assert status_res.status_code == 200
assert status_res.json()["job_id"] == job_id
assert status_res.json()["response"]["return"] == "Running"
assert "data" in status_res.json()["response"]
assert status_res.json()["return"] == "Running"
assert status_res.json()["data"] is None
def test_create_job_rejects_invalid_disabled_tool_names(tmp_path: Path, monkeypatch: Any) -> None:
app, _ = _build_app(tmp_path, monkeypatch, disable_ui=False)
client = TestClient(app)
headers = {"Authorization": "Bearer test_token"}
response = client.post(
"/api/jobs",
headers=headers,
json={"job": "Open amazon.de", "disabled_tools": ["not_a_real_tool"], "safety_override": True},
)
assert response.status_code == 400
assert "Unknown disabled tool" in response.json()["detail"]
def test_create_job_rejects_disabling_task_complete(tmp_path: Path, monkeypatch: Any) -> None:
app, _ = _build_app(tmp_path, monkeypatch, disable_ui=False)
client = TestClient(app)
headers = {"Authorization": "Bearer test_token"}
response = client.post(
"/api/jobs",
headers=headers,
json={"job": "Open amazon.de", "disabled_tools": ["task_complete"], "safety_override": True},
)
assert response.status_code == 400
assert "Cannot disable required tool" in response.json()["detail"]
def test_cancel_endpoint_and_events(tmp_path: Path, monkeypatch: Any) -> None:
@@ -407,7 +394,7 @@ def test_replay_endpoint_skips_visual_paths_outside_artifacts(tmp_path: Path, mo
assert payload["total_frames"] == 1
def test_analytics_endpoint_groups_by_category_and_time(tmp_path: Path, monkeypatch: Any) -> None:
def test_analytics_endpoint_groups_by_category(tmp_path: Path, monkeypatch: Any) -> None:
app, _ = _build_app(tmp_path, monkeypatch, disable_ui=False)
manager = app.state.manager
client = TestClient(app)
@@ -443,14 +430,6 @@ def test_analytics_endpoint_groups_by_category_and_time(tmp_path: Path, monkeypa
assert analytics.status_code == 200
payload = analytics.json()
assert payload["total_jobs"] == 3
assert payload["finished_jobs"] == 3
assert payload["completed_jobs"] == 2
assert payload["failed_jobs"] == 1
assert payload["success_rate"] == 66.67
assert payload["avg_steps"] == 6.67
assert payload["avg_cost_usd"] == 0.136667
browser = next(row for row in payload["by_category"] if row["label"] == "Browser / web")
terminal = next(row for row in payload["by_category"] if row["label"] == "Files / terminal")
assert browser["finished_jobs"] == 2
@@ -458,8 +437,6 @@ def test_analytics_endpoint_groups_by_category_and_time(tmp_path: Path, monkeypa
assert browser["avg_steps"] == 5.0
assert terminal["success_rate"] == 100.0
assert [row["label"] for row in payload["timeline"]] == ["2026-05-27", "2026-05-28"]
def test_ui_toggle(tmp_path: Path, monkeypatch: Any) -> None:
app_enabled, _ = _build_app(tmp_path / "enabled", monkeypatch, disable_ui=False)
+32 -14
View File
@@ -37,8 +37,8 @@ def test_history_db_job_and_events_roundtrip(tmp_path: Path) -> None:
assert job["status"] == "completed"
assert job["model"] == "gpt-5.4-mini"
assert job["disabled_tools"] == ["click"]
assert job["response"]["return"] == "Done"
assert job["response"]["data"]["files"] == ["a.txt", "b.txt"]
assert job["return"] == "Done"
assert job["data"]["files"] == ["a.txt", "b.txt"]
assert job["usage"]["estimated_cost_usd"] == 0.1234
events = db.get_job_events(job_id, limit=10)
@@ -70,11 +70,11 @@ def test_storage_response_fallback_uses_result_when_json_missing(tmp_path: Path)
db.update_job(job_id, status="completed", result="Legacy result string")
job = db.get_job(job_id)
assert job is not None
assert job["response"]["return"] == "Legacy result string"
assert job["response"]["data"] is None
assert job["return"] == "Legacy result string"
assert job["data"] is None
def test_history_db_analytics_groups_by_category_and_day(tmp_path: Path) -> None:
def test_history_db_analytics_groups_by_category(tmp_path: Path) -> None:
db = HistoryDB(tmp_path / "screenjob_test_analytics.db")
db.create_job(
@@ -108,14 +108,6 @@ def test_history_db_analytics_groups_by_category_and_day(tmp_path: Path) -> None
db.update_job("job_terminal_ok", status="completed", steps=10, estimated_cost_usd=0.05)
analytics = db.analytics()
assert analytics["total_jobs"] == 3
assert analytics["finished_jobs"] == 3
assert analytics["completed_jobs"] == 2
assert analytics["failed_jobs"] == 1
assert analytics["success_rate"] == 66.67
assert analytics["avg_steps"] == 6.67
assert analytics["avg_cost_usd"] == 0.136667
browser = next(row for row in analytics["by_category"] if row["label"] == "Browser / web")
terminal = next(row for row in analytics["by_category"] if row["label"] == "Files / terminal")
assert browser["finished_jobs"] == 2
@@ -123,4 +115,30 @@ def test_history_db_analytics_groups_by_category_and_day(tmp_path: Path) -> None
assert browser["avg_steps"] == 5.0
assert terminal["success_rate"] == 100.0
assert [row["label"] for row in analytics["timeline"]] == ["2026-05-27", "2026-05-28"]
def test_prune_older_than_removes_old_terminal_jobs(tmp_path: Path) -> None:
db = HistoryDB(tmp_path / "screenjob_prune.db")
db.create_job(
job_id="job_old",
objective="Old",
model="gpt-5.4-mini",
created_at="2000-01-01T00:00:00+00:00",
safety_override=False,
disabled_tools=[],
)
db.update_job("job_old", status="completed")
db.add_event(job_id="job_old", ts="2000-01-01T00:00:01+00:00", step=1, event_type="done", payload={})
db.create_job(
job_id="job_running",
objective="Running",
model="gpt-5.4-mini",
created_at="2000-01-01T00:00:00+00:00",
safety_override=False,
disabled_tools=[],
)
db.update_job("job_running", status="running")
assert db.prune_older_than(7) == 1
assert db.get_job("job_old") is None
assert db.get_job("job_running") is not None
+249
View File
@@ -0,0 +1,249 @@
from __future__ import annotations
import threading
from pathlib import Path
from typing import Any
import pytest
import src.task_manager as task_manager_module
from src.config import AppConfig
from src.models import AgentResult, RunArtifacts, UsageSummary
from src.storage import HistoryDB
from src.task_manager import JobManager
def _build_manager(tmp_path: Path) -> tuple[JobManager, HistoryDB, AppConfig]:
config = AppConfig(
openai_api_key="test-key",
screenjob_token="test-token",
disable_ui=False,
default_model="gpt-5.4-mini",
safety_model="gpt-5.4-mini",
host="127.0.0.1",
port=8787,
runs_dir=tmp_path / "runs",
db_path=tmp_path / "screenjob.db",
)
db = HistoryDB(config.db_path)
manager = JobManager(config=config, db=db)
return manager, db, config
def _artifacts(tmp_path: Path) -> RunArtifacts:
root = tmp_path / "run_artifacts"
return RunArtifacts(
run_id="test_run",
root_dir=root,
logs_dir=root / "logs",
shots_dir=root / "shots",
enhance_dir=root / "enhanced",
log_file=root / "logs" / "screenjob.log",
)
def _create_job(db: HistoryDB, job_id: str, objective: str) -> None:
db.create_job(
job_id=job_id,
objective=objective,
model="gpt-5.4-mini",
created_at="2026-05-30T12:00:00+00:00",
safety_override=True,
disabled_tools=[],
)
def test_completed_job_updates_status(tmp_path: Path, monkeypatch) -> None:
manager, db, _config = _build_manager(tmp_path)
job_id = "job_complete"
objective = "Save todo-demo.txt in Documents"
_create_job(db, job_id, objective)
result = AgentResult(
completed=True,
result="Saved todo-demo.txt",
return_message="Saved todo-demo.txt",
data={"observed_result": "todo-demo.txt - Notepad is visible"},
steps=11,
started_at=100.0,
ended_at=112.6,
usage=UsageSummary(),
)
monkeypatch.setattr(task_manager_module, "run_job", lambda **_kwargs: (result, _artifacts(tmp_path)))
manager._execute_job(
job_id=job_id,
objective=objective,
model="gpt-5.4-mini",
disabled_tools=[],
safety_override=True,
max_steps=60,
command_timeout=45,
type_interval=0.02,
click_pause=0.10,
reasoning_effort="medium",
screen_context_decay_steps=4,
max_visual_context_images=3,
native_automation_mode="prefer",
dialog_timeout_seconds=12.0,
focus_timeout_seconds=8.0,
ui_element_timeout_seconds=8.0,
max_retries_per_surface=3,
pretty_logs=False,
no_failsafe=False,
cancel_event=threading.Event(),
)
job = db.get_job(job_id)
assert job is not None
assert job["status"] == "completed"
assert job["return"] == "Saved todo-demo.txt"
def test_non_completed_jobs_are_recorded(tmp_path: Path, monkeypatch) -> None:
manager, db, _config = _build_manager(tmp_path)
failed_job_id = "job_failed"
_create_job(db, failed_job_id, "Fail intentionally")
failed_result = AgentResult(
completed=False,
result="Failure",
return_message="Failure",
data=None,
steps=7,
started_at=10.0,
ended_at=18.0,
usage=UsageSummary(),
error="Failure",
)
monkeypatch.setattr(task_manager_module, "run_job", lambda **_kwargs: (failed_result, _artifacts(tmp_path)))
manager._execute_job(
job_id=failed_job_id,
objective="Fail intentionally",
model="gpt-5.4-mini",
disabled_tools=[],
safety_override=True,
max_steps=60,
command_timeout=45,
type_interval=0.02,
click_pause=0.10,
reasoning_effort="medium",
screen_context_decay_steps=4,
max_visual_context_images=3,
native_automation_mode="prefer",
dialog_timeout_seconds=12.0,
focus_timeout_seconds=8.0,
ui_element_timeout_seconds=8.0,
max_retries_per_surface=3,
pretty_logs=False,
no_failsafe=False,
cancel_event=threading.Event(),
)
cancelled_job_id = "job_cancelled"
_create_job(db, cancelled_job_id, "Cancel intentionally")
cancelled_result = AgentResult(
completed=False,
result="Cancelled",
return_message="Cancelled",
data=None,
steps=4,
started_at=20.0,
ended_at=23.0,
usage=UsageSummary(),
error="Cancelled",
cancelled=True,
)
monkeypatch.setattr(task_manager_module, "run_job", lambda **_kwargs: (cancelled_result, _artifacts(tmp_path)))
manager._execute_job(
job_id=cancelled_job_id,
objective="Cancel intentionally",
model="gpt-5.4-mini",
disabled_tools=[],
safety_override=True,
max_steps=60,
command_timeout=45,
type_interval=0.02,
click_pause=0.10,
reasoning_effort="medium",
screen_context_decay_steps=4,
max_visual_context_images=3,
native_automation_mode="prefer",
dialog_timeout_seconds=12.0,
focus_timeout_seconds=8.0,
ui_element_timeout_seconds=8.0,
max_retries_per_surface=3,
pretty_logs=False,
no_failsafe=False,
cancel_event=threading.Event(),
)
assert db.get_job(failed_job_id)["status"] == "failed"
assert db.get_job(cancelled_job_id)["status"] == "cancelled"
def test_rejected_job_is_recorded(tmp_path: Path, monkeypatch) -> None:
manager, db, _config = _build_manager(tmp_path)
job_id = "job_rejected"
_create_job(db, job_id, "Do something unsafe")
monkeypatch.setattr(task_manager_module, "create_openai_client", lambda *_args, **_kwargs: object())
monkeypatch.setattr(
task_manager_module,
"assess_task_safety",
lambda *_args, **_kwargs: (False, "Unsafe request", {"decision": "blocked"}),
)
manager._execute_job(
job_id=job_id,
objective="Do something unsafe",
model="gpt-5.4-mini",
disabled_tools=[],
safety_override=False,
max_steps=60,
command_timeout=45,
type_interval=0.02,
click_pause=0.10,
reasoning_effort="medium",
screen_context_decay_steps=4,
max_visual_context_images=3,
native_automation_mode="prefer",
dialog_timeout_seconds=12.0,
focus_timeout_seconds=8.0,
ui_element_timeout_seconds=8.0,
max_retries_per_surface=3,
pretty_logs=False,
no_failsafe=False,
cancel_event=threading.Event(),
)
events = db.get_job_events(job_id)
assert events[-1]["event_type"] == "job_rejected"
def test_submit_job_rejects_when_another_run_is_active(tmp_path: Path) -> None:
manager, _db, _config = _build_manager(tmp_path)
ready = threading.Event()
release = threading.Event()
def _hold() -> None:
ready.set()
release.wait()
active_thread = threading.Thread(target=_hold)
active_thread.start()
ready.wait(1)
manager._running["job_active"] = task_manager_module._RunningJob(
thread=active_thread,
cancel_event=threading.Event(),
started_at="2026-05-30T12:00:00+00:00",
objective="Active",
model="gpt-5.4-mini",
)
try:
with pytest.raises(ValueError, match="already active"):
manager.submit_job(objective="Second run")
finally:
release.set()
active_thread.join(timeout=1)
+1
View File
@@ -1,6 +1,7 @@
# TODO
## P0
- [Bug] Fix CI & pytest
- [Bug] Enforce single active desktop-control run (or a strict queue) so concurrent jobs cannot fight over the same mouse/keyboard/screen session.
- [Bug] Fix run artifact collisions in `setup_artifacts()` (`run_id` is second-granularity, so two jobs in the same second can share/overwrite the same directory).
- [Bug] Remove global logger handler clobbering in `setup_logger()` (`logging.getLogger("screenjob").handlers.clear()` breaks concurrent runs and can redirect logs to the wrong file).
-53
View File
@@ -1,53 +0,0 @@
[CmdletBinding()]
param(
[ValidateSet("start", "stop", "restart")]
[string]$Action,
[string]$ServiceName = "ScreenJobBackend"
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
function Wait-ForStatus {
param(
[Parameter(Mandatory = $true)]$Service,
[Parameter(Mandatory = $true)][System.ServiceProcess.ServiceControllerStatus]$TargetStatus,
[int]$TimeoutSeconds = 20
)
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
$Service.Refresh()
if ($Service.Status -eq $TargetStatus) {
return
}
Start-Sleep -Milliseconds 350
}
throw "Timed out waiting for service '$($Service.ServiceName)' to reach status '$TargetStatus'."
}
$service = Get-Service -Name $ServiceName -ErrorAction Stop
switch ($Action) {
"start" {
if ($service.Status -ne [System.ServiceProcess.ServiceControllerStatus]::Running) {
Start-Service -Name $ServiceName -ErrorAction Stop
Wait-ForStatus -Service $service -TargetStatus ([System.ServiceProcess.ServiceControllerStatus]::Running)
}
}
"stop" {
if ($service.Status -ne [System.ServiceProcess.ServiceControllerStatus]::Stopped) {
Stop-Service -Name $ServiceName -Force -ErrorAction Stop
Wait-ForStatus -Service $service -TargetStatus ([System.ServiceProcess.ServiceControllerStatus]::Stopped)
}
}
"restart" {
if ($service.Status -eq [System.ServiceProcess.ServiceControllerStatus]::Running) {
Restart-Service -Name $ServiceName -Force -ErrorAction Stop
} else {
Start-Service -Name $ServiceName -ErrorAction Stop
}
Wait-ForStatus -Service $service -TargetStatus ([System.ServiceProcess.ServiceControllerStatus]::Running)
}
}
+28 -19
View File
@@ -1,36 +1,45 @@
[CmdletBinding(SupportsShouldProcess = $true)]
param(
[switch]$AllUsers,
[string]$ServiceName = "ScreenJobBackend"
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
function Test-IsAdministrator {
$identity = [Security.Principal.WindowsIdentity]::GetCurrent()
$principal = New-Object Security.Principal.WindowsPrincipal($identity)
return $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)
$scriptDir = Split-Path -Parent $PSCommandPath
$shortcutName = "ScreenJob Backend.lnk"
$startupFolder = if ($AllUsers) {
[Environment]::GetFolderPath("CommonStartup")
} else {
[Environment]::GetFolderPath("Startup")
}
if (-not (Test-IsAdministrator)) {
throw "Run this script from an elevated PowerShell session (Run as Administrator)."
}
$shortcutPath = Join-Path $startupFolder $shortcutName
$service = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($null -eq $service) {
Write-Host "Service '$ServiceName' is not installed."
exit 0
}
if ($null -ne $service) {
if ($PSCmdlet.ShouldProcess($ServiceName, "Remove legacy Windows service")) {
if ($service.Status -ne "Stopped") {
Stop-Service -Name $ServiceName -Force -ErrorAction Stop
}
if ($PSCmdlet.ShouldProcess($ServiceName, "Uninstall service")) {
if ($service.Status -ne "Stopped") {
Stop-Service -Name $ServiceName -Force -ErrorAction Stop
}
& sc.exe delete $ServiceName | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to delete service '$ServiceName' (sc.exe exit code $LASTEXITCODE)."
}
& sc.exe delete $ServiceName | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "Failed to delete service '$ServiceName' (sc.exe exit code $LASTEXITCODE)."
Write-Host "Removed legacy Windows service: $ServiceName"
}
}
Write-Host "Service '$ServiceName' uninstalled successfully." -ForegroundColor Green
if (Test-Path -LiteralPath $shortcutPath) {
if ($PSCmdlet.ShouldProcess($shortcutPath, "Remove backend startup shortcut")) {
Remove-Item -LiteralPath $shortcutPath -Force
Write-Host "Removed backend startup shortcut: $shortcutPath"
}
} else {
Write-Host "No backend startup shortcut found at: $shortcutPath"
}
Write-Host "Backend launcher uninstalled successfully." -ForegroundColor Green