custom-streamdeck/README.md

# Custom Streamdeck

Custom Streamdeck is a local control surface platform for a 10-button Raspberry Pi Pico device. It pairs physical button input with a desktop web app, persistent configuration, and a backend action engine so the deck can launch apps, send shortcuts, switch folders, and run backend plugins.

This repository is organized more like an application than a hardware experiment: the Pico emits button events, the backend manages state and automation, and the frontend provides a live configuration surface for day-to-day use.

## What The App Does

- Reads button press and release events from a Raspberry Pi Pico over USB serial
- Stores profiles, folders, button mappings, and action configuration in SQLite
- Serves a local web UI for deck configuration and live monitoring
- Executes built-in actions such as key presses, action chains, app launch, and folder navigation
- Loads backend plugins for feature-specific actions such as HTTP requests, media control, and clipboard tools
- Broadcasts live state updates over WebSocket so the UI stays in sync with the device
- Optionally runs a transparent desktop overlay that announces the active profile and folder

## Production View

The app is intended to run as a local always-on controller on a Windows workstation:

- The backend is the source of truth for device state and button configuration.
- The frontend is a built artifact served by FastAPI in production-style usage.
- The Pico is treated as an attached input device rather than a development-only serial toy.
- Button layouts are protected by a canonical hardware mapping so profile changes do not drift physical position assignments.
- Plugins extend backend behavior without requiring frontend-specific code for every new action.

This is still a local app, not a hosted SaaS service, but the codebase now behaves like an operational desktop product.

## Architecture

### Hardware Layer

The Pico firmware in `pico/main.py` scans the 10 physical buttons and prints JSON events over USB serial.

Button wiring order:

1. GP28
2. GP27
3. GP26
4. GP22
5. GP21
6. GP20
7. GP18
8. GP19
9. GP17
10. GP16

Each button’s other leg goes to GND. The firmware uses internal pull-ups, so a pressed button reads as LOW.

### Backend Layer

The FastAPI backend:

- auto-detects the Pico serial port, or uses a configured port override
- reads and validates button events
- persists state in SQLite under `data/`
- executes actions through a central action engine
- loads plugin modules from `plugins/`
- serves the built frontend and WebSocket updates

Key backend areas:

- `backend/main.py`: app lifecycle, API routes, WebSocket endpoint
- `backend/database.py`: SQLite schema, settings, profiles, folders, buttons, event history
- `backend/services/serial_service.py`: Pico connection and event loop
- `backend/services/actions.py`: action execution engine
- `backend/services/plugins.py`: runtime plugin loading and dispatch
- `backend/services/apps.py`: Windows app discovery and launch support

### Frontend Layer

The frontend is a React + Vite application that:

- shows connection and sync status
- manages profiles and folders
- edits button labels, colors, icons, trigger modes, and actions
- renders plugin action fields dynamically from backend metadata
- stays synchronized through backend WebSocket broadcasts

Primary frontend source lives in `frontend/src/`.

### Overlay Layer

The optional overlay in `overlay/` is an Electron app that connects to the backend WebSocket and displays the current deck profile and folder in a transparent, click-through window. It is designed for production desktop use: frameless, always on top, hidden from the taskbar, and controlled from the Windows tray.

## Current Action Model

Built-in actions currently include:

- No-op / display-only buttons
- Keyboard shortcut execution
- Chained multi-step actions
- Windows app launch
- Folder open
- Folder rotation
- Plugin-backed actions

Pre-shipped plugin actions include:

- HTTP Requests: call webhooks, local services, and automation endpoints with templated request data
- Media Controls: send media keys and repeated volume adjustments
- Clipboard Tools: copy preset text, paste snippets, and transform clipboard text
- OBS Integration: sample OBS-style scene and stream actions for plugin development
- WLED: control WLED device power, brightness, colors, effects, pixel ranges, and combined updates

## Plugin System

Backend plugins live in `plugins/` and expose a top-level `PLUGIN` object. The backend imports each plugin, reads its declared actions, and exposes those actions to the frontend automatically.

That means new capabilities can usually be added by shipping a Python plugin file without hand-building a matching frontend form.

Example shape:

```python
class MyPlugin:
    name = "My Plugin"
    desc = "Does a thing"
    version = "0.1.0"
    actions = [
        {
            "id": "do_thing",
            "name": "Do Thing",
            "fields": [{"id": "value", "label": "Value", "type": "text"}],
        }
    ]

    def on_load(self, ctx):
        pass

    def on_event(self, ctx, event):
        pass

    def execute_action(self, ctx, action_id, config, event):
        pass

PLUGIN = MyPlugin()
```

## Repository Layout

- `backend/`: FastAPI app, database, action engine, serial service, plugin runtime
- `frontend/`: React configuration UI
- `overlay/`: Electron transparent desktop overlay
- `plugins/`: backend plugin modules
- `pico/`: MicroPython firmware for the device
- `pc/`: utility scripts used during hardware bring-up and diagnostics
- `tests/`: backend tests
- `data/`: local runtime state, including SQLite data

## Local Setup

### Requirements

- Windows machine
- Python 3.13 or compatible recent Python 3
- Node.js and npm
- Raspberry Pi Pico flashed with the firmware in `pico/main.py`

### Install Backend Dependencies

```powershell
python -m pip install -r requirements.txt
```

### Install Frontend Dependencies

```powershell
cd frontend
npm install
cd ..
```

### Install Overlay Dependencies

```powershell
cd overlay
npm install
cd ..
```

## Running The App

### Production-Style Local Run

Build the frontend first:

```powershell
cd frontend
npm run build
cd ..
```

Start the backend:

```powershell
python -m uvicorn backend.main:app --host 127.0.0.1 --port 8000
```

Then open:

`http://127.0.0.1:8000/`

In this mode, FastAPI serves the built frontend from `frontend/dist`.

### Frontend Development Mode

Run the backend in one shell:

```powershell
python -m uvicorn backend.main:app --host 127.0.0.1 --port 8000
```

Run Vite in another:

```powershell
cd frontend
npm run dev
```

Then open the Vite URL, usually:

`http://127.0.0.1:5173/`

### Transparent Profile Overlay

The optional Electron overlay listens to the backend WebSocket at `ws://127.0.0.1:8000/ws` and shows a click-through popup when the visible deck changes. It appears on startup, active profile changes, and active folder changes, then fades away after 7 seconds.

Start the backend first, then run:

```powershell
cd overlay
npm run dev
```

The overlay is frameless, transparent, always on top, and non-interactable so it can sit over other desktop apps without stealing clicks. It does not put a button in the taskbar; the app lives in the Windows notification area with a tray menu for showing the last overlay state or quitting it.

To build the Windows installer locally:

```powershell
cd overlay
npm run dist:win
```

The installer is written to `overlay/release/`.

## Operational Notes

### Device Detection

By default the backend attempts to auto-detect the Pico using USB metadata. If that fails, the serial port can be set in the application settings.

### Persistence

Profiles, folders, button assignments, manual app entries, and event history are stored in SQLite under `data/streamdeck.sqlite`.

### Live Updates

The UI receives `state.updated` and action/device events over WebSocket. This keeps the browser view aligned with the live backend without manual refreshes.

### Windows Integration

App launch discovery currently focuses on Windows environments:

- Start Menu shortcuts
- installed applications from uninstall registry keys
- manually registered app paths

### Logging And Diagnostics

The backend records event history in SQLite. During hardware bring-up, `pc/listen_buttons.py` can still be used as a simple serial diagnostic tool.

## Testing

Run the backend test suite from the repo root:

```powershell
$env:PYTHONPATH = (Get-Location).Path
pytest -q
```

## Deployment Considerations

For a reliable day-to-day setup on a dedicated machine:

- build the frontend and serve the compiled assets from FastAPI
- pin Python dependencies from `requirements.txt`
- run the backend under a process manager, scheduled task, or startup shortcut
- keep the Pico on a stable USB port when possible
- treat `data/` as local runtime state and back it up if profiles matter

This app is best thought of as a local control appliance: hardware input on one side, desktop automation and operator UI on the other.

### Overlay Release Automation

The Gitea workflow in `.gitea/workflows/overlay-release.yml` builds the Electron overlay on a Windows runner for pushes and pull requests that touch `overlay/`. Pushing a tag that matches `v*` or `overlay-v*` also creates or reuses a Gitea release and uploads the generated installer `.exe`.

The workflow uses the built-in `GITEA_TOKEN` with `releases: write` permission, so the repository Actions settings must allow release writes for the job token.

## Legacy Utility

For raw serial logging without the full app:

```powershell
python .\pc\listen_buttons.py
```

If auto-detect fails:

```powershell
python .\pc\listen_buttons.py --port COM5
```