custom-streamdeck/README.md

Custom Streamdeck

Custom Streamdeck is a local control surface platform for a 10-button Raspberry Pi Pico device. It pairs physical button input with a desktop web app, persistent configuration, and a backend action engine so the deck can launch apps, send shortcuts, switch folders, and run backend plugins.

This repository is organized more like an application than a hardware experiment: the Pico emits button events, the backend manages state and automation, and the frontend provides a live configuration surface for day-to-day use.

What The App Does

• Reads button press and release events from a Raspberry Pi Pico over USB serial
• Stores profiles, folders, button mappings, and action configuration in SQLite
• Serves a local web UI for deck configuration and live monitoring
• Executes built-in actions such as key presses, action chains, app launch, and folder navigation
• Loads backend plugins for feature-specific actions such as HTTP requests, media control, and clipboard tools
• Broadcasts live state updates over WebSocket so the UI stays in sync with the device

Production View

The app is intended to run as a local always-on controller on a Windows workstation:

• The backend is the source of truth for device state and button configuration.
• The frontend is a built artifact served by FastAPI in production-style usage.
• The Pico is treated as an attached input device rather than a development-only serial toy.
• Button layouts are protected by a canonical hardware mapping so profile changes do not drift physical position assignments.
• Plugins extend backend behavior without requiring frontend-specific code for every new action.

This is still a local app, not a hosted SaaS service, but the codebase now behaves like an operational desktop product.

Architecture

Hardware Layer

The Pico firmware in pico/main.py scans the 10 physical buttons and prints JSON events over USB serial.

Button wiring order:

1. GP28
2. GP27
3. GP26
4. GP22
5. GP21
6. GP20
7. GP18
8. GP19
9. GP17
10. GP16

Each button’s other leg goes to GND. The firmware uses internal pull-ups, so a pressed button reads as LOW.

Backend Layer

The FastAPI backend:

• auto-detects the Pico serial port, or uses a configured port override
• reads and validates button events
• persists state in SQLite under data/
• executes actions through a central action engine
• loads plugin modules from plugins/
• serves the built frontend and WebSocket updates

Key backend areas:

• backend/main.py: app lifecycle, API routes, WebSocket endpoint
• backend/database.py: SQLite schema, settings, profiles, folders, buttons, event history
• backend/services/serial_service.py: Pico connection and event loop
• backend/services/actions.py: action execution engine
• backend/services/plugins.py: runtime plugin loading and dispatch
• backend/services/apps.py: Windows app discovery and launch support

Frontend Layer

The frontend is a React + Vite application that:

• shows connection and sync status
• manages profiles and folders
• edits button labels, colors, icons, trigger modes, and actions
• renders plugin action fields dynamically from backend metadata
• stays synchronized through backend WebSocket broadcasts

Primary frontend source lives in frontend/src/.

Current Action Model

Built-in actions currently include:

• No-op / display-only buttons
• Keyboard shortcut execution
• Chained multi-step actions
• Windows app launch
• Folder open
• Folder rotation
• Plugin-backed actions

Plugin actions currently include examples such as:

• HTTP requests
• Media controls
• Clipboard tools

Plugin System

Backend plugins live in plugins/ and expose a top-level PLUGIN object. The backend imports each plugin, reads its declared actions, and exposes those actions to the frontend automatically.

That means new capabilities can usually be added by shipping a Python plugin file without hand-building a matching frontend form.

Example shape:

class MyPlugin:
    name = "My Plugin"
    desc = "Does a thing"
    version = "0.1.0"
    actions = [
        {
            "id": "do_thing",
            "name": "Do Thing",
            "fields": [{"id": "value", "label": "Value", "type": "text"}],
        }
    ]

    def on_load(self, ctx):
        pass

    def on_event(self, ctx, event):
        pass

    def execute_action(self, ctx, action_id, config, event):
        pass

PLUGIN = MyPlugin()

Repository Layout

• backend/: FastAPI app, database, action engine, serial service, plugin runtime
• frontend/: React configuration UI
• plugins/: backend plugin modules
• pico/: MicroPython firmware for the device
• pc/: utility scripts used during hardware bring-up and diagnostics
• tests/: backend tests
• data/: local runtime state, including SQLite data

Local Setup

Requirements

• Windows machine
• Python 3.13 or compatible recent Python 3
• Node.js and npm
• Raspberry Pi Pico flashed with the firmware in pico/main.py

Install Backend Dependencies

python -m pip install -r requirements.txt

Install Frontend Dependencies

cd frontend
npm install
cd ..

Install Overlay Dependencies

cd overlay
npm install
cd ..

Running The App

Production-Style Local Run

Build the frontend first:

cd frontend
npm run build
cd ..

Start the backend:

python -m uvicorn backend.main:app --host 127.0.0.1 --port 8000

Then open:

http://127.0.0.1:8000/

In this mode, FastAPI serves the built frontend from frontend/dist.

Frontend Development Mode

Run the backend in one shell:

python -m uvicorn backend.main:app --host 127.0.0.1 --port 8000

Run Vite in another:

cd frontend
npm run dev

Then open the Vite URL, usually:

http://127.0.0.1:5173/

Transparent Profile Overlay

The optional Electron overlay listens to the backend WebSocket at ws://127.0.0.1:8000/ws and shows a click-through popup when the visible deck changes. It appears on startup, active profile changes, and active folder changes, then fades away after 7 seconds.

Start the backend first, then run:

cd overlay
npm run dev

The overlay is frameless, transparent, always on top, and non-interactable so it can sit over other desktop apps without stealing clicks. It does not put a button in the taskbar; the app lives in the Windows notification area with a tray menu for showing the last overlay state or quitting it.

To build the Windows installer locally:

cd overlay
npm run dist:win

The installer is written to overlay/release/.

Operational Notes

Device Detection

By default the backend attempts to auto-detect the Pico using USB metadata. If that fails, the serial port can be set in the application settings.

Persistence

Profiles, folders, button assignments, manual app entries, and event history are stored in SQLite under data/streamdeck.sqlite.

Live Updates

The UI receives state.updated and action/device events over WebSocket. This keeps the browser view aligned with the live backend without manual refreshes.

Windows Integration

App launch discovery currently focuses on Windows environments:

• Start Menu shortcuts
• installed applications from uninstall registry keys
• manually registered app paths

Logging And Diagnostics

The backend records event history in SQLite. During hardware bring-up, pc/listen_buttons.py can still be used as a simple serial diagnostic tool.

Testing

Run the backend test suite from the repo root:

$env:PYTHONPATH = (Get-Location).Path
pytest -q

Deployment Considerations

For a reliable day-to-day setup on a dedicated machine:

• build the frontend and serve the compiled assets from FastAPI
• pin Python dependencies from requirements.txt
• run the backend under a process manager, scheduled task, or startup shortcut
• keep the Pico on a stable USB port when possible
• treat data/ as local runtime state and back it up if profiles matter

This app is best thought of as a local control appliance: hardware input on one side, desktop automation and operator UI on the other.

Overlay Release Automation

The Gitea workflow in .gitea/workflows/overlay-release.yml builds the Electron overlay on a Windows runner for pushes and pull requests that touch overlay/. Pushing a tag that matches v* or overlay-v* also creates or reuses a Gitea release and uploads the generated installer .exe.

The workflow uses the built-in GITEA_TOKEN with releases: write permission, so the repository Actions settings must allow release writes for the job token.

Legacy Utility

For raw serial logging without the full app:

python .\pc\listen_buttons.py

If auto-detect fails:

python .\pc\listen_buttons.py --port COM5