feat. rebuild service in Go
Rebuild the Gitea Codex review bot from the product contract with a Go HTTP service, durable SQL queue, typed Gitea client, isolated runner, and deployment updates. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -4,173 +4,81 @@ Guidance for autonomous/code-assist agents working in this repository.
|
||||
|
||||
## Mission
|
||||
|
||||
Build and maintain a webhook-driven Gitea PR review bot that:
|
||||
Maintain a Go service that:
|
||||
|
||||
1. verifies webhook authenticity,
|
||||
1. verifies Gitea webhook authenticity,
|
||||
2. parses `@codex` commands,
|
||||
3. queues and executes review jobs,
|
||||
4. posts/updates PR comments with structured findings.
|
||||
3. queues and executes durable review jobs,
|
||||
4. checks out the exact pull-request head SHA in an isolated runner,
|
||||
5. posts structured findings back to Gitea.
|
||||
|
||||
Primary implementation lives under `src/gitea_codex_bot`.
|
||||
## Tech stack
|
||||
|
||||
## Tech Stack
|
||||
- Go 1.25+
|
||||
- `net/http`
|
||||
- `database/sql`
|
||||
- MariaDB in production; SQLite for local tests
|
||||
- embedded Go migrations
|
||||
- Docker-based review runner
|
||||
- standard Go tests, race detector, vet, and formatting checks
|
||||
|
||||
- Python `>=3.11`
|
||||
- FastAPI + Uvicorn
|
||||
- SQLAlchemy + Alembic
|
||||
- MariaDB (default), SQLite possible via `DATABASE_URL`
|
||||
- Pytest for tests
|
||||
- Docker-based review runner with host fallback
|
||||
## Repository map
|
||||
|
||||
## Repository Map
|
||||
- `cmd/gitea-codex/main.go` — process startup, signals, migrations, HTTP server, worker.
|
||||
- `internal/config` — environment loading and startup validation.
|
||||
- `internal/domain` — typed commands, job/run states, review results, and policies.
|
||||
- `internal/commands` — mention aliases and safe command lexer.
|
||||
- `internal/webhook` — raw-body HMAC and typed event extraction.
|
||||
- `internal/httpapi` — routes, webhook acknowledgements, and health endpoints.
|
||||
- `internal/store` and `internal/store/sqlstore` — storage contracts, migrations, transactions, and queue claims.
|
||||
- `internal/gitea` — typed Gitea REST client.
|
||||
- `internal/review` — repository config, prompts, result validation, and comment formatting.
|
||||
- `internal/runner` — Docker/Codex execution and cleanup.
|
||||
- `internal/worker` — queue orchestration, retries, stale recovery, and non-review commands.
|
||||
- `migrations` — logical compatibility baseline and migration notes.
|
||||
- `tests under internal/*` — unit and fake-service integration tests.
|
||||
|
||||
- `src/gitea_codex_bot/main.py`
|
||||
- FastAPI app, `/healthz`, `/webhook/gitea`, lifespan worker boot.
|
||||
- `src/gitea_codex_bot/config.py`
|
||||
- Environment-backed settings and DB URL composition.
|
||||
- `src/gitea_codex_bot/db.py`
|
||||
- Engine/session factory and dependency session provider.
|
||||
- `src/gitea_codex_bot/models.py`
|
||||
- ORM models: `WebhookEvent`, `ReviewJob`, `ReviewRun`, `BotComment`.
|
||||
- `src/gitea_codex_bot/services/commands.py`
|
||||
- `@codex` command parsing.
|
||||
- `src/gitea_codex_bot/services/jobs.py`
|
||||
- Event dedupe, queue transitions, cooldown logic.
|
||||
- `src/gitea_codex_bot/services/security.py`
|
||||
- HMAC signature verification and payload digest.
|
||||
- `src/gitea_codex_bot/services/gitea.py`
|
||||
- Gitea API client wrapper.
|
||||
- `src/gitea_codex_bot/services/reviewer.py`
|
||||
- PR checkout/diff collection/prompt build/OpenAI call/fallback/fix helpers.
|
||||
- `src/gitea_codex_bot/services/review_format.py`
|
||||
- Outbound comment formatting.
|
||||
- `src/gitea_codex_bot/services/comments.py`
|
||||
- Persistent summary comment id tracking.
|
||||
- `src/gitea_codex_bot/workers/dispatcher.py`
|
||||
- Job polling and orchestration.
|
||||
- `src/gitea_codex_bot/workers/container_runner.py`
|
||||
- Docker review execution + fallback.
|
||||
- `alembic/` + `alembic.ini`
|
||||
- Database migrations.
|
||||
- `tests/`
|
||||
- Unit/integration-ish tests across config/security/jobs/webhook/migrations.
|
||||
- `.gitea/workflows/ci.yml`
|
||||
- CI test + publish workflow.
|
||||
## Runtime flow
|
||||
|
||||
## Runtime Flow
|
||||
1. Gitea sends a signed webhook to `POST /webhook/gitea`.
|
||||
2. The handler verifies the raw body, filters event/repository/bot policy, parses the command, deduplicates, and persists a job.
|
||||
3. The worker claims the oldest queued job and records a run attempt.
|
||||
4. Review/rerun jobs fetch PR metadata, enforce fork policy, read `.codex-review.yml` at the exact head SHA, and invoke the isolated runner.
|
||||
5. The runner checks out the exact head SHA, verifies `git rev-parse HEAD`, invokes Codex, and returns strictly validated JSON.
|
||||
6. The worker posts a new result/failure/acknowledgement comment and finalizes durable state.
|
||||
|
||||
1. Gitea sends webhook to `POST /webhook/gitea`.
|
||||
2. Signature is validated (`X-Gitea-Signature`, sha256 HMAC).
|
||||
3. Non-supported events are ignored.
|
||||
4. PR context and command are extracted.
|
||||
5. Repo allowlist and dedupe checks run.
|
||||
6. Job is enqueued (`review_jobs`).
|
||||
7. Background worker claims queued jobs.
|
||||
8. For review/rerun:
|
||||
- fetch PR context,
|
||||
- run review in ephemeral container if possible,
|
||||
- fallback to host execution on failure,
|
||||
- post or edit persistent PR summary comment.
|
||||
9. Job/run status transitions are persisted.
|
||||
## Compatibility guardrails
|
||||
|
||||
## Supported Commands
|
||||
- Preserve HMAC verification, allowlisting, bot self-comment filtering, and event/job dedupe.
|
||||
- Preserve response reasons and command behavior unless intentionally versioned.
|
||||
- Keep cooldown for `review`; `rerun` bypasses cooldown.
|
||||
- Allow two requeues after an initial failed attempt, then fail terminally.
|
||||
- Recover running jobs after the five-minute lease timeout.
|
||||
- Skip fork reviews by default.
|
||||
- Do not add host-side Codex fallback.
|
||||
- Keep posting new review comments while updating the `bot_comments` latest mapping; do not silently change to edit-in-place behavior.
|
||||
- Never mark infrastructure failure as a successful review.
|
||||
- Treat PR content, comments, `.codex-review.yml`, and model output as untrusted data.
|
||||
|
||||
- `@codex review [security|performance|tests] [--full]`
|
||||
- `@codex rerun`
|
||||
- `@codex explain`
|
||||
- `@codex ignore`
|
||||
## Security-sensitive areas
|
||||
|
||||
## Local Development
|
||||
- Do not log Gitea/OpenAI tokens, auth JSON, Docker arguments containing secrets, raw prompts, or unbounded provider output.
|
||||
- Do not pass Docker socket access into review containers.
|
||||
- Pin runner images and Codex versions for production.
|
||||
- Enforce body/output limits, context cancellation, container cleanup, and exact SHA verification.
|
||||
- Keep repository configuration from controlling credentials, images, host paths, commands, privileges, or network policy.
|
||||
- Review changes to `internal/runner`, `internal/webhook`, `internal/store/sqlstore`, and `internal/gitea` carefully.
|
||||
|
||||
Install and run:
|
||||
## Development checks
|
||||
|
||||
Before proposing a change:
|
||||
|
||||
```bash
|
||||
python -m pip install -e .[dev]
|
||||
alembic upgrade head
|
||||
uvicorn gitea_codex_bot.main:app --host 0.0.0.0 --port 8000
|
||||
gofmt -w cmd internal
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build -trimpath ./cmd/gitea-codex
|
||||
```
|
||||
|
||||
Run tests:
|
||||
|
||||
```bash
|
||||
pytest
|
||||
```
|
||||
|
||||
Docker compose:
|
||||
> Locally only run this, is pre setup to use the dev compose file.
|
||||
```bash
|
||||
docker compose up --build -f docker-compose.dev.yml
|
||||
```
|
||||
|
||||
## Environment Contract
|
||||
|
||||
Required:
|
||||
|
||||
- `GITEA_BASE_URL`
|
||||
- `GITEA_TOKEN`
|
||||
- `GITEA_BOT_USERNAME`
|
||||
- `GITEA_WEBHOOK_SECRET`
|
||||
- `ALLOWED_REPOS`
|
||||
- `DB_HOST`, `DB_PORT`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`
|
||||
|
||||
Common optional:
|
||||
|
||||
- `DATABASE_URL` (overrides DB parts)
|
||||
- `OPENAI_API_KEY` (required when `CODEX_AUTH_MODE=api_key`)
|
||||
- `OPENAI_PROJECT_ID`, `OPENAI_ORG_ID`
|
||||
- `OPENAI_REVIEW_MODEL`
|
||||
- `CODEX_AUTH_MODE` (`api_key` default, `chatgpt` supported)
|
||||
- `CODEX_AUTH_JSON_PATH` (custom path to `auth.json` for `chatgpt` mode)
|
||||
- `WORKDIR`, `MAX_DIFF_BYTES`, `MAX_REVIEW_MINUTES`, `CONCURRENCY`
|
||||
- `REVIEW_RUNNER_IMAGE`
|
||||
- `ALLOW_UNTRUSTED_FORKS`
|
||||
|
||||
## Database and Migrations
|
||||
|
||||
- SQLAlchemy models are authoritative for runtime behavior.
|
||||
- Alembic migrations in `alembic/versions` must track schema changes.
|
||||
- If model schema changes, add a migration and keep migration tests passing.
|
||||
- CI runs `alembic upgrade head` before pytest.
|
||||
|
||||
## Testing Expectations
|
||||
|
||||
Before opening/merging changes:
|
||||
|
||||
1. run `pytest`,
|
||||
2. if DB/model changes were made, ensure migration test still passes,
|
||||
3. for webhook/queue logic, add or update focused tests in `tests/`.
|
||||
|
||||
Current tests rely on `tests/conftest.py` to inject default env and DB URL behavior.
|
||||
|
||||
## Change Guardrails
|
||||
|
||||
- Preserve webhook security checks and allowlist semantics.
|
||||
- Preserve dedupe constraints (`delivery_id`, `repo+comment_id`, `repo+trigger_comment_id`).
|
||||
- Keep bot self-comment ignore behavior.
|
||||
- Keep persistent comment update behavior (avoid comment spam regressions).
|
||||
- Be explicit when changing runner isolation/fallback behavior; this is a security-sensitive area.
|
||||
- Keep response payloads and command parsing backward compatible unless intentionally versioned.
|
||||
|
||||
## Known Risks / Active Gaps
|
||||
|
||||
See `TODO.md` for priority backlog, especially:
|
||||
|
||||
- stronger isolated runner flow,
|
||||
- stricter host fallback controls,
|
||||
- end-to-end integration coverage.
|
||||
|
||||
Treat these as high-sensitivity areas when modifying worker/runner paths.
|
||||
|
||||
## Recommended Workflow for Agents
|
||||
|
||||
1. Read touched service + corresponding tests first.
|
||||
2. Make minimal cohesive changes.
|
||||
3. Add/update tests with behavior changes.
|
||||
4. Run `pytest`.
|
||||
5. Summarize impact, risks, and follow-ups in PR/commit notes.
|
||||
|
||||
## Commiting After Completion
|
||||
If you are confident that your changes are ready to be committed, please follow the commit message format below:
|
||||
|
||||
```[type]. Short description (max 50 chars)```
|
||||
Push after commiting. Ask the user once if you have permission to commit and from then on commit without asking.
|
||||
Changes affecting migrations, queue claims, or HTTP contracts require focused tests. MariaDB locking behavior must be verified separately from SQLite; SQLite tests do not prove `FOR UPDATE SKIP LOCKED` correctness.
|
||||
|
||||
Reference in New Issue
Block a user