Expand OpenClaw usage metrics exporter

README.md

@@ -1,10 +1,24 @@
-# openclaw-daily-tokens-exporter
+# OpenClaw Usage Exporter
 
-Tiny FastAPI Prometheus exporter for OpenClaw daily token totals.
+FastAPI Prometheus exporter for OpenClaw usage, session, and runtime-adjacent metrics.
 
 ## What it exports
 
-Reads OpenClaw session JSONL files and exposes today's totals at `/metrics`:
+Reads OpenClaw session data from disk and exposes `/metrics` with as much non-sensitive data as possible.
+
+### Why timeframe switching looked wrong before
+
+Prometheus scrapes snapshots. The old exporter leaned heavily on "today so far" gauges, so a Grafana timeframe change could look almost unchanged unless the panel query explicitly used range math.
+
+The exporter now also exposes:
+
+- monotonic all-time counters for tokens, cost, and usage entries
+- retained daily buckets
+- retained hourly buckets
+
+That gives dashboards something real to aggregate over a selected time range.
+
+### Daily usage metrics
 
 - `openclaw_daily_total_tokens`
 - `openclaw_daily_input_tokens`
@@ -13,8 +27,98 @@ Reads OpenClaw session JSONL files and exposes today's totals at `/metrics`:
 - `openclaw_daily_cache_write_tokens`
 - `openclaw_daily_reasoning_tokens`
 - `openclaw_daily_usage_entries`
+- `openclaw_daily_cost_input`
+- `openclaw_daily_cost_output`
+- `openclaw_daily_cost_cache_read`
+- `openclaw_daily_cost_cache_write`
+- `openclaw_daily_cost_total`
+- `openclaw_daily_message_records`
+- `openclaw_daily_non_message_records`
+- `openclaw_daily_unique_sessions`
+- `openclaw_daily_unique_models`
+- `openclaw_daily_unique_providers`
 
-Timezone defaults to `Europe/Berlin`.
+### Daily breakdowns
+
+- `openclaw_daily_messages_total{role=...}`
+- `openclaw_daily_stop_reasons_total{stop_reason=...,model=...,provider=...}`
+- `openclaw_daily_tool_calls_total{tool_name=...}`
+- `openclaw_daily_errors_total{error_type=...}`
+- `openclaw_daily_model_total_tokens{model=...,provider=...}`
+- `openclaw_daily_model_input_tokens{model=...,provider=...}`
+- `openclaw_daily_model_output_tokens{model=...,provider=...}`
+- `openclaw_daily_model_reasoning_tokens{model=...,provider=...}`
+- `openclaw_daily_session_total_tokens{session_id=...}`
+- `openclaw_daily_session_input_tokens{session_id=...}`
+- `openclaw_daily_session_output_tokens{session_id=...}`
+- `openclaw_daily_session_messages_total{session_id=...,role=...}`
+- `openclaw_daily_session_errors_total{session_id=...}`
+- `openclaw_daily_session_tool_calls_total{session_id=...,tool_name=...}`
+
+### Session inventory metrics
+
+Derived from `sessions.json`:
+
+- `openclaw_sessions_visible_count`
+- `openclaw_sessions_total_count`
+- `openclaw_sessions_has_more`
+- `openclaw_session_info{session_id=...,status=...,chat_type=...,last_channel=...,model=...,provider=...,auth_profile_override=...,system_sent=...,aborted_last_run=...,total_tokens_fresh=...}`
+- `openclaw_session_age_seconds{session_id=...}`
+- `openclaw_session_updated_timestamp_seconds{session_id=...}`
+- `openclaw_session_started_timestamp_seconds{session_id=...}`
+- `openclaw_session_last_interaction_timestamp_seconds{session_id=...}`
+- `openclaw_session_total_tokens{session_id=...}`
+- `openclaw_session_input_tokens{session_id=...}`
+- `openclaw_session_output_tokens{session_id=...}`
+- `openclaw_session_context_tokens{session_id=...}`
+- `openclaw_session_runtime_seconds{session_id=...}`
+- `openclaw_session_estimated_cost_usd{session_id=...}`
+- `openclaw_session_compaction_count{session_id=...}`
+- `openclaw_session_usage_family_session_count{session_id=...}`
+
+### Time-range-friendly counters and buckets
+
+- `openclaw_total_tokens_total`
+- `openclaw_input_tokens_total`
+- `openclaw_output_tokens_total`
+- `openclaw_cache_read_tokens_total`
+- `openclaw_cache_write_tokens_total`
+- `openclaw_reasoning_tokens_total`
+- `openclaw_usage_entries_total`
+- `openclaw_cost_input_usd_total`
+- `openclaw_cost_output_usd_total`
+- `openclaw_cost_cache_read_usd_total`
+- `openclaw_cost_cache_write_usd_total`
+- `openclaw_cost_total_usd_total`
+- `openclaw_message_records_total`
+- `openclaw_non_message_records_total`
+- `openclaw_model_total_tokens_total{model=...,provider=...}`
+- `openclaw_model_input_tokens_total{model=...,provider=...}`
+- `openclaw_model_output_tokens_total{model=...,provider=...}`
+- `openclaw_session_total_tokens_total{session_id=...}`
+- `openclaw_day_total_tokens{date=...,tz=...}`
+- `openclaw_day_cost_total{date=...,tz=...}`
+- `openclaw_day_usage_entries{date=...,tz=...}`
+- `openclaw_hour_total_tokens{hour=...,tz=...}`
+- `openclaw_hour_cost_total{hour=...,tz=...}`
+- `openclaw_hour_usage_entries{hour=...,tz=...}`
+
+### Exporter/runtime metrics
+
+- `openclaw_exporter_files_scanned`
+- `openclaw_exporter_lines_scanned`
+- `openclaw_exporter_parse_errors`
+- `openclaw_exporter_history_days_retained`
+- `openclaw_exporter_history_hours_retained`
+- `openclaw_usage_cost_cache_files_tracked`
+- `openclaw_usage_cost_cache_version`
+- `openclaw_usage_cost_cache_updated_timestamp_seconds`
+
+## Privacy boundary
+
+This exporter intentionally skips obvious revealing fields like message content, sender labels, sender ids, phone numbers, response ids, idempotency keys, and session keys.
+
+Session ids are exported.
 
 ## Run locally
 
@@ -28,6 +132,7 @@ uvicorn app:app --host 0.0.0.0 --port 9487
 ## Runtime
 
 - Canonical repo path: `/root/projects/openclaw-usage-exporter`
+- Compatibility workspace path: `/root/.openclaw/workspace/openclaw-daily-tokens-exporter`
 - systemd user unit: `~/.config/systemd/user/openclaw-daily-tokens-exporter.service`
 - Direct metrics: `http://100.74.255.106:9487/metrics`
 - Tailscale Serve path: `https://luna.taile7522.ts.net/openclaw-daily-tokens`
@@ -36,4 +141,8 @@ uvicorn app:app --host 0.0.0.0 --port 9487
 
 - `OPENCLAW_SESSIONS_DIR` default: `/root/.openclaw/agents/main/sessions`
 - `OPENCLAW_EXPORTER_TZ` default: `Europe/Berlin`
-- `PORT` ignored by app directly; set it in the service/uvicorn command
+- `OPENCLAW_SESSIONS_INDEX` default: `/root/.openclaw/agents/main/sessions/sessions.json`
+- `OPENCLAW_USAGE_COST_CACHE` default: `/root/.openclaw/agents/main/sessions/.usage-cost-cache.json`
+- `OPENCLAW_HISTORY_DAYS` default: `30`
+- `OPENCLAW_HISTORY_HOURS` default: `168`
+- `PORT` ignored by app directly; set it in the service or uvicorn command