From 937f80a67af4e4dbd4b872feff046a841700c5df Mon Sep 17 00:00:00 2001 From: Andrew Miller Date: Thu, 9 Jul 2026 10:06:59 -0400 Subject: [PATCH] docs: fix auth model, ports, IMBE limits, and obsolete dashboard env Replace AUTH_ENABLED/WRITE_TOKEN guidance with tr-engine open/token/full auth. Document IMBE-only transcription limitation, correct Caddy/port table, strip obsolete TR_ENGINE_URL/TR_AUTH_TOKEN/SITE_ADDRESS from the dashboard service, and improve CPU/GPU/Pi troubleshooting notes. --- Caddyfile | 16 ++++++- README.md | 116 ++++++++++++++++++++++++++++++--------------- docker-compose.yml | 13 +++-- sample.env | 58 ++++++++++++++--------- 4 files changed, 134 insertions(+), 69 deletions(-) diff --git a/Caddyfile b/Caddyfile index 4ced892..af88c7d 100644 --- a/Caddyfile +++ b/Caddyfile @@ -1,10 +1,22 @@ {$SITE_ADDRESS::80} { - # When AUTH_ENABLED=true, inject read token for browser requests. - # Browsers that send their own Authorization header (e.g. with WRITE_TOKEN) pass through untouched. + # Optional public-read injection for token/full mode when AUTH_TOKEN is set. + # Browsers that already send Authorization (JWT, user-entered token, API key) + # are left untouched. In open mode AUTH_TOKEN is empty and this is a no-op. + # Prefer dashboard auth-init + JWT for full mode when possible. @no_auth not header Authorization * request_header @no_auth Authorization "Bearer {$AUTH_TOKEN:}" handle /api/* { + reverse_proxy tr-engine:8080 { + flush_interval -1 + } + } + + handle /audio/* { + reverse_proxy tr-engine:8080 + } + + handle /health/* { reverse_proxy tr-engine:8080 } diff --git a/README.md b/README.md index f4765cc..2051266 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,11 @@ Full P25 transcription stack in a single `docker compose up`. Includes trunk-rec | Service | Image | Purpose | |---|---|---| -| `trunk-recorder` | `ghcr.io/trunk-reporter/trunk-recorder` | P25 scanner with all common plugins | +| `trunk-recorder` | `ghcr.io/trunk-reporter/trunk-recorder` | P25 scanner with common plugins (incl. MQTT + DVCF) | | `tr-engine` | `ghcr.io/trunk-reporter/tr-engine` | Backend API + call ingestion + transcription routing | -| `tr-dashboard` | `ghcr.io/trunk-reporter/tr-dashboard` | Live web UI | +| `tr-dashboard` | `ghcr.io/trunk-reporter/tr-dashboard` | Live web UI (static files on port 3000) | | `imbe-asr` | `ghcr.io/trunk-reporter/imbe-asr-server` | IMBE vocoder → text (no audio reconstruction) | +| `caddy` | `caddy:2-alpine` | Reverse proxy: `/api/*` → engine, UI → dashboard | | `postgres` | `postgres:17-alpine` | Database | | `mosquitto` | `eclipse-mosquitto:2` | MQTT broker | @@ -38,10 +39,12 @@ docker compose up -d docker compose logs -f tr-engine ``` -**Dashboard:** http://YOUR_SERVER_IP (served by Caddy on port 80) -**API:** http://YOUR_SERVER_IP/api/v1 +**Dashboard:** http://YOUR_SERVER_IP (Caddy on port 80) +**API:** http://YOUR_SERVER_IP/api/v1 -Auth is disabled by default — no login required for local installs. See [Securing for Public Access](#securing-for-public-access) if you want to expose this externally. +Auth starts in **open mode** when `AUTH_TOKEN` and `ADMIN_PASSWORD` are unset (fine for trusted LAN). See [Authentication](#authentication) before exposing the stack externally. + +> **GPU:** The default compose file reserves an NVIDIA GPU for `imbe-asr`. On hosts without NVIDIA / nvidia-container-toolkit, use [CPU-only](#gpu) or the [Pi override](#running-on-raspberry-pi). ## Configuration @@ -54,20 +57,43 @@ POSTGRES_PASSWORD=your-secure-password SITE_ADDRESS=http://192.168.1.100 # your server's IP or hostname ``` -Auth is disabled by default (`AUTH_ENABLED=false`). The dashboard is accessible without login. See [Securing for Public Access](#securing-for-public-access) to enable auth. +See `sample.env` for the full annotated list (auth, MQTT topics, IMBE model, device). + +### Authentication + +tr-engine uses a **three-mode** auth model (same as standalone tr-engine). Mode is derived from environment variables — **`AUTH_ENABLED` is obsolete** and is ignored. + +| Mode | Config | Behavior | +|------|--------|----------| +| **open** | Neither `AUTH_TOKEN` nor `ADMIN_PASSWORD` set | No auth; API and UI are public | +| **token** | `AUTH_TOKEN` only | Shared bearer token required | +| **full** | `ADMIN_PASSWORD` set | JWT login (user `admin`). Optional `AUTH_TOKEN` = public read token from `/api/v1/auth-init` | + +**Public-facing example (recommended):** + +```env +ADMIN_PASSWORD= # openssl rand -base64 32 — login as admin +AUTH_TOKEN= # optional: openssl rand -base64 32 — guest read access +``` + +- Dashboard users log in with username **`admin`** and your `ADMIN_PASSWORD`. +- Caddy can inject `AUTH_TOKEN` for browser requests that do not already send `Authorization` (see `Caddyfile`). Prefer letting the dashboard use `auth-init` / JWT where possible. +- For upload plugins and scripts, create `tre_...` API keys in full mode rather than using the deprecated `WRITE_TOKEN`. -### Securing for Public Access +See the [tr-engine auth migration guide](https://github.com/trunk-reporter/tr-engine/blob/master/docs/migrating-auth.md) for details. -If you're exposing the stack to the internet, enable auth: +### Transcription (IMBE) + +Default stack config: ```env -AUTH_ENABLED=true -AUTH_TOKEN= # openssl rand -base64 32 -WRITE_TOKEN= # openssl rand -base64 32 -ADMIN_PASSWORD= # dashboard login password +STT_PROVIDER=imbe +IMBE_ASR_URL=http://imbe-asr:8000 ``` -With auth enabled, Caddy injects the read token for browser requests automatically. The dashboard will show a login prompt — use username `admin` and your `ADMIN_PASSWORD`. +This requires DVCF capture from trunk-recorder (`tr-plugin-dvcf` is included in the stack image plugins). + +**Limitation:** With `STT_PROVIDER=imbe`, only P25 digital calls that produce `.dvcf` data are transcribed. **Analog / conventional calls get no transcription.** Dual-provider routing (IMBE + audio STT fallback) is tracked as a tr-engine 1.0 blocker. ### config.json @@ -75,9 +101,7 @@ Edit `config.json` to match your SDR hardware and radio system. Key fields: - `sources` — your SDR device, center frequency, sample rate, gain - `systems` — P25 control channel frequency, system type, talkgroup CSV path -- `plugins` — pre-configured for MQTT + DVCF, update `broker` if using external MQTT - -The `broker` in the plugin config points to the internal `mosquitto` container — leave as-is unless you're using an external broker. +- `plugins` — pre-configured for MQTT + DVCF; leave the `broker` pointing at the internal `mosquitto` service unless you use an external broker ### Talkgroup CSV @@ -85,9 +109,10 @@ Place your talkgroup CSV at `talkgroups/talkgroups.csv`. RadioReference format w ## IMBE-ASR Models -Models download from Hugging Face on first run (~560MB for P25 fine-tuned model). The download happens inside the container on startup — check logs with `docker compose logs imbe-asr`. +Models download from Hugging Face on first run (~560MB for the P25 fine-tuned model). Check progress with `docker compose logs imbe-asr`. To pre-download: + ```bash pip install huggingface_hub python3 -c " @@ -96,24 +121,26 @@ snapshot_download('trunk-reporter/imbe-asr-base-512d-p25', local_dir='data/model " ``` -To use the large model (better accuracy on clean speech, 290M params): +Large model (better accuracy on clean speech, 290M params): + ```bash python3 -c " from huggingface_hub import snapshot_download snapshot_download('trunk-reporter/imbe-asr-large-1024d', local_dir='data/models') " ``` -Then set `IMBE_ASR_LM_ALPHA=0.7` and `IMBE_ASR_LM_BETA=2.0` in `.env`. + +Then set `IMBE_ASR_MODEL=trunk-reporter/imbe-asr-large-1024d` (and optional LM tuning vars if your image supports them) in `.env`. ## Running on Raspberry Pi -The stack runs on Raspberry Pi 5 (arm64) using CPU-only inference. A `docker-compose.pi.yml` override handles all Pi-specific configuration automatically. +The stack runs on Raspberry Pi 5 (arm64) using CPU-only inference. A `docker-compose.pi.yml` override handles Pi-specific configuration. ### Prerequisites - Raspberry Pi 5 (4GB+ RAM recommended, 8GB ideal) - 64-bit Raspberry Pi OS (Bookworm or later) -- Docker Engine + Docker Compose v2 installed +- Docker Engine + Docker Compose v2 - RTL-SDR or compatible SDR dongle ### Usage @@ -124,30 +151,27 @@ docker compose -f docker-compose.yml -f docker-compose.pi.yml up -d ``` The Pi override: -- Switches imbe-asr to the `:cpu` image tag (multi-arch, no GPU drivers needed) -- Uses the smaller `imbe-asr-base-512d` model by default (lower memory, faster on ARM) -- Sets `IMBE_ASR_DEVICE=cpu` -- Removes GPU reservation and privileged mode - -### SDR on Pi -RTL-SDR is the most common SDR for Pi deployments. Your `config.json` source settings will differ from x86 setups — make sure to set the correct device index and gain for your dongle. See the [trunk-recorder docs](https://trunkrecorder.com/docs/intro) for source configuration. +- Switches imbe-asr to the `:cpu` image tag (multi-arch, no GPU drivers) +- Uses the smaller `imbe-asr-base-512d` model by default +- Sets `IMBE_ASR_DEVICE=cpu` +- Removes GPU reservation and privileged mode where appropriate ### Performance -CPU inference on Pi is significantly slower than GPU. Expect higher latency on transcriptions. The base model (`imbe-asr-base-512d`) is recommended over the P25-tuned or large models to keep inference time reasonable. Monitor memory usage — if the Pi runs out of RAM, consider reducing `IMBE_ASR_BEAM_WIDTH` in `.env`. - -> **Note:** The [GPU](#gpu) section below does not apply to Pi deployments. +CPU inference on Pi is significantly slower than GPU. Prefer the base model. Monitor RAM; reduce `IMBE_ASR_BEAM_WIDTH` if the Pi swaps or OOMs. ## GPU -GPU is strongly recommended for imbe-asr. The stack assumes NVIDIA GPU with the Docker NVIDIA runtime installed. For CPU-only: +GPU is strongly recommended for imbe-asr. The default `docker-compose.yml` reserves one NVIDIA device. Requirements: NVIDIA driver + [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). + +**CPU-only (non-Pi):** ```env IMBE_ASR_DEVICE=cpu ``` -And remove the `deploy.resources` section from the `imbe-asr` service in `docker-compose.yml`. +And either use `docker-compose.pi.yml` as a template for a CPU override, or edit the `imbe-asr` service to remove `deploy.resources.reservations.devices` so Compose does not require an NVIDIA GPU. ## Upgrading @@ -155,15 +179,30 @@ And remove the `deploy.resources` section from the `imbe-asr` service in `docker docker compose pull && docker compose up -d ``` +Database and audio data under `./data` persist across upgrades. + ## Ports | Service | Port | Notes | |---|---|---| -| tr-engine API | 8080 | Set `HTTP_PORT` in `.env` | -| Caddy (dashboard + API) | 80 | Set `HTTP_PORT` in `.env` | -| Mosquitto MQTT | 1883 | For trunk-recorder instances on other machines | +| Caddy (dashboard + API) | `HTTP_PORT` (default **80**) | Browser entrypoint | +| Mosquitto MQTT | **1883** | For trunk-recorder on other machines | +| tr-engine | **8080** (internal Docker network) | Not published by default; use Caddy `/api/*` | +| tr-dashboard | **3000** (internal) | Static UI behind Caddy | + +All published ports bind to `0.0.0.0` by default. Set `BIND_IP` in `.env` to restrict to a specific interface. + +> **Security note:** Default Mosquitto allows anonymous MQTT. For untrusted networks, lock down the broker and/or bind MQTT to a private interface only. + +## Troubleshooting -All ports bind to `0.0.0.0` by default. Set `BIND_IP` in `.env` to restrict to a specific interface. +| Symptom | Check | +|---------|--------| +| Dashboard blank / API 502 | `docker compose logs caddy tr-engine` | +| Login fails | `ADMIN_PASSWORD` set? Restart after changing `.env` | +| No transcriptions | `STT_PROVIDER=imbe` needs DVCF plugin + healthy `imbe-asr`; analog calls will not transcribe | +| imbe-asr won't start | GPU reservation on non-NVIDIA host → use CPU override | +| MQTT not connecting | `MQTT_TOPICS` must match `config.json` plugin topics | ## Setup Guide @@ -171,12 +210,13 @@ Full step-by-step setup: [docs.luxprimatech.com/#/imbe-asr-setup](https://docs.l ## Roadmap -See the [Trunk Reporter Roadmap](https://github.com/orgs/trunk-reporter/projects/1) for the cross-repo project tracker with priorities and phases. +See the [Trunk Reporter Roadmap](https://github.com/orgs/trunk-reporter/projects/1) for the cross-repo project tracker. ## Related - [tr-docker](https://github.com/trunk-reporter/tr-docker) — trunk-recorder image source + CI - [tr-engine](https://github.com/trunk-reporter/tr-engine) — backend source +- [tr-dashboard](https://github.com/trunk-reporter/tr-dashboard) — UI source - [imbe-asr](https://github.com/trunk-reporter/imbe-asr) — ASR model source - [tr-plugin-dvcf](https://github.com/trunk-reporter/tr-plugin-dvcf) — DVCF plugin - [symbolstream](https://github.com/trunk-reporter/symbolstream) — live streaming plugin diff --git a/docker-compose.yml b/docker-compose.yml index 52b313e..9b12b72 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -51,15 +51,13 @@ services: condition: service_started # --------------------------------------------------------------------------- - # tr-dashboard — Web UI + # tr-dashboard — Web UI (static files on :3000; Caddy reverse-proxies) + # v0.10+ image does not need TR_ENGINE_URL / TR_AUTH_TOKEN / SITE_ADDRESS — + # the browser talks to same-origin /api/* which Caddy routes to tr-engine. # --------------------------------------------------------------------------- tr-dashboard: image: ghcr.io/trunk-reporter/tr-dashboard:latest restart: unless-stopped - environment: - TR_ENGINE_URL: http://tr-engine:8080 - TR_AUTH_TOKEN: ${AUTH_TOKEN:-} - SITE_ADDRESS: ${SITE_ADDRESS:-http://localhost} depends_on: - tr-engine @@ -111,7 +109,8 @@ services: - imbe-asr # --------------------------------------------------------------------------- - # caddy — Reverse proxy: single port for dashboard + API with auth injection + # caddy — Reverse proxy: single port for dashboard + API + # Optional AUTH_TOKEN injection for token/full mode (see Caddyfile) # --------------------------------------------------------------------------- caddy: image: caddy:2-alpine @@ -120,7 +119,7 @@ services: - "${BIND_IP:-0.0.0.0}:${HTTP_PORT:-80}:80" environment: SITE_ADDRESS: ${SITE_ADDRESS:-:80} - AUTH_TOKEN: ${AUTH_TOKEN:-placeholder} + AUTH_TOKEN: ${AUTH_TOKEN:-} volumes: - ./Caddyfile:/etc/caddy/Caddyfile - caddy_data:/data diff --git a/sample.env b/sample.env index 2618628..1fc2776 100644 --- a/sample.env +++ b/sample.env @@ -1,16 +1,18 @@ # ============================================================================= # tr-stack — Environment Configuration -# Copy to .env and fill in your values. +# Copy to .env and fill in your values: cp sample.env .env # ============================================================================= # IP to bind external ports (default: all interfaces) # BIND_IP=192.168.1.100 -# Port Caddy listens on +# Port Caddy listens on (dashboard + /api reverse proxy) HTTP_PORT=80 -# Public-facing address (what browsers use — your server's IP or hostname) -# Examples: http://192.168.1.100 or https://radio.example.com +# Public-facing address (what browsers use — your server's IP or hostname). +# Used by Caddy's site address matcher. Examples: +# SITE_ADDRESS=http://192.168.1.100 +# SITE_ADDRESS=https://radio.example.com SITE_ADDRESS=http://YOUR_SERVER_IP # PostgreSQL credentials @@ -18,36 +20,48 @@ POSTGRES_USER=trengine POSTGRES_PASSWORD=change-me POSTGRES_DB=trengine -# Auth — disabled by default for local installs -# Set AUTH_ENABLED=true and configure tokens/password for public-facing deployments -AUTH_ENABLED=false +# ----------------------------------------------------------------------------- +# Authentication (tr-engine three-mode model) +# Mode is derived from which vars you set — AUTH_ENABLED is obsolete. +# ----------------------------------------------------------------------------- +# open — leave AUTH_TOKEN and ADMIN_PASSWORD unset (default for trusted LAN) +# token — set AUTH_TOKEN only; browser clients use that shared bearer token +# full — set ADMIN_PASSWORD (seeds admin user + JWT login). Optional AUTH_TOKEN +# becomes a public read token returned by GET /api/v1/auth-init. +# +# For public-facing deployments: set ADMIN_PASSWORD. Prefer API keys (tre_...) +# for upload plugins/scripts. WRITE_TOKEN is deprecated in tr-engine. +# +# AUTH_TOKEN= # openssl rand -base64 32 +# ADMIN_PASSWORD= # dashboard login password (username: admin) -# API tokens (only needed when AUTH_ENABLED=true) -# AUTH_TOKEN= # generate with: openssl rand -base64 32 -# WRITE_TOKEN= -# ADMIN_PASSWORD= - -# MQTT topics — must match your trunk-recorder plugin config +# MQTT topics — must match your trunk-recorder plugin config in config.json MQTT_TOPICS=tr/# -# Transcription provider +# ----------------------------------------------------------------------------- +# Transcription +# ----------------------------------------------------------------------------- +# Default stack uses IMBE ASR (P25 digital via DVCF). Analog/conventional calls +# are NOT transcribed while STT_PROVIDER=imbe. For mixed systems you currently +# need a separate audio STT setup (or wait for dual-provider STT in tr-engine). STT_PROVIDER=imbe IMBE_ASR_URL=http://imbe-asr:8000 -# IMBE-ASR model selection (downloads automatically from HuggingFace on first run) +# IMBE-ASR model (downloaded from HuggingFace on first run if missing) # Options: -# trunk-reporter/imbe-asr-base-512d-p25 — P25 fine-tuned, best for live radio (default) -# trunk-reporter/imbe-asr-large-1024d — 290M model, best accuracy on clean speech -# trunk-reporter/imbe-asr-base-512d — base model, good for edge/Pi deployment +# trunk-reporter/imbe-asr-base-512d-p25 — P25 fine-tuned (default) +# trunk-reporter/imbe-asr-large-1024d — larger model, cleaner speech +# trunk-reporter/imbe-asr-base-512d — base model (good for Pi/edge) IMBE_ASR_MODEL=trunk-reporter/imbe-asr-base-512d-p25 IMBE_ASR_DEVICE=cuda # set to 'cpu' if no GPU (automatic with docker-compose.pi.yml) +# IMBE_ASR_BEAM_WIDTH=100 # Raspberry Pi notes: -# - Use docker-compose.pi.yml override (sets cpu device + smaller model automatically) -# - The base-512d model is recommended for Pi — lower memory and faster inference -# - See README.md "Running on Raspberry Pi" section for full details +# - Use: docker compose -f docker-compose.yml -f docker-compose.pi.yml up -d +# - Pi override forces CPU image + smaller default model +# - See README "Running on Raspberry Pi" # Optional: point at trunk-recorder's audio dir for file-based ingest # TR_AUDIO_DIR=/app/audio -# Path to libimbe.so on your host (required for IMBE decoding) +# Path to libimbe.so on your host (if required by your imbe-asr image) # LIBIMBE_PATH=/usr/local/lib/libimbe.so