Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 187 additions & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions packages/discovery-index/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ COPY packages/loopover-engine packages/loopover-engine
COPY packages/discovery-index packages/discovery-index
RUN npm run build --workspace @loopover/engine \
&& npm run build --workspace @loopover/discovery-index \
&& npm run validate:sourcemaps --workspace @loopover/discovery-index \
&& npm prune --omit=dev --workspace @loopover/discovery-index

FROM node:22-slim AS runtime
Expand All @@ -32,5 +33,13 @@ USER discovery-index
ENV PORT=8080
EXPOSE 8080
# Provide at runtime (NOT baked into the image): DISCOVERY_INDEX_SHARED_SECRET (bearer auth for callers),
# DISCOVERY_INDEX_GITHUB_TOKEN (this service's own GitHub token, isolated from any other component's).
CMD ["node", "dist/server.js"]
# DISCOVERY_INDEX_GITHUB_TOKEN (this service's own GitHub token, isolated from any other component's),
# SENTRY_DSN/SENTRY_ENVIRONMENT/SENTRY_RELEASE (error tracking, #4934 -- optional, no-op when SENTRY_DSN is
# unset), and for source-map upload specifically: SENTRY_AUTH_TOKEN, SENTRY_ORG, SENTRY_PROJECT (also
# optional -- upload-sourcemaps.ts skips itself with a log line, never fails the boot, when any is unset).
#
# upload-sourcemaps.js runs FIRST, at container startup (not baked into the image at build time, since
# SENTRY_AUTH_TOKEN is a runtime-only secret) -- deletes the .map files it just uploaded before the server
# starts, so they're never served or left on disk in the running container. Mirrors
# review-enrichment/Dockerfile's identical CMD shape.
CMD ["sh", "-c", "node dist/upload-sourcemaps.js && find dist -type f -name '*.map' -delete && node dist/server.js"]
2 changes: 2 additions & 0 deletions packages/discovery-index/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,5 @@ All opted-in miners authenticate with **one shared `DISCOVERY_INDEX_SHARED_SECRE
## Monitoring

`/health` (liveness) and `/ready` (readiness — reports whether `DISCOVERY_INDEX_GITHUB_TOKEN` is configured) are the two routes an external uptime check should poll; both are exempt from the rate limit. `/metrics` exposes Prometheus-format counters/histograms for request outcomes and latency. This service is **not** wired into the self-host fleet's own Grafana/Alloy stack — it isn't a self-hosted instance, it's the one hosted plane, deployed and observed separately (Cloudflare's own Worker observability, per `wrangler.jsonc`) rather than folded into infrastructure that assumes a self-hoster's own box.

**Error tracking (#4934):** route/process-level errors report into the shared `metagraphed` Sentry project (`jsonbored` org), the same project other infra/operational pieces already flow into — check it first when triaging a spike in `/metrics`' error counters, before digging into Cloudflare's own Worker logs. See `src/sentry.ts` for what gets redacted before an event ever leaves the process, and `README.md`'s "Error tracking & releases" section for the source-map/release-upload path that makes those Sentry stack traces point at real TypeScript instead of a minified bundle.
11 changes: 11 additions & 0 deletions packages/discovery-index/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ Soft-claim design note: the shipped client payload never carries caller identity
| `DISCOVERY_INDEX_CACHE_TTL_MS` | TTL for cached query results, per unique `(repos, orgs, searchTerms)` scope. Default `300000` (5 minutes). |
| `DISCOVERY_INDEX_SOFT_CLAIM_TTL_MS` | TTL for a soft claim before it's reclaimable. Default `1800000` (30 minutes). |
| `PORT` | HTTP port. Default `8080`. |
| `SENTRY_DSN` | Error tracking (#4934), reporting into the shared `metagraphed` Sentry project. Unset ⇒ Sentry stays inert (no-op) everywhere, matching this repo's other services. Non-sensitive (write-only ingest URL) — set as a plain `wrangler.jsonc` var, not a secret. |
| `SENTRY_ENVIRONMENT` | Sentry environment tag. Default `production`. |
| `SENTRY_RELEASE` | This deploy's release identifier (`loopover-discovery-index@<sha>`). Set per-deploy via `wrangler deploy --var`, not a static config value — see `wrangler.jsonc`'s header comment. Falls back to deriving one from `SENTRY_COMMIT_SHA` if unset. |
| `SENTRY_AUTH_TOKEN` | Full Sentry API token, used only by `upload-sourcemaps.ts` at container startup to upload this build's source maps and manage the release. Genuinely sensitive — always a `wrangler secret`. Unset ⇒ source-map upload skips itself with a log line; never fails the boot. |
| `SENTRY_ORG` / `SENTRY_PROJECT` | Sentry org/project slugs for source-map upload. Same non-sensitive/plain-var treatment as `SENTRY_DSN`. |

## Error tracking & releases (#4934)

Runtime errors (route handler failures, unhandled rejections/exceptions) report into the shared **`metagraphed`** Sentry project (`jsonbored` org) that this repo's other infra/operational pieces already flow into — not a separate project. See `src/sentry.ts` for the full redaction rules (secret-named fields and GitHub-token-shaped values are always filtered before anything leaves the process) and `OPERATIONS.md`'s Monitoring section for how this fits alongside `/metrics`.

Source maps and release/deploy tracking are handled by `src/upload-sourcemaps.ts`, which runs as the **first step of the container's runtime `CMD`** (not at Docker build time — `SENTRY_AUTH_TOKEN` is a runtime-only secret, never baked into an image layer): it creates the Sentry release, associates commits, uploads and validates source maps, then records a deploy and finalizes the release, before deleting the `.map` files it just uploaded and starting `server.ts`. Any step failing is non-fatal to the boot unless `DISCOVERY_INDEX_SENTRY_UPLOAD_STRICT=true` is set. Mirrors `review-enrichment/src/upload-sourcemaps.ts`'s identical design.

## Deployment

Expand Down
3 changes: 3 additions & 0 deletions packages/discovery-index/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
},
"scripts": {
"build": "tsc -p tsconfig.json",
"validate:sourcemaps": "node scripts/validate-sourcemaps.mjs",
"start": "node dist/server.js",
"dev": "node --experimental-strip-types --watch src/server.ts",
"cf:dev": "wrangler dev",
Expand All @@ -20,6 +21,8 @@
"@cloudflare/containers": "^0.3.7",
"@hono/node-server": "^1.19.14",
"@loopover/engine": "^3.4.0",
"@sentry/cli": "^3.6.0",
"@sentry/node": "^10.63.0",
"hono": "^4.12.27"
},
"devDependencies": {
Expand Down
Loading
Loading