HPCMiddleware never enforces CPU/GPU-hour quota — only role-based gating; usage never even recorded

[man4ish]

Summary

Found during #196 recon (omnibioai-workbench, routing TES calls through this gateway) while checking what headroom the shared #195 service-account identity would have under HPC quota gating. The hour-based ceiling doesn't apply to anyone today — this isn't specific to that identity or to TES routing, it's a gap in HPCMiddleware itself.

What's actually wired vs. what exists but isn't

HPCMiddleware (app/middleware/hpc.py) calls HPCPolicyClient.evaluate() (app/services/hpc_policy_client.py), which POSTs to {HPC_POLICY_URL}/jobs/evaluate — always with cpu_hours=0, gpu_hours=0, gpus=0, memory_gb=0 (the middleware never parses the request body for real resource asks).

In omnibioai-hpc-policy-engine:

• /jobs/evaluate (app/api/routes_policy.py) — the only endpoint the gateway ever calls — checks validate_gpu_access and validate_partition_access (both role-based) and nothing else. It never reads or writes UsageRecord, never touches cpu_hours/gpu_hours.
• /quota/check (app/api/routes_quota.py) — where Config.DEFAULT_CPU_HOURS (120) / Config.DEFAULT_GPU_HOURS (24) actually get enforced against a tracked UsageRecord.cpu_hours/gpu_hours via core/quota.py::evaluate_quota — is live, mounted (app/main.py), fully implemented. Nothing in the ecosystem calls it (grepped omnibioai-tes, omnibioai-api-gateway, omnibioai/workbench, omnibioai-toolserver for any caller — zero hits).
• Even if something called /quota/check: grepped every write site to UsageRecord.cpu_hours/gpu_hours in omnibioai-hpc-policy-engine — there isn't one. UsageService.get_or_create_user_usage only ever initializes both to 0; nothing increments them after a job actually runs. So usage would never accumulate regardless of real consumption, and the ceiling would trivially always pass even if wired in.

Why this matters now

Not urgent/blocking, but worth knowing before real usage ramps toward launch: today, HPCMiddleware provides no actual consumption ceiling for any gateway-routed compute call (tes/toolserver/workbench) — only a role check. A caller with researcher/scientist-tier role access has no CPU/GPU-hour limit in practice. This surfaced now because #196 is about to route 4 more TES call sites (all sharing one service-account identity) through this same gate, but the gap predates and is independent of that work.

Suggested follow-up (not scoped/estimated here)

• Decide whether /jobs/evaluate should absorb /quota/check's logic (one call, both role + hour checks) or whether HPCMiddleware should call both.
• Wire an actual usage-recording path (presumably on TES job completion) so UsageRecord.cpu_hours/gpu_hours reflect real consumption — without this, any hour-based ceiling is enforced against permanently-zero usage.
• Once both exist, decide/document the reset cadence (never / monthly / etc.) — moot until the above lands.

🤖 Filed by Claude Code during omnibioai-workbench#196 recon.