Queue/shedulerarea fix

[suryendudas]

What changed

Fixed a bug in recoverStaleLeases() (apps/backend/src/queue/queue.service.ts) where jobs whose worker lease expired were unconditionally reset to 'queued', with no check against maxAttempts. fail() enforces that cap, but only runs when a worker catches an error — a payload that crashes the worker outright (uncaught exception, OOM, kill) never reaches fail(), so recoverStaleLeases() was the only backstop, and it bypassed the cap entirely. A single "poison pill" job could crash a worker, get reclaimed, crash another worker, and repeat indefinitely.

recoverStaleLeases() now splits stale jobs into two groups using a $expr comparison between attempts and maxAttempts: jobs with attempts remaining are requeued as before, and jobs that already used their last attempt are now marked 'failed' with a clear error message, matching what fail() would have done.

Also updated the state-machine comment in job.model.ts to document the lease-expiry transition (previously undocumented), and added a regression test simulating the poison-pill scenario.

Related issue

Type of change

• [ ☑] Bug fix
• Feature
• Refactor (no behaviour change)
• Docs / comments only
• CI / tooling

Area affected

• [ ☑] Backend (Express / Mongoose)
• Frontend (React / Vite)
• Admin / Train tab (/admin/*)
• Community (/community — posts, comments, auto-answer)
• Search (hybrid text retrieval, training stats)
• Auth / middleware / samagama.in bridge
• [ ☑] Crons / schedulers / embedding-warm
• Observability (Sentry / logging / Discord alerts)
• [☑ ] Docs

CI verification

• cd apps/backend && npx tsc --noEmit exits 0
• cd apps/backend && npx vitest run — all tests pass
• cd apps/frontend && npx tsc --noEmit exits 0
• cd apps/frontend && npx vitest run — all tests pass
• pnpm run lint — 0 errors (152 warnings is the baseline)
• GitHub Actions green on the merge commit (CI, CodeQL, Build & Deploy)
• Tested with a real API hit or browser interaction if behaviour changed
• Tests added or updated for the change
• Single logical change — unrelated fixes noted in description, not fixed here
• Docs updated if route / API / env var / pipeline behaviour changed
• Rebased onto main, no merge commits

Notes for reviewer

Regression test added specifically for this fix: "poison-pill regression" in queue.service.test.ts, simulating a worker that crashes (not a caught error) after using its last attempt. Could not run the test suite in my dev sandbox (no network access to mongodb-memory-server's binary download) — please confirm it passes in your environment before merging.