Fix 13 audit findings (startup blocker, credential leaks, retrieval correctness) + CI gates - #1
Conversation
…ctness Ranked batch from the repository audit. Every item was reproduced before the fix and re-verified after; ruff, pytest (89 passed incl. tests/test_parser.py), vitest (12 passed) and tsc are green. Startup / deployment - config.py: CORS_ORIGINS is read as a raw str and parsed in a property. pydantic-settings JSON-decodes List[str] inside EnvSettingsSource *before* field validators run, so docker-compose's comma-separated default raised SettingsError at import and `docker compose up` could never start the API. - config.py: a blank env assignment (FOO=) now falls back to the declared default for the 12 settings where blank is always a mistake. A blank LOCAL_EMBEDDING_MODEL was otherwise a request for the model named "". - docker-compose.yml: forward LLM_PROVIDER, EMBEDDING_PROVIDER, GITHUB_TOKEN, ANTHROPIC_*, OLLAMA_*, DEBUG and the embedding rate-limit knobs. The api service had a hardcoded allowlist and no env_file, so Anthropic, Ollama and private-repo cloning were unreachable under the documented Docker path. Credentials - repo_manager.py: inject the GitHub token only when the parsed hostname is exactly github.com/www.github.com. The previous `"github.com" in url` test also matched github.com.attacker.tld, sending the token to it via an unauthenticated POST /api/repos/. - repo_manager.py: redact the token from git stderr before it is raised. That message is persisted to Repository.indexing_error and streamed by the public /api/repos/{id}/progress endpoint. - .dockerignore: exclude .env (any depth) and .venv. `COPY apps/api .` was baking the developer's live OPENAI_API_KEY into the image, plus a ~470MB Darwin-built venv. Retrieval correctness - chat_cache.py/pipeline.py: include a conversation-history digest in the answer cache key. The prompt contains history but the key did not, so a fresh session was served another session's history-conditioned answer for the 30-minute TTL. - llm/*: on a stream failure with nothing yet delivered, raise instead of yielding "[Error: ...]" as a token. That text was being cached and persisted as the assistant's answer, so one transient upstream error poisoned that question for every later asker. Partial streams still get an inline marker, and pipeline.py refuses to cache a response containing it. - tree_sitter_parser.py: use node.text instead of slicing the decoded str with start_byte/end_byte. Those are byte offsets, so one multi-byte character mis-aligned every chunk after it in the file. - tree_sitter_parser.py: parse .tsx with language_tsx(). The plain TypeScript grammar cannot parse JSX, so every .tsx file produced an ERROR tree. - tree_sitter_parser.py/indexing_service.py: surface has_errors and fall back to raw indexing. tree-sitter returns an ERROR tree rather than raising, so the existing except-clause never fired for a grammar mismatch. - ollama_embeddings.py: a 404 now raises OllamaModelNotFound (not retried, not failed-open) and fail-open gained a failure-ratio ceiling. The default model was a HuggingFace id Ollama cannot resolve, so every chunk exhausted a 10-attempt backoff and landed as a zero vector - which scores every chunk identically and makes retrieval arbitrary at full confidence. - config.py: default local_embedding_model to the Ollama tag nomic-embed-text (the class default was already correct; only config overrode it). API surface - learning.py: bound GenerateQuizRequest.context_content and apply the demo soft limit. It was the only LLM route with neither, so an unauthenticated caller could send an unbounded body straight to the model; added the missing "quiz" bucket, without which the limit call would have been a silent no-op. - main.py: /health uses status.value (a str-Enum f-strings to "IndexingStatus.COMPLETED", so demo mode reported "degraded" permanently) and the LLM check now reports "error: ..." so it actually fails critical_ok instead of passing while the provider is unreachable. - api-client.ts: read the error body once. res.json() consumes the stream even on a parse failure, so the res.text() fallback always threw and every non-JSON error body became a generic message. Also handle FastAPI's array `detail`, since typeof [] === 'object' turned every 422 into that fallback. Dependencies - requirements.txt: tree-sitter>=0.22 (Parser(language) and single-arg Language() are 0.22+; the 0.21 floor permitted a version where importing the parser raises TypeError and the API cannot boot), chromadb>=1.0 (chromadb.errors.NotFoundError), plus the undeclared pyyaml (imported by the /openapi.yaml route) and pytest-cov (required by pyproject's addopts, so pytest could not start on a clean install). - conftest.py: AsyncClient(transport=ASGITransport(...)); the app= shortcut was removed in httpx 0.28. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- ci.yml: run `pytest tests`, not `tests/unit tests/integration`.
tests/test_parser.py sits at the tests/ root, so CI silently skipped ~20 cases
and never constructed a tree-sitter parser -- while local runs (pyproject
testpaths = ["tests"]) covered more than CI did on the same commit.
- ci.yml: install only ruff ad hoc, pinned to 0.16.2. pytest/pytest-asyncio/
pytest-cov/httpx are now declared in requirements.txt, so a fresh clone can run
the suite; pinning stops an unrelated PR going red when ruff adds a rule.
Dropped pytest-mock, which nothing in the repo uses.
- ci.yml: `pnpm install --frozen-lockfile`, matching apps/web/vercel.json. CI was
repairing lockfile drift that the Vercel build then rejected, so a PR could go
green while the production deploy failed on the same commit.
- apps/web/package.json: `test` is `vitest run`; `test:watch` keeps the watcher.
Bare `vitest` only switches to run mode when it detects CI, so the documented
`pnpm test` hung forever locally inside a turbo task that expects an exit.
- apps/web/package.json: `type-check` runs `next typegen && tsc --noEmit`. Next 16
generates route types during dev/build only, so a bare tsc validated no route
types -- and tsconfig includes .next/types/**, absent on a clean checkout.
- apps/web/package.json + turbo.json: add the `clean` task that root
`pnpm clean` already invoked. `turbo clean` failed on an undefined task, so the
`&&` short-circuited and node_modules was never removed.
- turbo.json: `test` dependsOn `^build` instead of `build`. It was running a full
`next build --webpack` before a single vitest test, which is neither needed nor
what CI does.
- turbo.json: globalDependencies points at apps/web/.env.local, the file that
actually feeds the build. The root .env.local it named does not exist, so a
cached web build could ship a stale inlined NEXT_PUBLIC_* value.
- .env.example: comment out the OPENAI_API_KEY placeholder. `sk` is a non-empty
value, so a copied file left the app believing it was configured.
- .env.example + docker/README.md: state where .env goes, which differs by path.
Local dev reads apps/api/.env (config.py loads ".env" relative to the cwd, and
the README starts uvicorn from apps/api); Compose expands ${VAR} from
docker/.env and ignores a repo-root .env. The two docs previously contradicted
each other and neither was right for both cases.
Verified: pytest 89 passed, vitest 12 passed and now exits with CI unset,
next typegen succeeds, turbo clean resolves, ruff clean, and tsc reports zero
errors in project source.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bed0605207
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| context_content: str | ||
| # Bounded like ChatMessageCreate.content (schemas.py): this string goes straight | ||
| # into an LLM prompt, so an unbounded body is an unmetered spend vector. | ||
| context_content: str = Field(..., max_length=20000) |
There was a problem hiding this comment.
Avoid rejecting long lessons before truncation
When a generated lesson's content_markdown exceeds 20 KB, the frontend still sends the full lesson body to this endpoint (api.generateQuiz(..., content.content_markdown)), but LearningService.generate_quiz() only uses context_content[:2000] in the actual prompt. This new validation rejects those long lessons with a 422 before the server can use the safe 2 KB slice, so users can no longer generate quizzes for longer lessons even though no extra LLM spend would occur. Consider truncating before validation or having the client send the already-bounded excerpt.
Useful? React with 👍 / 👎.
Findings from a full-repository audit, ordered by cost today over risk of changing it. Every item was reproduced before the fix and re-verified after.
Why this is worth reviewing first
Two of these are not latent:
docker compose upcould never start the API. pydantic-settings JSON-decodesList[str]insideEnvSettingsSourcebefore field validators run, so the comma-separatedCORS_ORIGINSthatdocker-compose.ymlships as its own default raisedSettingsErrorat import ofsrc.config— before uvicorn bound a port.config.pynow reads it as astrand splits in a property."github.com" in url, which also matchesgithub.com.attacker.tld.POST /api/repos/needs no auth, so an attacker-chosen host received the token. Host is now compared exactly against the parsed hostname, git stderr is redacted before it is raised, and.dockerignorestopsCOPY apps/api .baking a live.envinto the image.What changed
Startup / deployment —
CORS_ORIGINSparsing; blank env assignments fall back to declared defaults rather than becoming""; compose forwardsLLM_PROVIDER,EMBEDDING_PROVIDER,GITHUB_TOKEN,ANTHROPIC_*,OLLAMA_*and the embedding rate-limit knobs, which its hardcoded allowlist had been dropping (Anthropic and Ollama were unreachable under the documented Docker path).Credentials — exact-host token injection; token redacted from
indexing_error, which the public/api/repos/{id}/progressSSE endpoint streams;.envand.venvexcluded from the Docker build context.Retrieval correctness
[Error: ...]as a token. That text was being cached and persisted as the assistant's answer, so one transient upstream error poisoned a question for every later asker.tree_sitter_parserusesnode.textinstead of slicing the decodedstrwithstart_byte/end_byte. Those are byte offsets, so a single multi-byte character mis-aligned every chunk after it in the file..tsxis parsed withlanguage_tsx(). The plain TypeScript grammar cannot parse JSX, andhas_errorwas never checked — tree-sitter returns an ERROR tree rather than raising, so the existingexceptclause never fired. Chunks are now checked and fall back to raw indexing.API surface — quiz input bounded and rate-limited (it was the only LLM route with neither, and the
quizbucket it needed did not exist, so the limit call would have been a silent no-op);/healthuses.value(astr-Enum f-strings toIndexingStatus.COMPLETED, so demo mode reporteddegradedpermanently) and an unreachable LLM now actually fails the check;api-client.tsreads the error body once and handles FastAPI's arraydetail, so 422s and non-JSON bodies stop collapsing to a generic message.Dependencies —
tree-sitter>=0.22(Parser(language)is 0.22+; the old floor permitted a version where importing the parser raisesTypeErrorand the API cannot boot),chromadb>=1.0, plus the undeclaredpyyamlandpytest-cov.Second commit: CI gates that passed for the wrong reasons
pytest tests/unit tests/integration, silently skippingtests/test_parser.pyat thetests/root — so CI never constructed a tree-sitter parser, and local runs covered more than CI on the same commit. Nowpytest tests.pnpm install --frozen-lockfile, matchingvercel.json. CI was repairing lockfile drift that the Vercel build then rejected, so a PR could go green while the deploy failed on the same commit.testisvitest run(barevitestonly exits when it detects CI, sopnpm testhung locally);type-checkrunsnext typegenfirst, since Next 16 generates route types only during dev/build;turbo cleannow resolves, so rootpnpm cleanreaches itsrm -rf node_modulesinstead of short-circuiting.turbo.json:testdepends on^buildnotbuild, dropping a fullnext build --webpackbefore the first test;globalDependenciespoints atapps/web/.env.local, the file that actually exists..env.example/docker/README.mdcontradicted each other on where.envgoes and neither was right for both paths — local dev readsapps/api/.env, Compose expands${VAR}fromdocker/.env. Also commented out theOPENAI_API_KEY=skplaceholder, which was a truthy value that left the app believing it was configured.Verification
pytest testsruff check src testsvitest run(CIunset)next typegentsc --noEmitpytestalso could not run at all on a clean install before this:pyproject'saddoptsrequirespytest-cov, whichrequirements.txtnever declared.Not included
.tsxand byte-offset fixes are correct in code but existing indexes need a re-index to benefit — 28.tsxfiles in the pinned demo repo are still mis-chunked.l2space (embeddings are normalized for cosine) is deliberately left for that same re-index, since the space is immutable after collection creation.parse_github_urlstill accepts non-GitHub hosts, sogit clonecan reach internal addresses. The credential exposure is closed; restricting the host set is a functional decision.🤖 Generated with Claude Code