docs(sglang): rewrite the (sglang, LMCache) sample for the shipped MP data plane#161
Open
EdHasNoLife wants to merge 3 commits into
Open
docs(sglang): rewrite the (sglang, LMCache) sample for the shipped MP data plane#161EdHasNoLife wants to merge 3 commits into
EdHasNoLife wants to merge 3 commits into
Conversation
… data plane The sample still described the pre-MP lm:// model with a 'KNOWN LIMITATION / caches nothing / fix pending' warning — now false: SGLang offloads via the MP data plane (managed Redis L2 + node-local MP-worker sidecar + --lmcache-config-file). Rewrite the sample to that model: redisImage (not the lm:// serverImage), the K8s>=1.29 native-sidecar requirement, and the MP worker tunables. verify-samples passes.
Codex reviewFiles reviewedconfig/samples/
FindingsBlockingNone. Should-fix
NitNone. Per-category coverage
Verdictchanges-requested |
Distinguish the two input surfaces: a pod-template value colliding with an adapter-owned key is upserted (overwritten), while the hard-reject at admission is specifically for engineOverrides that touch a reserved arg/env.
Codex reviewFiles reviewed
FindingsBlockingNone. Should-fixNone. Nitconfig/samples/cachebackend-sglang.yaml:25 — The list abbreviates environment-variable names and omits config/samples/cachebackend-sglang.yaml:74 — This comment is 109 characters and breaks the surrounding wrapping style. Wrap it onto another line for consistency and readability. Per-category coverage
Verdictapprove-with-nits |
EdHasNoLife
force-pushed
the
docs/sglang-mp-sample
branch
from
July 23, 2026 20:46
2eb3018 to
a3031d6
Compare
EdHasNoLife
force-pushed
the
docs/sglang-mp-sample
branch
from
July 23, 2026 20:47
a3031d6 to
858ec19
Compare
Codex reviewFiles reviewed
FindingsBlockingNone. Should-fixNone. NitNone. Per-category coverage
Verdictapprove |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
(sglang, LMCache)sample still described the pre-MPlm://model with a "KNOWN LIMITATION (GPU-validated, fix pending) … this pair can reconcile Ready while caching nothing" header — which is now false. #149 landed the SGLang LMCache MP data plane, so SGLang genuinely offloads. An operator copying the sample today is told a working feature is broken.Rewrite it to the shipped MP model:
redisImagefor the managed Redis L2 store the MP worker offloads to — replaces thelm://serverImage(lm://is not a valid MP--l2-adaptertype, so SGLang doesn't render the standalone lmcache-server).--enable-lmcache,--lmcache-config-file,LMCACHE_USE_EXPERIMENTAL), and the fact that the oldlm://env /VLLM_USE_V1/PYTHONHASHSEEDare not injected for SGLang.workerImage/chunkSize/mpPort/l1SizeGB) as optional reference.make verify-samplespasses (server-side admission dry-run):cachebackend-sglang.yaml — OK.Scope note. While scoping this I found the
cachebackend-api.mdSGLang section was already rewritten to MP (part of the #149 line) — so only the sample was stale here. The reference-stack (docs/reference-stack/manifests/sglang-lmcache/) is also still on thelm://model, but it's a GPU-validated hand manifest that wants kind re-validation, not a blind edit — it stays tracked in CAC-251 along with the missingVERSIONS.mdtuple and the MP smoke assert.CAC-251.