openai_server: use COLI_TEMP as the gateway default temperature - #986
Open
Blakeolson21 wants to merge 1 commit into
Open
openai_server: use COLI_TEMP as the gateway default temperature#986Blakeolson21 wants to merge 1 commit into
Blakeolson21 wants to merge 1 commit into
Conversation
tnako
added a commit
to tnako/colibri
that referenced
this pull request
Aug 13, 2026
…ustVugg#986) coli serve --temp 0.3 (or any engine default published through COLI_TEMP) was silently discarded: the gateway's generation_options() replaced a request that omitted temperature with a hardcoded 0.7 before the engine could see the operator's choice. Now the SERVE frame uses COLI_TEMP as its default, so the launcher value reaches every engine (laguna included) unless the request sets temperature explicitly. Invalid COLI_TEMP values (malformed, out of [0,2], nan) fall back to 0.7. Tests updated to pin the new default and the explicit-override path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COLI_TEMPas the gateway default when a request omitstemperatureCOLI_TEMPfrom--tempincoli serve, whose gateway runs in-process and never saw the launcher env0.7whenCOLI_TEMPis absent, malformed, or outside the gateway's 0..2 rangeProblem
The launcher publishes
--tempthroughCOLI_TEMP, following #509 and #968. The API gateway then replaced that value with its own hard-coded0.7whenever request JSON omittedtemperature.The built-in non-GLM chat client omits the field, so
coli chat --temp 0.25silently generated at0.7.coli serveandcoli webhad the same mismatch twice over: the gateway ignored an exportedCOLI_TEMP, and--tempnever reached the in-process gateway at all becauseenv_for_enginebuilds a copy for the engine child only.Use
COLI_TEMPat the point where the gateway chooses a missing request default, and havecmd_serveseed it from--tempfor the in-process gateway. Explicit API values still win, and an env value that would not pass the gateway's own temperature validation falls back to0.7instead of failing requests that never sent the parameter.Validation
python3 -m unittest tests.test_openai_server.TemplateTest.test_coli_temp_is_the_default_for_requests_that_omit_temperature -vpython3 -m py_compile openai_server.pymake checkThe focused test covers the environment default, an explicit request value of zero, and malformed or out-of-range values (
malformed,5,-1,nan,1e999), each falling back to0.7.make checkpassed all C tests and 446 Python tests, with 57 dependency or platform skips.