Skip to content

openai_server: use COLI_TEMP as the gateway default temperature - #986

Open
Blakeolson21 wants to merge 1 commit into
JustVugg:devfrom
Blakeolson21:blake/sib-server-temp-default
Open

openai_server: use COLI_TEMP as the gateway default temperature#986
Blakeolson21 wants to merge 1 commit into
JustVugg:devfrom
Blakeolson21:blake/sib-server-temp-default

Conversation

@Blakeolson21

@Blakeolson21 Blakeolson21 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • use COLI_TEMP as the gateway default when a request omits temperature
  • seed COLI_TEMP from --temp in coli serve, whose gateway runs in-process and never saw the launcher env
  • keep an explicit request temperature authoritative
  • fall back to the existing 0.7 when COLI_TEMP is absent, malformed, or outside the gateway's 0..2 range

Problem

The launcher publishes --temp through COLI_TEMP, following #509 and #968. The API gateway then replaced that value with its own hard-coded 0.7 whenever request JSON omitted temperature.

The built-in non-GLM chat client omits the field, so coli chat --temp 0.25 silently generated at 0.7. coli serve and coli web had the same mismatch twice over: the gateway ignored an exported COLI_TEMP, and --temp never reached the in-process gateway at all because env_for_engine builds a copy for the engine child only.

Use COLI_TEMP at the point where the gateway chooses a missing request default, and have cmd_serve seed it from --temp for the in-process gateway. Explicit API values still win, and an env value that would not pass the gateway's own temperature validation falls back to 0.7 instead of failing requests that never sent the parameter.

Validation

  • python3 -m unittest tests.test_openai_server.TemplateTest.test_coli_temp_is_the_default_for_requests_that_omit_temperature -v
  • python3 -m py_compile openai_server.py
  • make check

The focused test covers the environment default, an explicit request value of zero, and malformed or out-of-range values (malformed, 5, -1, nan, 1e999), each falling back to 0.7. make check passed all C tests and 446 Python tests, with 57 dependency or platform skips.

tnako added a commit to tnako/colibri that referenced this pull request Aug 13, 2026
…ustVugg#986)

coli serve --temp 0.3 (or any engine default published through COLI_TEMP)
was silently discarded: the gateway's generation_options() replaced a
request that omitted temperature with a hardcoded 0.7 before the engine
could see the operator's choice. Now the SERVE frame uses COLI_TEMP as
its default, so the launcher value reaches every engine (laguna included)
unless the request sets temperature explicitly.

Invalid COLI_TEMP values (malformed, out of [0,2], nan) fall back to 0.7.
Tests updated to pin the new default and the explicit-override path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant