Skip to content

OpenAI provider incorrectly maps params(top_k=) to top_logprobs #412

Description

@cpsievert

Summary

chatlas/_provider_openai.py maps the standardized top_k chat parameter onto OpenAI's top_logprobs field:

# chatlas/_provider_openai.py:596-597
if "top_k" in params:
    res["top_logprobs"] = params["top_k"]

These are unrelated parameters — top_k controls sampling (limiting the candidate token pool), while top_logprobs controls how many log-probabilities are returned per token. Conflating them means setting top_k silently changes response metadata (logprobs count) instead of doing anything related to sampling, and vice versa.

ellmer just fixed the equivalent bug across its OpenAI-based providers (tidyverse/ellmer#1115, tidyverse/ellmer#1113): top_k is no longer sent as top_logprobs for chat_openai(), chat_deepseek(), or the generic OpenAI-compatible provider.

Expected fix

Stop mapping top_ktop_logprobs in chatlas/_provider_openai.py (and check chatlas/_provider_openai_completions.py / other OpenAI-compatible providers for the same pattern). Since OpenAI's API doesn't support top_k for sampling, the standardized top_k param should likely be dropped/warned-on for OpenAI-based providers rather than silently repurposed, matching ellmer's resolution.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Priority: MediumValid bug or well-defined request with moderate impact or a workaround.ai-triage:doneMarks an issue whose AI triage workflow is complete.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions