Skip to content

Added a feature to VLLM and HuggingFaceLM that forces a response prefix#288

Merged
junya-takayama merged 4 commits into
mainfrom
add_prefix_conditioned_decode
May 25, 2026
Merged

Added a feature to VLLM and HuggingFaceLM that forces a response prefix#288
junya-takayama merged 4 commits into
mainfrom
add_prefix_conditioned_decode

Conversation

@junya-takayama

Copy link
Copy Markdown
Collaborator

Add a feature to VLLM and HuggingFaceLM that forces a prefix for the assistant's responses.

Example

from flexeval import VLLM

lm = VLLM(model="/path/to/model", prefix_str_for_chat="<think></think>")
output = lm.generate_chat_response(["role": "user", "content": "hogehoge"])
print(output.text)
# <think></think>fugafuga

@junya-takayama junya-takayama force-pushed the add_prefix_conditioned_decode branch from 4e14491 to 9fefd96 Compare May 21, 2026 22:26

@Kotaro-Aono Kotaro-Aono left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

問題なさそうです!
LGTM!

@junya-takayama junya-takayama merged commit 37b85c1 into main May 25, 2026
8 checks passed
@junya-takayama junya-takayama deleted the add_prefix_conditioned_decode branch May 25, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants