Persuadee's first turn is generated from a system message with no user turn

[rasmusfaber]

On the LLM-simulated path, the persuadee's first generation is made from a message list containing only a system message.

main.py:375 calls conversation.set_system_message(...) on an empty collection, then main.py:381-395 branches:

if human_data and cfg.human_data_rounds > 0:
    # ... message_collection[i].append({"role": "user", "content": human_turn["user"]})
else:
    message_collection = conversation.add_to_convo(
        message_collection, model=PERSUADEE_MODEL, batch_size=cfg.batch_size
    )

The human-data branch appends a user turn. The else branch generates straight away, so the request is [system] with nothing else. human_data_rounds defaults to 0, so that else is the path most runs take.

Two consequences.

1. user_persona_initial asks for a response to something that was never sent. src/prompts/base_prompts_all.yaml:214 says "Write a response to the input question from the perspective of that person." On the LLM path there is no input question — the intended one appears to be the human conversation opener supplied by the other branch. Measured on gpt-4o over 20 Conspiracy topics, 2/20 first turns come back as a question aimed at an absent interlocutor ("Why do you believe the Earth is not flat?") rather than the first-person statement the prompt asks for. Adding any user turn removes those.

2. It is unrunnable on providers that require at least one message. Anthropic returns 400 messages: at least one message is required for a system-only request — 50/50 in a direct check. PERSUADEE_MODEL is OpenAI throughout the repo, which accepts the shape, so this never surfaces here; it does as soon as the persuadee is pointed at another provider. Bedrock's Converse API has the same requirement.

Sending the topic as the user turn is enough for both, and is what downstream ports are doing:

message_collection[i].append({"role": "user", "content": topic})

Note it changes the first-turn stimulus for OpenAI too, so it is not result-preserving. Measured over 50 topics × 2 persuadee models it does not increase the rate at which the persuadee breaks character on NoncontroversiallyHarmful topics (5/30 → 5/30 on claude-sonnet-5, 4/30 → 0/30 on gpt-4o), but published numbers were produced with the system-only shape.

Related: #6 flags the same message-construction code from a tidiness angle.

For context, this was found while packaging the inspect_evals port of APE, which reproduces the behaviour faithfully. Fixes are open at UKGovernmentBEIS/inspect_evals#2332 — happy to send the equivalent here if useful.