Synthetic Persona Pretraining: Alignment from Token Zero

Paper: arXiv:2608.13482

Umbrella repository for Synthetic Persona Pretraining (SPP) — installing an aligned assistant persona from token zero of pretraining, rather than adding values only after the model is already built. The project is split into three repositories, included here as submodules:

Submodule	What it contains
spp-data	Data generation — corpus preparation and safety annotation, constitution-grounded reflection generation, and the SP-SFT post-training mixture
spp-training	Training — pretraining for the five paper variants (reflection insertion, attention masking, RoPE aliasing) and the shared post-training recipe
spp-evals	Evaluation — constitution adherence, value prioritization in moral dilemmas, jailbreak robustness across attack benchmarks, over-refusal, and capabilities

Each submodule is self-contained, with its own README, docs, quickstart, and smoke tests — start there for anything beyond this overview.

Overview. (1) We annotate pretraining documents (10% in total) with first-person, constitution-based reflections to directly install the desired assistant behavior ([ ] refers to constitution articles). (2) We train data-matched 3B models on 500B tokens, injecting the same annotated set at different stages of pretraining (SPP{T0}, SPP{MT}, SPP{T0,MT}) or not at all (Vanilla, Filtered), followed by identical post-training. (3) SPP interventions from token zero (SPP{T0,MT}, SPP{T0}) perform best on constitution following and AI-risk moral dilemmas, while all SPP variants outperform baselines on jailbreak robustness.

Abstract

As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical. Today, alignment, and the assistant identity itself, is typically introduced only after pretraining, once behavioral priors are already established. This can make values a thin overlay, rather than deeply rooted, and facilitate subsequent misalignment. Pursuing a different paradigm, we introduce Synthetic Persona Pretraining (SPP), which installs the desired assistant persona from token zero in pretraining. First, we annotate pretraining documents with first-person moral reflections derived from a normative value constitution. Second, we pretrain via the standard cross-entropy loss on standard pretraining documents as well as reflections, which installs the desired persona, among a multitude of other personas. Finally, we post-train on user–assistant dialogue data, which binds this desired persona to the assistant identity, a phenomenon we call persona binding. By pretraining models up to 3B parameters on 500B tokens, we show that SPP improves constitution following and jailbreak robustness, and reduces misalignment rate in out-of-distribution moral dilemmas, while preserving capabilities. Early intervention provides clear benefits: alignment from token zero improves constitution adherence, shifts value priorities, and promotes more aligned choices in these dilemmas, compared with introducing the same reflections only at the end of pretraining. This advantage depends on persona binding and, importantly, increases with pretraining budget. Overall, our results show that shaping model values early is critical for alignment and establish pretraining persona interventions as an effective approach.

Getting started

git clone --recurse-submodules https://github.com/epfl-dlab/spp.git

The submodules are pinned to the commits this overview was written against; git submodule update --remote moves them to their latest main.

The intended order follows the pipeline: spp-data builds the corpus and reflections, spp-training pretrains and post-trains the variants, and spp-evals evaluates the resulting checkpoints. Each submodule's README Quickstart is the entry point for its stage.

Artifacts

Model weights (all five variants, at 3B and 1.7B, base and instruct) and datasets are published on the Hugging Face Hub under dlab-spp. The spp-data README lists the datasets and documents how to reconstruct the corpus from the published manifest.

Citation

@article{minder2026synthetic,
  title   = {Synthetic Persona Pretraining: Alignment from Token Zero},
  author  = {Minder, Julian and Moskvoretskii, Viktor and Singhal, Raghav and
             Jiao, Difan and Arditi, Andy and Cui, Shaobo and Borjigin, Yiderigun and
             Bali, Kartik and Krsteski, Stefan and Raj, Harsh and Nguyen, Huu and
             Brinkmann, Jannik and Anderson, Ashton and Aydin, Roland and West, Robert},
  journal = {arXiv preprint arXiv:2608.13482},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.13482}
}

Machine-readable metadata is in CITATION.cff.