AntiDoom & FTPO — Empirical Research Reproduction

An empirical reproduction and extended bench-marking pipeline for Liquid AI's AntiDoom research and Final Token Preference Optimization (FTPO).

This repository implements the end-to-end data extraction, preference pair construction, LoRA FTPO training, and evaluation pipeline designed to eliminate infinite repetition loops ("doom loops") in reasoning language models (e.g., LiquidAI/LFM2.5-1.2B-Base).

---

Key Experimental Findings

Using vLLM and a fingerprint sliding-window loop detection algorithm, we evaluated 4,996 prompts from LiquidAI/antidoom-mix-v1.0 on Kaggle Dual NVIDIA T4 GPUs:

Metric	Baseline (Pre-FTPO)	Post-FTPO (Verified Model)	Improvement / Delta
Evaluated Prompts	4,996 prompts	300 test prompts	—
Doom Loop Count	476 loops	24 loops	452 loops eliminated
Doom Loop Percentage (%)	9.53%	8.00%	-1.53% percentage points
Relative Doom Reduction	Baseline	16.0% Relative Reduction	16.0% fewer doom loops
Training Duration	—	13.2 minutes	1 epoch (14 steps, fp16 + LoRA)

---

Visualizations & Empirical Analysis

1. FTPO Impact Dashboard

The dashboard summarizes the overall impact of FTPO on model generation stability, showing the proportion of doom loops eliminated versus remaining non-loop completions.

---

2. Doom Loop Reduction Rate

Relative percentage reduction in repetition degeneration following FTPO LoRA adaptation.

---

3. Pre vs. Post FTPO Loop Percentage

Direct side-by-side comparison of baseline repetition rates vs. post-FTPO verified model rates across evaluation batches.

---

4. Top Doom-Inducing Trigger Tokens

Distribution of the top rejected tokens that triggered runaway repetition loops during baseline sampling (e.g., overtrained transition tokens like Wait, So, Alternatively).

---

5. Completion Length Distribution

Histogram comparing completion character lengths for normal reasoning completions vs. runaway doom loops.

---

6. Multi-Model Loop Comparison

Comparative benchmark of repetition failure rates across baseline vs. aligned checkpoints.

---

Temperature Sensitivity Analysis

We evaluated generation stability across a range of sampling temperatures ($T = 0.01$ to $T = 0.70$):

Temperature ($T$)	Baseline Loop Rate (Pre-FTPO)	Post-FTPO Verified Rate	Observation
0.01	9.09% (4/44)	9.67% (29/300)	Near-deterministic greedy sampling
0.05	4.55% (2/44)	9.00% (27/300)	Low-entropy sampling regime
0.10	6.82% (3/44)	8.67% (26/300)	Recommended baseline sampling temperature
0.30	2.27% (1/44)	9.67% (29/300)	Moderate exploration
0.70	0.00% (0/44)	6.67% (20/300)	High-entropy sampling (natural loop escape)

---

How FTPO Works

The Failure Mode: "Doom Loops"

Doom loops occur during complex step-by-step reasoning when three factors align:

1. Overtrained Transition Tokens: Tokens like Wait, So, Alternatively become over-represented during synthetic reasoning tuning.
2. Self-Reinforcing Context: Once a phrase repeats once, the self-attention mechanism assigns higher probability to repeating it again.
3. Low-Temperature Sampling: At low temperatures ($T \approx 0.1$), the model cannot escape locally reinforced high-probability tokens.

graph TD
    A["Prompt Context"] --> B["Generate Token z_rejected (e.g. 'Wait')"]
    B --> C{"Is Token Repeated?"}
    C -- "Yes" --> D["Doom Loop Triggered! (Infinite Repetition)"]
    C -- "No" --> E["Continue Normal Reasoning"]
    D --> F["Extract Rejected Token + Alternative Chosen Logprobs"]
    F --> G["Train LoRA Adapter with FTPO Loss"]
    G --> H["Aligned Model (Loop Free)"]

Loading

The FTPO Loss Function

Instead of training on full sequence targets, FTPO (Final Token Preference Optimization) penalizes only the single rejected token that triggered the loop boundary, while tethering non-target vocabulary to the reference model:

$$\mathcal{L}_{\text{FTPO}} = \mathcal{L}_{\text{pref}} + \lambda_{\text{mse}} \mathcal{L}_{\text{tether}} + \lambda_{\text{tgt}} \mathcal{L}_{\text{deadzone}}$$

1. Logit Margin Preference Loss ($\mathcal{L}_{\text{pref}}$): $$\mathcal{L}{\text{pref}} = \text{softplus}\left(\frac{\epsilon - (z{\text{chosen}} - z_{\text{rejected}})}{\tau}\right) \cdot w$$
2. Vocabulary MSE Tether ($\mathcal{L}_{\text{tether}}$): Prevents catastrophic forgetting by anchoring non-target logits to the base model.
3. Target Deadzone Loss ($\mathcal{L}_{\text{deadzone}}$): Allows target token logits to adjust freely within a tolerance margin $\tau_{\text{tgt}}$.

---

Kaggle Execution Guide (Dual T4 / P100 GPUs)

The full end-to-end pipeline is packaged into a zero-setup Jupyter notebook full_tranning_nootook_kaggle.ipynb and Python script kaggle_antidoom_notebook.py.

Running on Kaggle:

1. Upload this repository folder as a Kaggle Dataset (e.g., named antidoom-repo).
2. Open a new Kaggle Notebook:
   • Accelerator: GPU T4 ×2 (or P100)
   • Internet: Turned ON
3. Import full_tranning_nootook_kaggle.ipynb.

Fast Test Mode vs. Production Mode:

In Part 1 (Cell 2) of the notebook, set FAST_SAMPLE_MODE:

# Set True for a 1-minute dry-run test; Set False for full training
FAST_SAMPLE_MODE = False

if FAST_SAMPLE_MODE:
    print("⚡ FAST_SAMPLE_MODE Enabled — 1-minute test run")
    TARGET_PAIRS       = 10
    MAX_PROMPTS        = 50
    MAX_TRAIN_EXAMPLES = 10
    EVAL_PROMPTS       = 10
    MAX_NEW_TOKENS     = 500
else:
    print("🏋️ Full Production Mode Enabled")
    TARGET_PAIRS       = 3000
    MAX_PROMPTS        = 5000
    MAX_TRAIN_EXAMPLES = 3000
    EVAL_PROMPTS       = 300
    MAX_NEW_TOKENS     = 4000

---

Local Installation & CLI Usage

Prerequisites

• Python $\ge 3.10$
• PyTorch $\ge 2.3$
• CUDA-compatible GPU (16GB+ VRAM)

# Clone the repository
git clone https://github.com/Liquid4All/antidoom.git
cd antidoom

# Install dependencies via uv or pip
pip install -e .

Run Full Generation & Training CLI:

antidoom -c configs/default.yaml \
  --model-name LiquidAI/LFM2.5-1.2B-Base \
  --run-dir runs/experiment_1

---

Repository Structure

anti_doom/
├── full_tranning_nootook_kaggle.ipynb  # Complete self-contained Kaggle execution notebook
├── kaggle_antidoom_notebook.py         # Modular Python script version of the notebook
├── configs/
│   └── default.yaml                    # Hyperparameter and generation configuration
├── plots/                              # Generated publication-quality analytical figures
│   ├── com_graph.png                   # Pre vs Post FTPO loop rate comparison
│   ├── com_length.png                  # Completion length distribution
│   ├── compare.png                     # Benchmark comparison across runs
│   ├── imapact_das.png                 # FTPO impact dashboard (pie chart)
│   ├── loop_reduce.png                 # Relative reduction percentage bar chart
│   └── loop_word.png                   # Top doom-loop trigger tokens
├── src/antidoom/
│   ├── cli.py                          # Command line interface
│   ├── config.py                       # Dataclass configuration schemas
│   ├── ftpo_data.py                    # Dataset tokenization and regularization
│   ├── ftpo_train.py                   # LoRA training execution loop
│   ├── generate.py                     # vLLM generation and pair extraction
│   ├── reference_ftpo_trainer.py       # Custom FTPOTrainer & FTPO loss math
│   └── repetition.py                   # Fingerprint sliding-window loop detector
└── pyproject.toml                      # Project metadata and dependencies

---

References & Acknowledgments

• Liquid AI AntiDoom Blog: https://www.liquid.ai/blog/antidoom
• Antislop Paper: Auto-Antislop / Antislop (Paech et al., 2025)
• vLLM Engine: https://github.com/vllm-project/vllm
• Hugging Face TRL & PEFT: https://github.com/huggingface/trl

---

License

This repository is released under the Apache 2.0 License.