DPO vs SimPO: What Your Preference Trainer Is Actually Optimizing

SalesConversion-Bench had one uncomfortable preference-tuning mismatch: the code trained with TRL DPOTrainer, while the methodology narrative argued for SimPO. That is not just a naming issue. DPO and SimPO turn the same (prompt, chosen, rejected) pair into different update signals. If the held-out