DeepSeek V4's post-training recipe

What's this about?

A researcher's close reading of DeepSeek V4's technical report, focused on post-training. The striking finding: no explicit preference tuning at all — no RLHF, no DPO. The recipe leans on RLVR (RL with verifiable rewards) plus multi-teacher on-policy distillation.

What exactly is RLHF vs RLVR?

RLHF (RL from human feedback) teaches a model what humans *prefer* — raters compare outputs, the model chases approval; it made ChatGPT pleasant but can teach flattery over correctness. RLVR replaces the human judge with automatic verification — the reward comes from being *checkably right*. One of the strongest open models skipping preference tuning entirely signals where frontier training is heading.

What's the community saying?

1. RLVR-only as the emerging consensus for reasoning models — matches today's Ring-Zero paper.
2. Interest in multi-teacher on-policy distillation: several strong models supervising one student, on the student's own outputs.