Summary: Today, llms are trained in a multi step process post SFT. RL -> Generate quality synthetic data → Self-Distillation on that → another round of RL (simplified).
Summary: Today, llms are trained in a multi step process post SFT. RL -> Generate quality synthetic data → Self-Distillation on that → another round of RL (simplified).