Overtaught, Underlearned: Learner-Centered Distillation from Multi-Agent Discussion

Multi-agent discussion produces reasoning collaboratively, while deployment still relies on a smaller student that must answer alone. Full-graph distillation transfers such discussions by treating each correct solution as a separate training target, but what teachers provide need not match what the