AI models that learn by teaching themselves at deeper thinking levels
A new method lets language models use their own deeper reasoning loops as teachers, creating a self-improving training cycle.
Teaching an AI to be its own tutor
Language models spend a lot of time learning from data — reading text, answering questions, solving problems. But there is a growing idea that the best teacher for a model may be the model itself.
Researchers have proposed a method called LoopOPD (Looped On-Policy Distillation) that lets a language model use its own deeper reasoning steps as a source of training signal 1. The approach is built around a type of model known as a Looped Language Model, or LoopLM. These models reuse the same set of parameters across multiple computation steps, like running the same logic over and over while building a longer chain of thought. Think of it as letting a student think through a problem step by step rather than jumping straight to an answer.
The key challenge has been how to train these models effectively. Previous methods either relied on sparse reward signals that are hard to scale across loops, or required an external teacher model — which is expensive and can create a mismatch between what the teacher knows and what the student sees 1.
How the self-teaching works
LoopOPD flips the problem on its head. Instead of bringing in an outside teacher, the method uses a frozen copy of the model itself — one that has run additional computation loops — as the teacher for a student version still in the middle of its loops 1. The student generates its own responses, called rollouts, and then learns by matching the token-level predictions of the deeper, finished version of itself.
This creates dense, step-by-step supervision without needing any external data or model 1. The teacher has access to information the student does not yet have — like the final steps of reasoning — and that advantage is exactly what the student learns from.
Making the teacher evolve
The next step was to make the system truly self-improving. In the basic version, the teacher stays frozen. But the researchers introduced Dynamic LoopOPD (D-LoopOPD), which periodically refreshes the teacher as the shared model parameters are updated 1. This means the student is always being guided by the latest and best version of itself, creating a continuous improvement loop.
The team showed mathematically that a single training update can produce simultaneous local improvement at both the shallow and deep loop levels under certain conditions 1. They tested the approach on Ouro-Thinking models and found that LoopOPD improved mathematical reasoning, while D-LoopOPD delivered further gains through its dynamic teacher updates . Interestingly, even though the models were trained only on math data, they also improved on general reasoning and code generation tasks, suggesting the method teaches transferable skills .
The work was submitted to arXiv in October 2026 and is under review at ICLR 2027 1.
Key facts
- LoopOPD lets a model use its own deeper reasoning loops as its teacher
- Dynamic LoopOPD refreshes the teacher as parameters update, enabling continuous self-improvement
- Models trained only on math data also improved at general reasoning and code generation
- The method was tested on Ouro-Thinking models and submitted to arXiv in October 2026
Sources
- Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models
- Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation ... — papers.cool
- 2609.30652 Recursive Self-Improvement via On-Policy Distillation for ...
- Fugu-MT 論文翻訳 (概要): Recurrent Self-Improvement: Dynamic Cross-Loop On ... — fugumt.com
- Awesome Loop Models - GitHub