AI Researchers Build a System That Learns What It Doesn't Know to Make Better Training Data

The Problem with AI Training Data

Large language models keep getting better, but a big reason is that they now learn from synthetic experience — made-up scenarios created by computers rather than collected from the real world. For interactive AI agents, this means generating virtual situations where the agent can practice and improve.

But there is a catch. Current methods can create executable paths and verify them, yet they still struggle to turn a learner's mistakes into useful changes for future data. When an AI agent fails, it does not automatically know how to adjust its training data generation to avoid that same failure later. This gap between observing a weakness and deciding what to change about future data is called the "feedback-to-decision gap."

A New Multi-Agent Solution

Researchers led by Wei Yang have proposed EvoSyn, a framework that uses multiple specialized AI agents working together to solve this problem. The system is built around a "revisable state" — basically a clear list of what factors the system can deliberately control when generating new training experiences, and where it should focus its efforts.

EvoSyn uses three main roles. A Designer looks at the feedback and decides which direction the synthesis should take. A Planner takes those directions and turns them into actual executable experiences. And role-specialized agents translate the learner's feedback into concrete revisions of the data. This closed-loop process means the system learns from its own mistakes and adapts.

How It Evolves Over Time

What makes EvoSyn special is that it does two things at once. First, it adapts within the current space of possible experiences, filling in gaps where the agent is weak or under-exposed. Second, it expands the synthesis space itself — meaning it can promote newly discovered factors into explicit controls. In plain terms, if the system notices a type of situation it never thought to create before, it can add that type as a new knob it can deliberately turn.

The system was tested on τ^2-Bench and CAR-Bench, two standard benchmarks for evaluating interactive agents. The results showed that EvoSyn improved average task success by 13.3 points compared to the strongest existing synthesis baseline.

Why It Matters

The findings suggest that giving AI systems the ability to recognize what they do not know — and then actively evolving how they generate training data — is a meaningful step forward. Instead of randomly sampling new scenarios, the system now targets the specific gaps in its own understanding. As AI agents take on more complex, real-world tasks, the ability to self-direct their own training data could become essential.

The paper was published on arXiv in October 2026 by Wei Yang, Qing Ping, Youzhi Luo, Yawei Wang, Abhishek Kumar, Panpan Xu, and Jun Huan.

Key facts

• EvoSyn is a multi-agent framework that uses feedback from learners to revise how training data is generated
• It improves average task success by 13.3 points over the strongest synthesis baseline on τ²-Bench and CAR-Bench
• The system both adapts within existing generation spaces and expands those spaces to include new controllable factors
• Three main roles coordinate the process: Designer, Planner, and role-specialized agents

Sources

1. Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning
2. Know What You Don't Know: Agentic Data Synthesis by Evolving Generation ... — arxivsignals.io
3. Know What You Don't Know: Agentic Data Synthesis by Evolving Generation ... — academy.dair.ai
4. Know What You Don't Know: Agentic Data Synthesis by Evolving Generation ... — observatorioblockchain.com