Researchers Achieve One-Step Text Generation With Seq-Drifting
A new framework maps random noise directly to complete text in a single pass, skipping the slow multi-step process most AI models rely on.
A Faster Way to Generate Text
Most AI language models that use diffusion technology generate text by starting with random noise and gradually refining it through multiple steps — like slowly sharpening a blurry photo. While these models can produce many tokens at once, they still need several rounds of sampling before the text looks good.
Researchers including Jiayu Liu, Ruizhi Zhao, Tianyun Ji, Yee Whye Teh, Zhenya Huang, and Ning Miao have proposed a new method called Seq-Drifting that skips those extra steps entirely 1. Their approach generates an entire sequence of text in a single forward pass, taking random noise and turning it directly into coherent language.
How It Works
Seq-Drifting applies a concept called "drifting" to sequences of continuous token representations. In simple terms, the model learns a direct mapping from Gaussian noise — pure randomness — to actual text 1.
The key difference from earlier diffusion language models is how the system guides each token during training. Traditional models derive their training signal directly from the actual tokens in a training sequence. Seq-Drifting instead builds what the researchers call a "drift field," where the target for each position adapts based on the context already being generated 1.
At every position, two forces work together. An attraction force guides each output toward token representations that make sense in context. A repulsion force works in the opposite direction, preventing different samples from collapsing into identical outputs and stopping the model from repeating itself within a single sequence 1.
During training, the generator learns from this drift field. Then at inference time, it produces all tokens jointly in just one pass.
Strong Results in One Step
The researchers tested Seq-Drifting on the LM1B language modeling benchmark and achieved a generative perplexity score of 36.01 in a single step 1. Perplexity is a common measure of how well a language model predicts text — lower scores mean better performance.
The method performed competitively in both unconditional generation, where the model produces text on its own, and conditional generation, where it follows a given prompt or instruction 1.
Beyond open-ended text creation, the team also ran experiments on translation, summarization, and reasoning tasks, finding that the framework remains effective across these different applications 1. The results suggest that drifting over token sequences is a viable path toward efficient one-step text generation.
The researchers released their code on GitHub for others to build on 2.
Key facts
- Seq-Drifting generates full text sequences in a single forward pass, eliminating multi-step diffusion sampling
- It achieves a generative perplexity of 36.01 on the LM1B benchmark in one step
- The method uses a drift field with attraction and repulsion forces to guide token generation
- Seq-Drifting was tested on translation, summarization, and reasoning tasks beyond open-ended generation
- The code is publicly available on GitHub
Sources
- One-Step Text Generation by Seq-Drifting
- One-Step Text Generation by Seq-Drifting | Arxiv - DeepPaper — arxiv.deeppaper.ai
- 基于序列漂移的单步文本生成｜arXiv 2610.12879｜arXivDaily — arxivdaily.com
- Artificial Intelligence - arXiv.org
- One-Step Generation via Riemannian Wasserstein Gradient Flows