How Google's RRSI lets AI agents improve themselves without going off the rails

What is RRSI?

Google Research has released a guide and code for a method called Regularized Recursive Self-Improvement, or RRSI. It teaches AI agents to rewrite parts of their own system — their prompts, tools, memory, and control flow — while keeping the underlying language model frozen. The goal is to let agents evolve without overfitting to the specific tasks they are training on.

The research team also includes members from UNC-Chapel Hill, Stanford University, and Washington University in St. Louis. Their paper and project page are linked from the GitHub repository.

The self-improvement loop

In the full RRSI setup, a draft of each proposed edit is created by calling Claude on Vertex AI, and the quality of each edit is scored inside Docker containers running benchmark tasks ¹. However, the part that actually decides which edits survive is pure Python — no API calls needed.

The system works in rounds. Each round proposes several candidate edits, scores them on real task benchmarks, and then applies a set of rules to pick the best ones. These rules are the core of RRSI's contribution: they keep improvement steady without letting the agent drift into overfitted or unstable behavior.

How RRSI stays stable

Several mechanisms work together to prevent chaos:

• Noise band: Before comparing scores, the system adds calibrated random noise so small differences don't cause wild swings in decisions ¹.
• Cost rule: Each edit has a cost measured in tokens. Cheaper edits get preferred when performance gains are similar.
• Annealed edit budget: Early in training the agent can make many edits; later it gets fewer, so it stops tinkering once it finds a good setup.
• Leakage screen: A critic module checks whether a proposed edit might accidentally expose test answers to the agent, blocking any suspicious changes before they are applied.
• History tracking: RRSI records every edit, detects when progress stalls, and can boost exploration if the agent gets stuck ¹.
• Novelty check: The system measures whether a new edit is structurally different from past ones, rewarding genuine variation.

Why this matters

Most self-improving AI research focuses on changing the model itself — retraining or fine-tuning it. RRSI takes a different path: the model stays frozen, and the agent improves its surrounding "harness" instead. That means faster iteration, lower cost, and a safer way to study autonomous self-modification without risking model degradation.

The full RRSI loop requires Claude and Docker benchmarks, but anyone can run the decision logic locally to understand how the algorithm picks which edits to keep ¹.

Quick facts

• RRSI keeps the language model frozen and evolves only the agent's harness.
• Edits are drafted with Claude and scored in Docker benchmark environments.
• The decision logic is plain Python and can run without API keys or GPUs.
• A leakage screen blocks edits that might expose test data.
• An annealed budget gradually reduces how many edits the agent can make.

Key facts

• RRSI keeps the base language model frozen and evolves only the agent's surrounding harness.
• Proposed edits are drafted with Claude on Vertex AI and scored in Docker benchmarks.
• The core edit-selection logic runs in plain Python without API keys or GPUs.
• A leakage screen blocks edits that might accidentally reveal test answers.
• An annealed budget lets the agent make many edits early and fewer later.

Sources

1. Google Research RRSI Guide: Mastering Self-Improving AI Agents — marktechpost.com
2. GitHub - google-research/rrsi
3. PDF RRSI: Regularized Recursive Self-Improvement of Agent Harnesses — regularized-rsi.com