Echoverse: a new way to train computer-use agents inside repairable practice apps
A new paper presents Echoverse, a system that builds stateful practice applications and repairs them alongside the model training on them.
What the paper argues
Computer-use agents learn from what their actions change, the paper says, so training one needs applications it can act on, break, and reset. The applications that matter most for this kind of training are login-gated and stateful, so researchers have turned to synthetic environments that stand in for them. Recent pipelines can generate large numbers of these environments, the paper says, which shifts the real bottleneck from how many environments exist to what is actually inside each one.
The paper, whose author list includes Yash Pandya and 12 other authors, reports that the payoff comes from three properties: how much behavioral depth an environment carries, whether it targets the interaction an agent actually fails at, and whether the environment improves alongside the model being trained on it.
How Echoverse works
The paper presents Echoverse, a system that compiles written specifications into stateful applications whose tasks are graded against the application's own database rather than against a fixed answer key. Echoverse also runs what the researchers call a co-evolution loop: every graded rollout is read twice, once as a signal for repairing the environment, its tasks, and its verifier, and once as training signal for the model itself.
What the experiments measured
Training on twelve such environments, the researchers report that a 9B model improved from thirty-six point five percent to sixty-seven point one percent accuracy across fourteen evaluation splits, landing within fourteen points of the much larger frontier model that had taught it.
The paper also isolates each of its three properties. On matched domains, the researchers say shallow environments pushed live-site accuracy below the untrained base model's own score, from eighty percent down to seventy-five percent, while deeper environments raised it, from eighty percent up to eighty-five percent in one case and from forty-eight percent up to sixty-five percent in another. Drilling a single interface control across many different renderings, the paper reports, transferred to held-out families of on-screen widgets and to the open web. Repairing just one environment, the researchers say, lifted the accuracy of the model trained on it from sixteen point two percent to thirty-eight point five percent.
The same synthetic worlds also work as reinforcement-learning environments, according to the paper: combining the system's grounded verifier with a dense, per-step judge as the reward signal raised a held-out score from fifty-eight point eight percent to sixty-eight percent.
What the researchers released
The team says it is releasing four of its environments as a public benchmark, packaged with their applications, seed data, and grounded graders, alongside code linked from the paper.
Key facts
- The paper presents Echoverse, a system that compiles specifications into stateful, database-graded practice applications for training computer-use agents.
- A 9B model trained on twelve Echoverse environments improved from thirty-six point five percent to sixty-seven point one percent across fourteen evaluation splits, per the paper.
- Shallow environments pushed live-site accuracy down, from eighty percent to seventy-five percent, while deep environments raised it, from eighty percent to eighty-five percent and from forty-eight percent to sixty-five percent, the researchers report.
- Repairing a single environment lifted that environment's trained-model accuracy from sixteen point two percent to thirty-eight point five percent, according to the paper.
- The researchers say they are releasing four environments as a benchmark, with applications, seed data, and grounded graders.
Check your understanding
What accuracy range does the paper report for its 9B model across fourteen evaluation splits after training on twelve Echoverse environments? According to the paper, what happened to live-site accuracy when researchers used shallow environments instead of deep ones?