Change the examples.
The agent designs training data. Training settings and the execution harness stay fixed.
An environment for AI
to improve AI.
Reusable research services. Configurable budgets.
Measure the progress with RSI-Index.
Training, inference and evaluation are mature services, yet agents often rebuild them. RSIGym makes them directly callable, leaving the agent’s budget and attention for improving the target system.
Fixed evaluation protocol
Official scorePermissions define what the researcher can change.
The agent designs training data. Training settings and the execution harness stay fixed.
The agent revises execution code, tool use and control flow with the target’s weights fixed.
The agent can revise data, training settings and the harness within the same research run.
Six research agents improving Qwen3.5-35B-A3B-Base with a minimal harness.
Platform services: $500 per run. Final scores: mean reward over three attempts per task.
Opus 5 closes the largest average share of the remaining performance gap.
Joint runs improve on the starting system. All six agents have a positive RSI-Index.
Researchers lead at least one benchmark. Sol 6 ranks last overall but leads SkillsBench.
The remaining challenge: Opus 5 closes 96.75% of the AIME gap. No agent closes a quarter of the gap on Terminal-Bench or SkillsBench.
Claude Code research agent
0.4809RSI-IndexComplete five-domain suite
Final gain ÷ remaining gap to 100. Calculated from the displayed benchmark scores.
For each benchmark, divide the score gain by the gap from the initial score to a perfect score. Then average the five fractions with equal weight.
of the available improvement achieved
Illustration uses displayed scores; the index uses full-precision results.
Final scores average three attempts per task (avg@3). These repeat task execution, not the research process. Final evaluations reuse tasks available during development.
The $500 allocation covers platform services; research-agent inference and final official evaluation are billed separately. Post-run audits check training-data overlap, checkpoint provenance and research behavior.
The experiments measure improvement within a budgeted research run. Gains across subsequent research cycles remain to be evaluated.
Start from an existing harness, or let the target model conduct its own research. The same services support both settings.
Opus 5 iterates DSH with Qwen3.6-35B-A3B-Instruct fixed. It repairs truncated-output handling, preserves background services and verifies work before finishing.
Tasks solved on all three attempts increase from 15 to 26.
It trains candidates and selects a 472-example checkpoint from a six-task probe. That local improvement does not carry through to the full evaluation.
On all 37 tasks, the official score falls by 0.40 percentage points.
Read the experiment design and analysis, or explore the RSIGym implementation.