Evolvent AIEvolvent AIRSI–Index
AN ENVIRONMENT FOR AUTONOMOUS RESEARCH

RSIGym.

An environment for AI
to improve AI.

Reusable research services. Configurable budgets.
Measure the progress with RSI-Index.

THE RESEARCH LOOP
Train → Evaluate → ReviseAGENT-NATIVE RESEARCH
6research agents
5benchmark domains
$500per benchmark run
Joint trackData + training + harness
THE ENVIRONMENT

Research services, ready for the agent.

Training, inference and evaluation are mature services, yet agents often rebuild them. RSIGym makes them directly callable, leaving the agent’s budget and attention for improving the target system.

RESEARCH AGENTLightweight, CPU-only workspace
Use the evidence to choose the next experiment
SUBMIT THE SELECTED SYSTEMCheckpoint + harness

Fixed evaluation protocol

Official score
FIVE REUSABLE SERVICESGenerate examples with frontier models and test them in isolated sandboxes.
TTrainingData → checkpoint
IInferenceCall base or trained models
RRolloutGenerate data and judge
EEvaluationScores and execution logs
SSandboxIsolated computation
Shared authorizationOne key per runPermission checksBudget accounting

Three ways to study improvement.

Permissions define what the researcher can change.

01 / DATA

Change the examples.

The agent designs training data. Training settings and the execution harness stay fixed.

DataTraining settingsHarness
02 / HARNESS

Change how the model acts.

The agent revises execution code, tool use and control flow with the target’s weights fixed.

DataTraining settingsHarness
03 / JOINT · MAIN STUDY

Improve them together.

The agent can revise data, training settings and the harness within the same research run.

DataTraining settingsHarness
THE RESULTS

RSI-Index leaderboard

Six research agents improving Qwen3.5-35B-A3B-Base with a minimal harness.

Updated
Swipe to compare all benchmarks →
Joint-track results. RSI-Index uses the paper’s native scale; benchmark scores are percentages. Select a column to sort.
Bold: best reported result · Benchmark scores in %

6 complete suites · 30 benchmark runs

Platform services: $500 per run. Final scores: mean reward over three attempts per task.

STRONGEST OVERALL48.09%

Opus 5 closes the largest average share of the remaining performance gap.

BROAD IMPROVEMENT29/30

Joint runs improve on the starting system. All six agents have a positive RSI-Index.

NO UNIVERSAL LEADER4

Researchers lead at least one benchmark. Sol 6 ranks last overall but leads SkillsBench.

The remaining challenge: Opus 5 closes 96.75% of the AIME gap. No agent closes a quarter of the gap on Terminal-Bench or SkillsBench.

↗ Explore a researcher Opus 5⌄

Opus 5

Claude Code research agent

0.4809RSI-Index

Complete five-domain suite

Share of available improvement achieved0% to 100%

Final gain ÷ remaining gap to 100. Calculated from the displayed benchmark scores.

How RSI-Index is calculated
ONE NUMBER, FIVE DOMAINS

Improvement relative
to what was possible.

For each benchmark, divide the score gain by the gap from the initial score to a perfect score. Then average the five fractions with equal weight.

RSI-Index = mean((final − initial) / (1 − initial))
0 Initial system1 Perfect system< 0 Regression
EXPLORE A RESULT
Initial score
Final score
0%100%

of the available improvement achieved

Illustration uses displayed scores; the index uses full-precision results.

Evaluation protocol & benchmark subsets

Five evaluation sets

  • SWE-bench Verified — 100 tasks
  • Terminal-Bench 2.0 — 89 tasks
  • AIME 2024/2025 — 60 tasks
  • GPQA Diamond — 100 tasks
  • SkillsBench — 37 tasks, science / office / finance

One research run per configuration

Final scores average three attempts per task (avg@3). These repeat task execution, not the research process. Final evaluations reuse tasks available during development.

The $500 allocation covers platform services; research-agent inference and final official evaluation are billed separately. Post-run audits check training-data overlap, checkpoint provenance and research behavior.

The experiments measure improvement within a budgeted research run. Gains across subsequent research cycles remain to be evaluated.

TWO FURTHER EXPERIMENTS

What else can this environment study?

Start from an existing harness, or let the target model conduct its own research. The same services support both settings.

01 / ITERATING AN EXISTING HARNESS

Better execution.
The same model weights.

Opus 5 iterates DSH with Qwen3.6-35B-A3B-Instruct fixed. It repairs truncated-output handling, preserves background services and verifies work before finishing.

30.34%→40.82%

Tasks solved on all three attempts increase from 15 to 26.

More tasks solved consistentlyTerminal-Bench · 89 tasks · 3 attempts each
Bands connect outcome categories, not individual task transitions. Scores are official avg@3.
02 / THE MODEL AS ITS OWN RESEARCHER

Qwen3.8-27B completes the research loop but does not improve itself.

It trains candidates and selects a 472-example checkpoint from a six-task probe. That local improvement does not carry through to the full evaluation.

56.66%→56.26%

On all 37 tasks, the official score falls by 0.40 percentage points.

Candidate selection → full evaluationSkillsBench · Qwen3.8-27B as researcher and target
The left panel uses 6 tasks; the right uses all 37. Full-set development uses 2 attempts per task; official evaluation uses 3. “Reduced reasoning” uses the 472-example checkpoint.
CONTINUE EXPLORING

The research behind the results.

Read the experiment design and analysis, or explore the RSIGym implementation.

DATA PROVENANCE

Source & methodology

The leaderboard reports all six research agents from Figure 2 of the RSIGym paper. Scores are a snapshot of the paper’s reported experiments.

Revision
4 October 2026 · fd8f3be
Source
Section 4 · Figure 2 · Joint-track results
Display scale
RSI-Index: original paper value
Benchmark scores: percentages

RSI-Index averages (final − baseline) / (1 − baseline) across five benchmarks. Benchmark columns show raw final scores. The aggregate uses the paper’s published precision.

Each research agent has five independent benchmark runs. All six suites are complete. The shared starting system is Qwen3.5-35B-A3B-Base with a minimal harness.

View reported results ↗View experiment protocol ↗RSIGym implementation ↗Paper snapshot: 4 Oct 2026. Repository access may be required.
PROFESSIONAL EVALUATIONS

Get evaluated

Tell us what you’d like to test. We’ll confirm the scope, pricing and deliverables before starting a paid evaluation.

We’ll use these details to respond to your inquiry. Please don’t include API keys, credentials or confidential model access details.

Prefer email? [email protected] ↗