Training environments for clinical AI

We build synthetic clinical tasks and private evaluations for teams training AI agents.

Request a private eval

View leaderboard →

The benchmark

234 synthetic tasks across claims, clinical coding and document-to-FHIR conversion. Each task is scored by deterministic checks.

Compare 12 models →

charcot-gym · episodeILLUSTRATIVE

SYNTHETIC TASK

Code today’s visit and link the diagnosis to the encounter.

AGENT ACTIONS

↳fhir.search(Encounter)

↳inbox.open(lab_report.pdf)

↳fhir.create(Condition)

↳portal.check_coverage(...)

DETERMINISTIC VERIFIER

✓ICD-10-CM code

✓encounter reference

✓onset date

×unit

SCORE

0.80

Environments

Training environments

RL · TOOL USE

Tool-use tasks for RL, with deterministic graders and quarterly refreshes.

Private evaluations

HELD-OUT · AGGREGATE

Your checkpoints, our held-out tasks. Aggregate scores by workflow.

Custom environments

YOUR WORKFLOW

New clinical workflows built around the capabilities you want to train.

Physician review

RUBRICS · CALIBRATION

Physicians review outputs and calibrate task-specific rubrics.

How scoring works

Document

HbA1c · 9.8%

Agent actions

fhir.create(Observation)

Final record

LOINC 4548-4 · UCUM %

Deterministic verifier

code · value · unit · date

Scored on the final record state. Offline. No LLM judge.

Leaderboard

Standard tier · private split · v1.0 · October 2026

claude-opus-5.5

95% CI 0.926–1.000

0.972

claude-sonnet-5.5

95% CI 0.896–0.977

0.940

gpt-6.1-sol

95% CI 0.880–0.962

0.925

gemini-4-argon

95% CI 0.858–0.967

0.916

grok-4.7

95% CI 0.829–0.932

0.884

How we score

Submit a model

Contact

Tell us about the model or clinical workflow you’re working on.

solano@charcot.ai