Works with our synthetic tasks, your data or your data partner’s data.
The benchmark
234 public tasks · 99 private tasks
Claims · Clinical coding · Document → FHIR
Standard & Hard · Text & scans · Deterministic scoring
Compare 12 models →
Charcot is the verification layer for clinical AI: deterministic graders, RL environments, private evaluations and physician-calibrated rubrics.
Training environments
RL · TOOL USE
RL and tool-use tasks with deterministic graders and quarterly refreshes.
Private evaluations
HELD-OUT · AGGREGATE
Your checkpoints on our held-out tasks, with aggregate scores by workflow.
Custom environments and graders
YOUR WORKFLOW
New clinical workflows, on synthetic data or on data from your data partner.
Synthetic patient cohorts
SYNTHETIC · AGGREGATE STATISTICS
Synthetic patients and records whose distributions match aggregate clinical statistics, for training and evaluation. Every record is synthetic; only aggregate statistics shape the distributions.
Physician review
RUBRICS · CALIBRATION
Rubric calibration and output review.
01
Document
HbA1c · 9.8%
02
Agent actions
fhir.create(Observation)
03
Final record
LOINC 4548-4 · UCUM %
04
Deterministic verifier
code · value · unit · date
Scored on the final record state. Offline. No LLM judge.
The same score is the benchmark number and the RL reward. Public text tasks include reference solutions (oracle_turns).
Coming in v2
01
Source difficulty levels
CSV, HL7 v2, broken-table PDFs, superseded reports, non-standard units, ambiguous dates and same-name patients.
02
Difficulty bands
Difficulty bands for every task.
03
Clinical scores
Calculators implemented from the primary publications, including the correct refusal when inputs are missing.
04
Web EHR computer use
A synthetic web EHR with screenshots, coordinates, an accessibility tree and human approval steps, scored on the final FHIR state.
05
A larger public task set
Published as a versioned dataset.
For data partners
Data providers and brokers can add Charcot graders, environments and physician-calibrated rubrics to their datasets.
Tell us about the model or clinical workflow you’re working on.