Charcot

Charcot 27B

Qwen3.8-27B fine-tuned on synthetic Charcot Gym episodes for healthcare admin work: claims, clinical coding, and document → FHIR R5. On private tasks it has never seen, the mean score rises from 0.505 to 0.727.

Field Value
Base model Qwen/Qwen3.8-27B
Training data charcot-ai/gym-sft
Benchmark charcot-ai/gym · Charcot-AI/gym
Input / output Text in, text and tool calls out (vision tower unchanged)
License Apache-2.0

Quickstart

Serve the model with an OpenAI-compatible API:

vllm serve charcot-ai/charcot-27b --dtype bfloat16 --max-model-len 131072

Send a request:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="charcot-ai/charcot-27b",
    messages=[{"role": "user", "content": "Which ICD-10-CM code fits type 2 diabetes without complications?"}],
)
print(response.choices[0].message.content)

Evaluate it on the public Charcot Gym split:

pip install "git+https://github.com/Charcot-AI/gym.git@v1.1.0"
export OPENAI_API_KEY=EMPTY
charcot-gym eval \
  --model charcot-ai/charcot-27b \
  --base-url http://localhost:8000/v1 \
  --api-key-env OPENAI_API_KEY \
  --split public --text-only --attempts 3 \
  --out results/charcot-27b

The run writes summary.json with overall and per-family scores.

Results

Mean score from 0 to 1, 3 attempts per task.

Evaluation Tasks Qwen3.8-27B Charcot 27B
Private split 162 0.51 0.73
Public held-out 144 0.37 0.54

Private split by family:

Family Qwen3.8-27B Charcot 27B Change
Claims 0.37 0.71 ↑
Clinical coding 0.42 0.69 ↑
Document → FHIR 0.73 0.78 ≈

↑ gain, ↓ drop, ≈ no significant difference (95% confidence interval of the difference).

Training

Field Value
Data 801 synthetic text episodes: 153 public-split tasks and 648 generated with the public generators (8 published seeds)
Method LoRA SFT (rank 32) on the response only, merged into FP32 weights
Private data None: no private task or seed was used in training

Limitations

  • Lower scores on Hard clinical coding: Δ −0.149 [−0.25, −0.05] on the public held-out and −0.174 [−0.34, −0.01] on the private split.
  • Text only: the vision tower is unchanged from the base model.
  • Trained and evaluated on synthetic data only. Not for clinical use.
  • Private tasks are not published; only aggregate scores are.
Evaluation details
  • Held-out tasks use fresh seeds and share no seed, content, or initial state with training or private tasks.
  • Public held-out intervals: paired 95% bootstrap over tasks.
  • Private split intervals: 95% bootstrap over tasks within each cell, combined across cells with task-count weights. It is unpaired, so it is conservative.
  • Errored attempts were retried up to twice and excluded if they still failed (base: 6 of 486 private attempts).

Citation

@misc{charcot2026charcot27b,
  title  = {Charcot 27B},
  author = {{Charcot AI}},
  year   = {2026},
  url    = {https://huggingface.co/charcot-ai/charcot-27b}
}

License

Apache-2.0. The base model remains subject to its own license.

Downloads last month
146
Safetensors
Model size
28B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for charcot-ai/charcot-27b

Base model

Qwen/Qwen3.8-27B
Finetuned
(959)
this model
Quantizations
1 model

Datasets used to train charcot-ai/charcot-27b