Instructions to use charcot-ai/charcot-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use charcot-ai/charcot-27b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="charcot-ai/charcot-27b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("charcot-ai/charcot-27b") model = AutoModelForMultimodalLM.from_pretrained("charcot-ai/charcot-27b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use charcot-ai/charcot-27b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "charcot-ai/charcot-27b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/charcot-27b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/charcot-ai/charcot-27b
- SGLang
How to use charcot-ai/charcot-27b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "charcot-ai/charcot-27b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/charcot-27b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "charcot-ai/charcot-27b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/charcot-27b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use charcot-ai/charcot-27b with Docker Model Runner:
docker model run hf.co/charcot-ai/charcot-27b
Charcot 27B
Qwen3.8-27B fine-tuned on synthetic Charcot Gym episodes for healthcare admin work: claims, clinical coding, and document → FHIR R5. On private tasks it has never seen, the mean score rises from 0.505 to 0.727.
| Field | Value |
|---|---|
| Base model | Qwen/Qwen3.8-27B |
| Training data | charcot-ai/gym-sft |
| Benchmark | charcot-ai/gym · Charcot-AI/gym |
| Input / output | Text in, text and tool calls out (vision tower unchanged) |
| License | Apache-2.0 |
Quickstart
Serve the model with an OpenAI-compatible API:
vllm serve charcot-ai/charcot-27b --dtype bfloat16 --max-model-len 131072
Send a request:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="charcot-ai/charcot-27b",
messages=[{"role": "user", "content": "Which ICD-10-CM code fits type 2 diabetes without complications?"}],
)
print(response.choices[0].message.content)
Evaluate it on the public Charcot Gym split:
pip install "git+https://github.com/Charcot-AI/gym.git@v1.1.0"
export OPENAI_API_KEY=EMPTY
charcot-gym eval \
--model charcot-ai/charcot-27b \
--base-url http://localhost:8000/v1 \
--api-key-env OPENAI_API_KEY \
--split public --text-only --attempts 3 \
--out results/charcot-27b
The run writes summary.json with overall and per-family scores.
Results
Mean score from 0 to 1, 3 attempts per task.
| Evaluation | Tasks | Qwen3.8-27B | Charcot 27B |
|---|---|---|---|
| Private split | 162 | 0.51 | 0.73 |
| Public held-out | 144 | 0.37 | 0.54 |
Private split by family:
| Family | Qwen3.8-27B | Charcot 27B | Change |
|---|---|---|---|
| Claims | 0.37 | 0.71 | ↑ |
| Clinical coding | 0.42 | 0.69 | ↑ |
| Document → FHIR | 0.73 | 0.78 | ≈ |
↑ gain, ↓ drop, ≈ no significant difference (95% confidence interval of the difference).
Training
| Field | Value |
|---|---|
| Data | 801 synthetic text episodes: 153 public-split tasks and 648 generated with the public generators (8 published seeds) |
| Method | LoRA SFT (rank 32) on the response only, merged into FP32 weights |
| Private data | None: no private task or seed was used in training |
Limitations
- Lower scores on Hard clinical coding: Δ −0.149 [−0.25, −0.05] on the public held-out and −0.174 [−0.34, −0.01] on the private split.
- Text only: the vision tower is unchanged from the base model.
- Trained and evaluated on synthetic data only. Not for clinical use.
- Private tasks are not published; only aggregate scores are.
Evaluation details
- Held-out tasks use fresh seeds and share no seed, content, or initial state with training or private tasks.
- Public held-out intervals: paired 95% bootstrap over tasks.
- Private split intervals: 95% bootstrap over tasks within each cell, combined across cells with task-count weights. It is unpaired, so it is conservative.
- Errored attempts were retried up to twice and excluded if they still failed (base: 6 of 486 private attempts).
Citation
@misc{charcot2026charcot27b,
title = {Charcot 27B},
author = {{Charcot AI}},
year = {2026},
url = {https://huggingface.co/charcot-ai/charcot-27b}
}
License
Apache-2.0. The base model remains subject to its own license.
- Downloads last month
- 146