Instructions to use charcot-ai/crs-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use charcot-ai/crs-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="charcot-ai/crs-4b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("charcot-ai/crs-4b") model = AutoModelForMultimodalLM.from_pretrained("charcot-ai/crs-4b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use charcot-ai/crs-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "charcot-ai/crs-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/crs-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/charcot-ai/crs-4b
- SGLang
How to use charcot-ai/crs-4b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "charcot-ai/crs-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/crs-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "charcot-ai/crs-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "charcot-ai/crs-4b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use charcot-ai/crs-4b with Docker Model Runner:
docker model run hf.co/charcot-ai/crs-4b
CRS-4B
CRS-4B is a 4B clinical tool-use model. It is fine-tuned from Qwen3.5-4B on public synthetic episodes from Charcot Gym, an EHR environment where agents read clinical context, call tools, and write structured FHIR records.
| Property | Value |
|---|---|
| Base model | Qwen/Qwen3.5-4B |
| Parameters | 4.66B (BF16) |
| Training data | Public synthetic tasks from charcot-ai/gym |
| Evaluation | 99 held-out private Charcot Gym tasks, 3 attempts per task |
| Task families | Document → FHIR, Clinical coding, Claims |
| Intended use | Research and evaluation of clinical tool use |
Quickstart
Install a recent transformers (the weights were saved with 5.17), then ask the model to act through a tool:
pip install "transformers>=5.17" torch accelerate
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "charcot-ai/crs-4b"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")
tools = [{
"type": "function",
"function": {
"name": "fhir_create",
"description": "Create a FHIR R5 resource in the patient's chart.",
"parameters": {
"type": "object",
"properties": {"resource": {"type": "object", "description": "FHIR R5 resource JSON"}},
"required": ["resource"],
},
},
}]
messages = [{
"role": "user",
"content": "Patient/avery-1. New lab report, 2025-07-06: ALT 49 U/L. Save this result to the chart.",
}]
inputs = processor.apply_chat_template(
messages, tools=tools, add_generation_prompt=True,
tokenize=True, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Outputs
The chat template opens a <think> block, so the decoded text is the model's reasoning, a closing </think>, and then zero or more tool calls in the Qwen3.5 format:
<tool_call>
<function=fhir_create>
<parameter=resource>
{"resourceType": "...", ...}
</parameter>
</function>
</tool_call>
| Part | What to do with it |
|---|---|
Text before </think> |
Reasoning; drop it or log it |
<function=NAME> |
Name of the tool to call |
<parameter=KEY> |
One argument; the value is JSON when the schema type is an object |
Run the tool, append the result as a {"role": "tool", "content": ...} message, and generate again until the model answers without a tool call.
To score the model on real tasks, serve it behind an OpenAI-compatible endpoint and run charcot-gym eval as described in the Charcot Gym README.
Results
| Model | Mean task score |
|---|---|
| Qwen3.5-4B (base) | 0.16 |
| CRS-4B (fine-tuned) | 0.86 |
| Frontier models on the same tasks | 0.76–0.89 |
99 private tasks, 3 attempts per task, November 2026. The private tasks were never seen in training. A deterministic offline verifier checks the final record and task constraints; no LLM judge is used. These scores describe performance on the evaluated tasks; they do not establish a general ranking outside this evaluation.
Training and evaluation
| Stage | Detail |
|---|---|
| Training | Supervised fine-tuning on public synthetic Gym episodes only |
| Evaluation | Held-out private split, repeated attempts per task |
| Integrity checks | Patient identity and preservation of existing resources |
| Private reporting | Aggregate scores only; private task contents, per-task outputs, and trajectories are not returned to submitters |
See the evaluation methodology for scoring and the submission policy.
Files
| Path | Contents |
|---|---|
model.safetensors-* |
Merged weights; load with the Quickstart above |
adapter/ |
The LoRA adapter (rank 32) for Qwen/Qwen3.5-4B |
chat_template.jinja |
Chat template with tool-call formatting |
Limitations
- The reported result measures synthetic task completion, not performance in patient care. Do not use the model for clinical decisions.
- Training and evaluation data are synthetic and in English.
- Scores cover the three task families above; other Charcot Gym families and other clinical tasks were not reported here.
Citation
@misc{charcot2026crs4b,
title = {CRS-4B: A clinical tool-use model trained in a synthetic EHR},
author = {{Charcot AI}},
year = {2026},
howpublished = {\url{https://huggingface.co/charcot-ai/crs-4b}}
}
Longitudinal data and evaluation infrastructure for healthcare AI agents. charcot.ai · Charcot Gym · Public tasks
- Downloads last month
- 77

