Instructions to use EPFLiGHT/Apertus-70B-MeditronFO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EPFLiGHT/Apertus-70B-MeditronFO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="EPFLiGHT/Apertus-70B-MeditronFO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("EPFLiGHT/Apertus-70B-MeditronFO")
model = AutoModelForCausalLM.from_pretrained("EPFLiGHT/Apertus-70B-MeditronFO")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use EPFLiGHT/Apertus-70B-MeditronFO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "EPFLiGHT/Apertus-70B-MeditronFO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EPFLiGHT/Apertus-70B-MeditronFO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/EPFLiGHT/Apertus-70B-MeditronFO

SGLang

How to use EPFLiGHT/Apertus-70B-MeditronFO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "EPFLiGHT/Apertus-70B-MeditronFO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EPFLiGHT/Apertus-70B-MeditronFO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "EPFLiGHT/Apertus-70B-MeditronFO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EPFLiGHT/Apertus-70B-MeditronFO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use EPFLiGHT/Apertus-70B-MeditronFO with Docker Model Runner:
```
docker model run hf.co/EPFLiGHT/Apertus-70B-MeditronFO
```

Apertus-70B-MeditronFO

Apertus-70B-MeditronFO is a 70B-parameter medical specialist LLM, produced by supervised fine-tuning of Apertus-70B-Instruct on the Fully Open Meditron Corpus.

This model is part of the Fully Open Meditron family — the first end-to-end auditable pipeline for clinical LLMs, with open weights, open data, open training recipe, and clinician-vetted corpus construction.

Apertus-70B-MeditronFO establishes a new state of the art among fully open medical LLMs, and is preferred over Llama-3.1-70B-Meditron in 96.6% of pairwise Auto-MOOVE comparisons.

📄 Paper: Fully Open Meditron: An Auditable Pipeline for Clinical LLMs
💻 Code: github.com/EPFLiGHT/FullyOpenMeditron
📚 Collection: MeditronFO
🗂️ Training corpus: EPFLiGHT/fully-open-meditron

Performance

Accuracy (%) on standard medical benchmarks. See the paper for full evaluation details, confidence intervals, and open-ended Auto-MOOVE results.

Benchmark	Apertus-70B-Instruct	Apertus-70B-MeditronFO	Δ
MedMCQA	52.43	56.32	+3.89
MedQA	60.64	68.58	+7.94
PubMedQA	66.80	75.20	+8.40
MedXpertQA	12.33	16.90	+4.57
HealthBench Hard	32.28	40.14	+7.86
Average	44.90	51.43	+6.53

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "EPFLiGHT/Apertus-70B-MeditronFO"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "A 62-year-old woman presents with a three-day history of dyspnea on exertion and a productive cough. What is the differential diagnosis?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Training

Base model: Apertus-70B-Instruct
Corpus: Fully Open Meditron — ~~601k examples (~~150M tokens), aggregating eight public medical QA datasets with three clinician-vetted synthetic components: exam-style QA, guideline-grounded QA from 46,469 clinical practice guidelines, and open-ended clinical vignettes
Hardware: NVIDIA GH200 nodes
Framework: Axolotl with FSDP v2 / DeepSpeed ZeRO-3, Flash Attention 2, bf16 mixed precision
Decontamination: System-wide two-stage n-gram and token-alignment decontamination against all evaluation benchmarks

Full hyperparameters are in Appendix I of the paper.

Intended Use

Research only. This model is intended to support research on medical LLMs, auditing of clinical AI systems, and reproducibility of the Fully Open Meditron pipeline.

It is not validated for clinical deployment, individual patient advice, autonomous decision-making, or any other deployment-adjacent use. Conduct independent domain-specific safety evaluation before any such use.