Instructions to use BrainHealthAI/BrainMed-8B-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use BrainHealthAI/BrainMed-8B-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="BrainHealthAI/BrainMed-8B-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("BrainHealthAI/BrainMed-8B-SFT")
model = AutoModelForCausalLM.from_pretrained("BrainHealthAI/BrainMed-8B-SFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use BrainHealthAI/BrainMed-8B-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "BrainHealthAI/BrainMed-8B-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BrainHealthAI/BrainMed-8B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/BrainHealthAI/BrainMed-8B-SFT

SGLang

How to use BrainHealthAI/BrainMed-8B-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "BrainHealthAI/BrainMed-8B-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BrainHealthAI/BrainMed-8B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "BrainHealthAI/BrainMed-8B-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BrainHealthAI/BrainMed-8B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use BrainHealthAI/BrainMed-8B-SFT with Docker Model Runner:
```
docker model run hf.co/BrainHealthAI/BrainMed-8B-SFT
```

BrainMed-8B-SFT / evaluation /REPORT.md

Williamsanderson

evaluation results and figures

8a33135 verified 16 days ago

preview code

Raw

History Blame Contribute Delete

36.7 kB

	# BrainMed reasoning SFT - evaluation report

	Rows marked (ours) were measured by this pipeline (greedy decoding, strict prompt,
	MedReason scorer with `<answer>` support, training system prompt applied at inference).
	All other rows are transcribed from MedReason (arXiv:2504.00993, Table 4) and were
	not re-run.

	## Comparison with published 7B-8B models (MedReason Table 4)

	\| Model \| MB-op4 \| MB-op5 \| MedXpert \| MedQA \| MedMCQA \| PubMedQA \| Avg \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Llama3.1-Instruct-8B \| 43.2 \| 40.9 \| 14.3 \| 58.7 \| 56.0 \| 75.2 \| 48.0 \|
	\| Qwen2.5-Instruct-7B \| 50.0 \| 41.6 \| 12.6 \| 57.0 \| 55.6 \| 72.7 \| 48.2 \|
	\| Mistral-Instruct-7B \| 43.5 \| 33.4 \| 11.4 \| 48.2 \| 44.9 \| 50.1 \| 38.6 \|
	\| Medical-Llama3-8B \| 33.4 \| 25.3 \| 9.0 \| 40.3 \| 46.8 \| 48.0 \| 33.8 \|
	\| OpenBioLLM-8B \| 39.2 \| 35.7 \| 10.7 \| 57.7 \| 54.1 \| 74.1 \| 45.3 \|
	\| BioMistral-7B \| 46.4 \| 33.1 \| 12.4 \| 45.0 \| 40.2 \| 66.9 \| 40.7 \|
	\| Medical-CoT-8B \| 39.3 \| 34.1 \| 12.6 \| 49.0 \| 42.6 \| 68.0 \| 40.9 \|
	\| DeepSeek-Distill-8B \| 41.9 \| 35.1 \| 13.5 \| 55.4 \| 49.0 \| 73.9 \| 44.8 \|
	\| Huatuo-o1-SFT-8B \| 53.3 \| 49.7 \| 17.3 \| 70.2 \| 58.2 \| 76.1 \| 54.1 \|
	\| Huatuo-o1-RL-8B \| 55.2 \| 51.3 \| 16.7 \| 72.6 \| 60.4 \| 79.2 \| 55.9 \|
	\| MedReason-8B \| 57.5 \| 55.5 \| 19.0 \| 71.8 \| 60.7 \| 79.4 \| 57.3 \|
	\| base-HuatuoGPT-o1-8B (ours) \| 59.09 \| 55.19 \| 17.05 \| 77.93 \| 63.3 \| 80.2 \| 58.79 \|
	\| brainmed-8b-final (ours) \| 62.01 \| 54.87 \| 18.63 \| 76.67 \| 64.07 \| 79.1 \| 59.23 \|
	\| brainmed-8b-v1 (ours) \| 56.49 \| 50.65 \| 17.81 \| 72.58 \| 60.22 \| 77.7 \| 55.91 \|
	\| brainmed-8b-v1__best (ours) \| 56.49 \| 50.65 \| 17.81 \| 72.58 \| 60.22 \| 77.7 \| 55.91 \|
	\| brainmed-8b-v1__epoch-1 (ours) \| 55.52 \| 50.0 \| 15.87 \| 70.62 \| 59.77 \| 77.5 \| 54.88 \|
	\| brainmed-8b-v1__epoch-2 (ours) \| 56.49 \| 50.65 \| 17.81 \| 72.58 \| 60.22 \| 77.7 \| 55.91 \|
	\| brainmed-8b-v1__epoch-3 (ours) \| 58.77 \| 56.82 \| 18.08 \| 72.11 \| 61.2 \| 77.1 \| 57.35 \|
	\| brainmed-8b-v1__last (ours) \| 58.77 \| 56.82 \| 18.08 \| 72.11 \| 61.2 \| 77.1 \| 57.35 \|
	\| brainmed-8b-v1__soup-a0.15 (ours) \| 60.06 \| 50.97 \| 16.08 \| 77.85 \| 63.21 \| 81.2 \| 58.23 \|
	\| brainmed-8b-v1__soup-a0.3 (ours) \| 59.42 \| 54.87 \| 16.43 \| 76.28 \| 62.54 \| 80.0 \| 58.26 \|
	\| brainmed-8b-v1__soup-last-a0.3 (ours) \| 62.01 \| 52.92 \| 18.63 \| 76.67 \| 64.07 \| 79.1 \| 58.9 \|
	\| brainmed-8b-v1__soup-last-a0.5 (ours) \| 58.77 \| 54.55 \| 18.91 \| 76.9 \| 64.36 \| 78.9 \| 58.73 \|
	\| brainmed-8b-v1__soup-last-a0.7 (ours) \| 60.71 \| 56.49 \| 19.32 \| 76.12 \| 62.87 \| 77.9 \| 58.9 \|


	## Table 2 style - effect of the training data (MedReason Table 2)

	Published rows compare the same backbone trained on Huatuo CoT vs MedReason.
	Our row adds the union corpus. Only rows sharing a backbone are comparable:
	read the Llama3.1 block against a Llama3.1 run, not against a Huatuo-o1 run.

	\| Model / data \| MedQA \| MedMCQA \| PubMedQA \| MMLU-Pro \| MB-op4 \| MB-op5 \| MedXpert \| HLE(med) \| Avg \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Llama3.1-Instruct-8B (base) \| 58.7 \| 56.0 \| 75.2 \| 58.2 \| 48.7 \| 42.5 \| 13.2 \| 13.6 \| 45.8 \|
	\| Llama3.1-Instruct-8B + Huatuo CoT \| 70.2 \| 58.2 \| 76.1 \| 59.9 \| 53.3 \| 49.7 \| 17.3 \| 14.6 \| 49.9 \|
	\| Llama3.1-Instruct-8B + MedReason \| 68.4 \| 57.5 \| 77.6 \| 63.1 \| 57.5 \| 52.3 \| 16.4 \| 16.5 \| 51.2 \|
	\| Mistral-Instruct-7B (base) \| 48.2 \| 44.9 \| 50.1 \| 42.7 \| 43.5 \| 33.4 \| 11.4 \| 14.6 \| 36.1 \|
	\| Mistral-Instruct-7B + Huatuo CoT \| 59.9 \| 46.9 \| 57.5 \| 47.6 \| 50.0 \| 46.1 \| 14.4 \| 14.6 \| 42.1 \|
	\| Mistral-Instruct-7B + MedReason \| 58.7 \| 48.9 \| 59.2 \| 50.8 \| 52.3 \| 47.1 \| 16.6 \| 24.3 \| 44.7 \|
	\| base-HuatuoGPT-o1-8B + ours (union) (ours) \| 77.93 \| 63.3 \| 80.2 \| 64.56 \| 59.09 \| 55.19 \| 17.05 \| 11.65 \| 53.62 \|
	\| brainmed-8b-final + ours (union) (ours) \| 76.67 \| 64.07 \| 79.1 \| 65.73 \| 62.01 \| 54.87 \| 18.63 \| 10.68 \| 53.97 \|
	\| brainmed-8b-v1 + ours (union) (ours) \| 72.58 \| 60.22 \| 77.7 \| 61.82 \| 56.49 \| 50.65 \| 17.81 \| 20.39 \| 52.21 \|

	## Table 3 style - common vs challenging averages (MedReason Table 3)

	\| Model / data \| Avg common (4) \| Avg challenging (4) \| Avg overall (8) \|
	\|---\|---\|---\|---\|
	\| Medical-CoT-8B (base) \| 52.1 \| 25.4 \| 38.75 \|
	\| Medical-CoT-8B + MedReason \| 57.4 \| 30.6 \| 44.0 \|
	\| DeepSeek-Distill-8B (base) \| 58.0 \| 25.5 \| 41.75 \|
	\| DeepSeek-Distill-8B + MedReason \| 61.5 \| 33.3 \| 47.4 \|
	\| base-HuatuoGPT-o1-8B (ours) \| 71.5 \| 35.75 \| 53.62 \|
	\| brainmed-8b-final (ours) \| 71.39 \| 36.55 \| 53.97 \|
	\| brainmed-8b-v1 (ours) \| 68.08 \| 36.34 \| 52.21 \|

	## HuatuoGPT-o1 Table 1/2 style - core medical benchmarks

	`MMLU-Pro (Med)` and `GPQA (Med)` are the merged medical tracks shipped in
	`eval_data.json`; Huatuo's Table 1 splits them per track, which the released files
	do not allow us to reproduce, so this uses their Table 2 granularity.

	\| Model \| MedQA \| MedMCQA \| PubMedQA \| MMLU-Pro \| GPQA(med) \| Avg \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| LLaMA-3.1-8B-Instruct (base) \| 58.7 \| 56.0 \| 75.2 \| 58.2 \| 44.1 \| 58.44 \|
	\| SFT w/ original exam data \| 60.0 \| 55.5 \| 74.1 \| 54.3 \| 46.9 \| 58.16 \|
	\| SFT w/o CoT \| 65.2 \| 58.1 \| 75.4 \| 58.5 \| 48.7 \| 61.18 \|
	\| SFT w/ Simple CoT \| 66.6 \| 59.2 \| 75.4 \| 57.0 \| 46.7 \| 60.98 \|
	\| SFT w/ Complex CoT \| 69.0 \| 57.9 \| 77.7 \| 59.4 \| 51.0 \| 63.0 \|
	\| SFT w/o CoT + RL (PPO) \| 66.4 \| 58.6 \| 76.3 \| 60.1 \| 49.8 \| 62.24 \|
	\| SFT w/ Simple CoT + RL (PPO) \| 68.7 \| 58.4 \| 77.5 \| 60.2 \| 53.1 \| 63.58 \|
	\| SFT w/ Complex CoT + RL (PPO) = o1-8B \| 72.6 \| 60.4 \| 79.2 \| 63.1 \| 57.5 \| 66.56 \|
	\| SFT w/ Complex CoT + RL (DPO) \| 72.2 \| 58.4 \| 77.3 \| 60.4 \| 52.5 \| 64.16 \|
	\| SFT w/ Complex CoT + RL (RLOO) \| 71.1 \| 60.1 \| 78.1 \| 60.9 \| 58.2 \| 65.68 \|
	\| base-HuatuoGPT-o1-8B (ours) \| 77.93 \| 63.3 \| 80.2 \| 64.56 \| 49.74 \| 67.15 \|
	\| brainmed-8b-final (ours) \| 76.67 \| 64.07 \| 79.1 \| 65.73 \| 59.74 \| 69.06 \|
	\| brainmed-8b-v1 (ours) \| 72.58 \| 60.22 \| 77.7 \| 61.82 \| 49.23 \| 64.31 \|

	## Table 5 style - pipeline ablations

	MedReason's Table 5 ablates quality filtering. The equivalent knobs here are
	decontamination (`--no_decontaminate`) and MCQ answer-format alignment
	(`--no_answer_alignment`). Each additional arm costs one more training run - the
	table below fills in from whatever runs exist.

	\| Run \| MedQA \| MedMCQA \| PubMedQA \| MMLU-Pro \| MB-op4 \| MB-op5 \| MedXpert \| HLE(med) \| Avg (8) \| Avg (Table 4, 6) \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| brainmed-8b-final \| 76.67 \| 64.07 \| 79.1 \| 65.73 \| 62.01 \| 54.87 \| 18.63 \| 10.68 \| 53.97 \| 59.23 \|
	\| brainmed-8b-v1 \| 72.58 \| 60.22 \| 77.7 \| 61.82 \| 56.49 \| 50.65 \| 17.81 \| 20.39 \| 52.21 \| 55.91 \|
	\| brainmed-8b-v1__best \| 72.58 \| 60.22 \| 77.7 \| \| 56.49 \| 50.65 \| 17.81 \| \| - \| 55.91 \|
	\| brainmed-8b-v1__epoch-1 \| 70.62 \| 59.77 \| 77.5 \| \| 55.52 \| 50.0 \| 15.87 \| \| - \| 54.88 \|
	\| brainmed-8b-v1__epoch-2 \| 72.58 \| 60.22 \| 77.7 \| \| 56.49 \| 50.65 \| 17.81 \| \| - \| 55.91 \|
	\| brainmed-8b-v1__epoch-3 \| 72.11 \| 61.2 \| 77.1 \| \| 58.77 \| 56.82 \| 18.08 \| \| - \| 57.35 \|
	\| brainmed-8b-v1__last \| 72.11 \| 61.2 \| 77.1 \| \| 58.77 \| 56.82 \| 18.08 \| \| - \| 57.35 \|
	\| brainmed-8b-v1__soup-a0.15 \| 77.85 \| 63.21 \| 81.2 \| \| 60.06 \| 50.97 \| 16.08 \| \| - \| 58.23 \|
	\| brainmed-8b-v1__soup-a0.3 \| 76.28 \| 62.54 \| 80.0 \| \| 59.42 \| 54.87 \| 16.43 \| \| - \| 58.26 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| 76.67 \| 64.07 \| 79.1 \| \| 62.01 \| 52.92 \| 18.63 \| \| - \| 58.9 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| 76.9 \| 64.36 \| 78.9 \| \| 58.77 \| 54.55 \| 18.91 \| \| - \| 58.73 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| 76.12 \| 62.87 \| 77.9 \| \| 60.71 \| 56.49 \| 19.32 \| \| - \| 58.9 \|

	The two averages cover different column sets and must never be compared to each other; `Avg (Table 4, 6)` is the one the published tables use.

	## All evaluated benchmarks

	Every benchmark scored in this run (10 of them). The paper's averages
	cover only the eight it reports; the extras below are measured and kept for the
	record but excluded from those averages.

	\| Benchmark \| n \| In paper avg \| base-HuatuoGPT-o1-8B \| brainmed-8b-final \| brainmed-8b-v1 \| brainmed-8b-v1__best \| brainmed-8b-v1__epoch-1 \| brainmed-8b-v1__epoch-2 \| brainmed-8b-v1__epoch-3 \| brainmed-8b-v1__last \| brainmed-8b-v1__soup-a0.15 \| brainmed-8b-v1__soup-a0.3 \| brainmed-8b-v1__soup-last-a0.3 \| brainmed-8b-v1__soup-last-a0.5 \| brainmed-8b-v1__soup-last-a0.7 \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| MedQA \| 1273 \| yes \| 77.93 \| 76.67 \| 72.58 \| 72.58 \| 70.62 \| 72.58 \| 72.11 \| 72.11 \| 77.85 \| 76.28 \| 76.67 \| 76.9 \| 76.12 \|
	\| MedMCQA \| 4183 \| yes \| 63.3 \| 64.07 \| 60.22 \| 60.22 \| 59.77 \| 60.22 \| 61.2 \| 61.2 \| 63.21 \| 62.54 \| 64.07 \| 64.36 \| 62.87 \|
	\| PubMedQA \| 1000 \| yes \| 80.2 \| 79.1 \| 77.7 \| 77.7 \| 77.5 \| 77.7 \| 77.1 \| 77.1 \| 81.2 \| 80.0 \| 79.1 \| 78.9 \| 77.9 \|
	\| MMLU-Pro \| 1535 \| yes \| 64.56 \| 65.73 \| 61.82 \| - \| - \| - \| - \| - \| - \| - \| - \| - \| - \|
	\| MB-op4 \| 308 \| yes \| 59.09 \| 62.01 \| 56.49 \| 56.49 \| 55.52 \| 56.49 \| 58.77 \| 58.77 \| 60.06 \| 59.42 \| 62.01 \| 58.77 \| 60.71 \|
	\| MB-op5 \| 308 \| yes \| 55.19 \| 54.87 \| 50.65 \| 50.65 \| 50.0 \| 50.65 \| 56.82 \| 56.82 \| 50.97 \| 54.87 \| 52.92 \| 54.55 \| 56.49 \|
	\| MedXpert \| 1449 \| yes \| 17.05 \| 18.63 \| 17.81 \| 17.81 \| 15.87 \| 17.81 \| 18.08 \| 18.08 \| 16.08 \| 16.43 \| 18.63 \| 18.91 \| 19.32 \|
	\| HLE(med) \| 103 \| yes \| 11.65 \| 10.68 \| 20.39 \| - \| - \| - \| - \| - \| - \| - \| - \| - \| - \|
	\| GPQA(med) \| 390 \| extra \| 49.74 \| 59.74 \| 49.23 \| - \| - \| - \| - \| - \| - \| - \| - \| - \| - \|
	\| MedQA-5opt \| 1273 \| extra \| 72.82 \| 73.68 \| 68.81 \| - \| - \| - \| - \| - \| - \| - \| - \| - \| - \|

	## Paper-comparable suite (8 benchmarks)

	\| Run \| MedQA \| MedMCQA \| PubMedQA \| MMLU-Pro \| Avg common \| MB-op4 \| MB-op5 \| MedXpert \| HLE(med) \| Avg challenging \| Avg overall \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| base-HuatuoGPT-o1-8B \| 77.93 \| 63.3 \| 80.2 \| 64.56 \| 71.5 \| 59.09 \| 55.19 \| 17.05 \| 11.65 \| 35.75 \| 53.62 \|
	\| brainmed-8b-final \| 76.67 \| 64.07 \| 79.1 \| 65.73 \| 71.39 \| 62.01 \| 54.87 \| 18.63 \| 10.68 \| 36.55 \| 53.97 \|
	\| brainmed-8b-v1 \| 72.58 \| 60.22 \| 77.7 \| 61.82 \| 68.08 \| 56.49 \| 50.65 \| 17.81 \| 20.39 \| 36.34 \| 52.21 \|
	\| brainmed-8b-v1__best \| 72.58 \| 60.22 \| 77.7 \| \| - \| 56.49 \| 50.65 \| 17.81 \| \| - \| - \|
	\| brainmed-8b-v1__epoch-1 \| 70.62 \| 59.77 \| 77.5 \| \| - \| 55.52 \| 50.0 \| 15.87 \| \| - \| - \|
	\| brainmed-8b-v1__epoch-2 \| 72.58 \| 60.22 \| 77.7 \| \| - \| 56.49 \| 50.65 \| 17.81 \| \| - \| - \|
	\| brainmed-8b-v1__epoch-3 \| 72.11 \| 61.2 \| 77.1 \| \| - \| 58.77 \| 56.82 \| 18.08 \| \| - \| - \|
	\| brainmed-8b-v1__last \| 72.11 \| 61.2 \| 77.1 \| \| - \| 58.77 \| 56.82 \| 18.08 \| \| - \| - \|
	\| brainmed-8b-v1__soup-a0.15 \| 77.85 \| 63.21 \| 81.2 \| \| - \| 60.06 \| 50.97 \| 16.08 \| \| - \| - \|
	\| brainmed-8b-v1__soup-a0.3 \| 76.28 \| 62.54 \| 80.0 \| \| - \| 59.42 \| 54.87 \| 16.43 \| \| - \| - \|
	\| brainmed-8b-v1__soup-last-a0.3 \| 76.67 \| 64.07 \| 79.1 \| \| - \| 62.01 \| 52.92 \| 18.63 \| \| - \| - \|
	\| brainmed-8b-v1__soup-last-a0.5 \| 76.9 \| 64.36 \| 78.9 \| \| - \| 58.77 \| 54.55 \| 18.91 \| \| - \| - \|
	\| brainmed-8b-v1__soup-last-a0.7 \| 76.12 \| 62.87 \| 77.9 \| \| - \| 60.71 \| 56.49 \| 19.32 \| \| - \| - \|

	`-` marks an average that cannot be computed: brainmed-8b-v1__best, brainmed-8b-v1__epoch-1, brainmed-8b-v1__epoch-2, brainmed-8b-v1__epoch-3, brainmed-8b-v1__last, brainmed-8b-v1__soup-a0.15, brainmed-8b-v1__soup-a0.3, brainmed-8b-v1__soup-last-a0.3, brainmed-8b-v1__soup-last-a0.5, brainmed-8b-v1__soup-last-a0.7 were scored on a subset of the suite (QUICK sweep). Re-run the full bundle before quoting an 8-benchmark average for them.

	## Win conditions

	Two things must both hold: the model beats the backbone it started from, and it
	beats every published 7B-8B model in MedReason Table 4. The backbone is compared
	on numbers measured here; the others on their published rows.

	\| Run \| Condition \| Reference \| Ours \| Delta \| Verdict \|
	\|---\|---\|---\|---\|---\|---\|
	\| brainmed-8b-final \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 59.23 \| +0.44 \| PASS \|
	\| brainmed-8b-final \| beats published best (MedReason-8B) \| 57.3 \| 59.23 \| +1.93 \| PASS \|
	\| brainmed-8b-final \| beats all 11 published models \| - \| 59.23 \| 11/11 \| PASS \|
	\| brainmed-8b-v1 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 55.91 \| -2.88 \| FAIL \|
	\| brainmed-8b-v1 \| beats published best (MedReason-8B) \| 57.3 \| 55.91 \| -1.39 \| FAIL \|
	\| brainmed-8b-v1 \| beats all 11 published models \| - \| 55.91 \| 10/11 \| FAIL (MedReason-8B) \|
	\| brainmed-8b-v1__best \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 55.91 \| -2.88 \| FAIL \|
	\| brainmed-8b-v1__best \| beats published best (MedReason-8B) \| 57.3 \| 55.91 \| -1.39 \| FAIL \|
	\| brainmed-8b-v1__best \| beats all 11 published models \| - \| 55.91 \| 10/11 \| FAIL (MedReason-8B) \|
	\| brainmed-8b-v1__epoch-1 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 54.88 \| -3.91 \| FAIL \|
	\| brainmed-8b-v1__epoch-1 \| beats published best (MedReason-8B) \| 57.3 \| 54.88 \| -2.42 \| FAIL \|
	\| brainmed-8b-v1__epoch-1 \| beats all 11 published models \| - \| 54.88 \| 9/11 \| FAIL (Huatuo-o1-RL-8B, MedReason-8B) \|
	\| brainmed-8b-v1__epoch-2 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 55.91 \| -2.88 \| FAIL \|
	\| brainmed-8b-v1__epoch-2 \| beats published best (MedReason-8B) \| 57.3 \| 55.91 \| -1.39 \| FAIL \|
	\| brainmed-8b-v1__epoch-2 \| beats all 11 published models \| - \| 55.91 \| 10/11 \| FAIL (MedReason-8B) \|
	\| brainmed-8b-v1__epoch-3 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 57.35 \| -1.44 \| FAIL \|
	\| brainmed-8b-v1__epoch-3 \| beats published best (MedReason-8B) \| 57.3 \| 57.35 \| +0.05 \| PASS \|
	\| brainmed-8b-v1__epoch-3 \| beats all 11 published models \| - \| 57.35 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__last \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 57.35 \| -1.44 \| FAIL \|
	\| brainmed-8b-v1__last \| beats published best (MedReason-8B) \| 57.3 \| 57.35 \| +0.05 \| PASS \|
	\| brainmed-8b-v1__last \| beats all 11 published models \| - \| 57.35 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__soup-a0.15 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 58.23 \| -0.56 \| FAIL \|
	\| brainmed-8b-v1__soup-a0.15 \| beats published best (MedReason-8B) \| 57.3 \| 58.23 \| +0.93 \| PASS \|
	\| brainmed-8b-v1__soup-a0.15 \| beats all 11 published models \| - \| 58.23 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__soup-a0.3 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 58.26 \| -0.53 \| FAIL \|
	\| brainmed-8b-v1__soup-a0.3 \| beats published best (MedReason-8B) \| 57.3 \| 58.26 \| +0.96 \| PASS \|
	\| brainmed-8b-v1__soup-a0.3 \| beats all 11 published models \| - \| 58.26 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.3 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 58.9 \| +0.11 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.3 \| beats published best (MedReason-8B) \| 57.3 \| 58.9 \| +1.60 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.3 \| beats all 11 published models \| - \| 58.9 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.5 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 58.73 \| -0.06 \| FAIL \|
	\| brainmed-8b-v1__soup-last-a0.5 \| beats published best (MedReason-8B) \| 57.3 \| 58.73 \| +1.43 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.5 \| beats all 11 published models \| - \| 58.73 \| 11/11 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.7 \| beats backbone (base-HuatuoGPT-o1-8B, measured) \| 58.79 \| 58.9 \| +0.11 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.7 \| beats published best (MedReason-8B) \| 57.3 \| 58.9 \| +1.60 \| PASS \|
	\| brainmed-8b-v1__soup-last-a0.7 \| beats all 11 published models \| - \| 58.9 \| 11/11 \| PASS \|

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-final)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 76.67 \| -1.26 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 64.07 \| +0.77 \| OK \|
	\| PubMedQA \| 80.2 \| 79.1 \| -1.10 \| REGRESSION \|
	\| MMLU-Pro \| 64.56 \| 65.73 \| +1.17 \| OK \|
	\| MB-op4 \| 59.09 \| 62.01 \| +2.92 \| OK \|
	\| MB-op5 \| 55.19 \| 54.87 \| -0.32 \| OK \|
	\| MedXpert \| 17.05 \| 18.63 \| +1.58 \| OK \|
	\| HLE(med) \| 11.65 \| 10.68 \| -0.97 \| REGRESSION \|
	\| Average \| 53.62 \| 53.97 \| +0.35 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -1.26, PubMedQA -1.10, HLE(med) -0.97.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 72.58 \| -5.35 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 60.22 \| -3.08 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.7 \| -2.50 \| REGRESSION \|
	\| MMLU-Pro \| 64.56 \| 61.82 \| -2.74 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 56.49 \| -2.60 \| REGRESSION \|
	\| MB-op5 \| 55.19 \| 50.65 \| -4.54 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 17.81 \| +0.76 \| OK \|
	\| HLE(med) \| 11.65 \| 20.39 \| +8.74 \| OK \|
	\| Average \| 53.62 \| 52.21 \| -1.41 \| \|

	GATE FAILED - 6 benchmark(s) below the -0.5 pt tolerance: MedQA -5.35, MedMCQA -3.08, PubMedQA -2.50, MMLU-Pro -2.74, MB-op4 -2.60, MB-op5 -4.54.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__best)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 72.58 \| -5.35 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 60.22 \| -3.08 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.7 \| -2.50 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 56.49 \| -2.60 \| REGRESSION \|
	\| MB-op5 \| 55.19 \| 50.65 \| -4.54 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 17.81 \| +0.76 \| OK \|
	\| Average \| 58.79 \| 55.91 \| -2.88 \| \|

	GATE FAILED - 5 benchmark(s) below the -0.5 pt tolerance: MedQA -5.35, MedMCQA -3.08, PubMedQA -2.50, MB-op4 -2.60, MB-op5 -4.54.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__epoch-1)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 70.62 \| -7.31 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 59.77 \| -3.53 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.5 \| -2.70 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 55.52 \| -3.57 \| REGRESSION \|
	\| MB-op5 \| 55.19 \| 50.0 \| -5.19 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 15.87 \| -1.18 \| REGRESSION \|
	\| Average \| 58.79 \| 54.88 \| -3.91 \| \|

	GATE FAILED - 6 benchmark(s) below the -0.5 pt tolerance: MedQA -7.31, MedMCQA -3.53, PubMedQA -2.70, MB-op4 -3.57, MB-op5 -5.19, MedXpert -1.18.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__epoch-2)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 72.58 \| -5.35 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 60.22 \| -3.08 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.7 \| -2.50 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 56.49 \| -2.60 \| REGRESSION \|
	\| MB-op5 \| 55.19 \| 50.65 \| -4.54 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 17.81 \| +0.76 \| OK \|
	\| Average \| 58.79 \| 55.91 \| -2.88 \| \|

	GATE FAILED - 5 benchmark(s) below the -0.5 pt tolerance: MedQA -5.35, MedMCQA -3.08, PubMedQA -2.50, MB-op4 -2.60, MB-op5 -4.54.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__epoch-3)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 72.11 \| -5.82 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 61.2 \| -2.10 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.1 \| -3.10 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 58.77 \| -0.32 \| OK \|
	\| MB-op5 \| 55.19 \| 56.82 \| +1.63 \| OK \|
	\| MedXpert \| 17.05 \| 18.08 \| +1.03 \| OK \|
	\| Average \| 58.79 \| 57.35 \| -1.45 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -5.82, MedMCQA -2.10, PubMedQA -3.10.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__last)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 72.11 \| -5.82 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 61.2 \| -2.10 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 77.1 \| -3.10 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 58.77 \| -0.32 \| OK \|
	\| MB-op5 \| 55.19 \| 56.82 \| +1.63 \| OK \|
	\| MedXpert \| 17.05 \| 18.08 \| +1.03 \| OK \|
	\| Average \| 58.79 \| 57.35 \| -1.45 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -5.82, MedMCQA -2.10, PubMedQA -3.10.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__soup-a0.15)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 77.85 \| -0.08 \| OK \|
	\| MedMCQA \| 63.3 \| 63.21 \| -0.09 \| OK \|
	\| PubMedQA \| 80.2 \| 81.2 \| +1.00 \| OK \|
	\| MB-op4 \| 59.09 \| 60.06 \| +0.97 \| OK \|
	\| MB-op5 \| 55.19 \| 50.97 \| -4.22 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 16.08 \| -0.97 \| REGRESSION \|
	\| Average \| 58.79 \| 58.23 \| -0.56 \| \|

	GATE FAILED - 2 benchmark(s) below the -0.5 pt tolerance: MB-op5 -4.22, MedXpert -0.97.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__soup-a0.3)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 76.28 \| -1.65 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 62.54 \| -0.76 \| REGRESSION \|
	\| PubMedQA \| 80.2 \| 80.0 \| -0.20 \| OK \|
	\| MB-op4 \| 59.09 \| 59.42 \| +0.33 \| OK \|
	\| MB-op5 \| 55.19 \| 54.87 \| -0.32 \| OK \|
	\| MedXpert \| 17.05 \| 16.43 \| -0.62 \| REGRESSION \|
	\| Average \| 58.79 \| 58.26 \| -0.54 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -1.65, MedMCQA -0.76, MedXpert -0.62.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__soup-last-a0.3)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 76.67 \| -1.26 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 64.07 \| +0.77 \| OK \|
	\| PubMedQA \| 80.2 \| 79.1 \| -1.10 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 62.01 \| +2.92 \| OK \|
	\| MB-op5 \| 55.19 \| 52.92 \| -2.27 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 18.63 \| +1.58 \| OK \|
	\| Average \| 58.79 \| 58.9 \| +0.11 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -1.26, PubMedQA -1.10, MB-op5 -2.27.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__soup-last-a0.5)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 76.9 \| -1.03 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 64.36 \| +1.06 \| OK \|
	\| PubMedQA \| 80.2 \| 78.9 \| -1.30 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 58.77 \| -0.32 \| OK \|
	\| MB-op5 \| 55.19 \| 54.55 \| -0.64 \| REGRESSION \|
	\| MedXpert \| 17.05 \| 18.91 \| +1.86 \| OK \|
	\| Average \| 58.79 \| 58.73 \| -0.06 \| \|

	GATE FAILED - 3 benchmark(s) below the -0.5 pt tolerance: MedQA -1.03, PubMedQA -1.30, MB-op5 -0.64.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Controlled before/after - same harness (base-HuatuoGPT-o1-8B -> brainmed-8b-v1__soup-last-a0.7)

	\| Benchmark \| Before \| After \| Delta \| Gate \|
	\|---\|---\|---\|---\|---\|
	\| MedQA \| 77.93 \| 76.12 \| -1.81 \| REGRESSION \|
	\| MedMCQA \| 63.3 \| 62.87 \| -0.43 \| OK \|
	\| PubMedQA \| 80.2 \| 77.9 \| -2.30 \| REGRESSION \|
	\| MB-op4 \| 59.09 \| 60.71 \| +1.62 \| OK \|
	\| MB-op5 \| 55.19 \| 56.49 \| +1.30 \| OK \|
	\| MedXpert \| 17.05 \| 19.32 \| +2.27 \| OK \|
	\| Average \| 58.79 \| 58.9 \| +0.11 \| \|

	GATE FAILED - 2 benchmark(s) below the -0.5 pt tolerance: MedQA -1.81, PubMedQA -2.30.

	The fine-tune traded existing strengths for its gains. Before shipping, either lower the learning rate / shorten the schedule, or interpolate towards the backbone with `scripts/weight_soup.py` and re-evaluate.

	## Harness calibration - backbone measured here vs its published row

	Backbone `base-HuatuoGPT-o1-8B` against the published Huatuo-o1-RL-8B. Small gaps are
	expected; a large one means the harness, not the training, is moving the
	numbers, and every delta downstream has to be read with that in mind.

	\| Benchmark \| Published \| Measured here \| Gap \|
	\|---\|---\|---\|---\|
	\| MB-op4 \| 55.2 \| 59.09 \| +3.89 \|
	\| MB-op5 \| 51.3 \| 55.19 \| +3.89 \|
	\| MedXpert \| 16.7 \| 17.05 \| +0.35 \|
	\| MedQA \| 72.6 \| 77.93 \| +5.33 \|
	\| MedMCQA \| 60.4 \| 63.3 \| +2.90 \|
	\| PubMedQA \| 79.2 \| 80.2 \| +1.00 \|
	\| Mean gap \| \| \| +2.89 \|

	Largest single-benchmark gap: 5.33 pt. That is large enough to confound a data-driven gain - prefer the measured before/after over comparisons with published rows.

	## Raw score vs clean subset

	The training set kept the rows overlapping these benchmarks (the upstream
	setting, which is what the published tables were produced under). `Raw` is
	therefore the number comparable to the paper; `Clean` excludes the overlapping
	items and is the number to quote as an unbiased estimate. A large gap between
	the two means the score leans on memorised evaluation items.

	\| Run \| Benchmark \| n \| Contaminated \| Raw \| Clean \| Bias \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| base-HuatuoGPT-o1-8B \| MedQA \| 1273 \| 1 \| 77.93 \| 77.99 \| -0.06 \|
	\| base-HuatuoGPT-o1-8B \| MMLU-Pro \| 1535 \| 340 \| 64.56 \| 64.52 \| +0.04 \|
	\| base-HuatuoGPT-o1-8B \| MB-op4 \| 308 \| 35 \| 59.09 \| 58.97 \| +0.12 \|
	\| base-HuatuoGPT-o1-8B \| MB-op5 \| 308 \| 35 \| 55.19 \| 53.48 \| +1.71 \|
	\| base-HuatuoGPT-o1-8B \| MedXpert \| 1449 \| 15 \| 17.05 \| 16.88 \| +0.17 \|
	\| base-HuatuoGPT-o1-8B \| HLE(med) \| 103 \| 30 \| 11.65 \| 8.22 \| +3.43 \|
	\| brainmed-8b-final \| MedQA \| 1273 \| 1 \| 76.67 \| 76.73 \| -0.06 \|
	\| brainmed-8b-final \| MMLU-Pro \| 1535 \| 340 \| 65.73 \| 65.44 \| +0.29 \|
	\| brainmed-8b-final \| MB-op4 \| 308 \| 35 \| 62.01 \| 62.64 \| -0.63 \|
	\| brainmed-8b-final \| MB-op5 \| 308 \| 35 \| 54.87 \| 54.21 \| +0.66 \|
	\| brainmed-8b-final \| MedXpert \| 1449 \| 15 \| 18.63 \| 18.34 \| +0.29 \|
	\| brainmed-8b-final \| HLE(med) \| 103 \| 30 \| 10.68 \| 6.85 \| +3.83 \|
	\| brainmed-8b-v1 \| MedQA \| 1273 \| 1 \| 72.58 \| 72.56 \| +0.02 \|
	\| brainmed-8b-v1 \| MMLU-Pro \| 1535 \| 340 \| 61.82 \| 60.67 \| +1.15 \|
	\| brainmed-8b-v1 \| MB-op4 \| 308 \| 35 \| 56.49 \| 56.41 \| +0.08 \|
	\| brainmed-8b-v1 \| MB-op5 \| 308 \| 35 \| 50.65 \| 49.08 \| +1.57 \|
	\| brainmed-8b-v1 \| MedXpert \| 1449 \| 15 \| 17.81 \| 17.5 \| +0.31 \|
	\| brainmed-8b-v1 \| HLE(med) \| 103 \| 30 \| 20.39 \| 17.81 \| +2.58 \|
	\| brainmed-8b-v1__best \| MedQA \| 1273 \| 1 \| 72.58 \| 72.56 \| +0.02 \|
	\| brainmed-8b-v1__best \| MB-op4 \| 308 \| 35 \| 56.49 \| 56.41 \| +0.08 \|
	\| brainmed-8b-v1__best \| MB-op5 \| 308 \| 35 \| 50.65 \| 49.08 \| +1.57 \|
	\| brainmed-8b-v1__best \| MedXpert \| 1449 \| 15 \| 17.81 \| 17.5 \| +0.31 \|
	\| brainmed-8b-v1__epoch-1 \| MedQA \| 1273 \| 1 \| 70.62 \| 70.6 \| +0.02 \|
	\| brainmed-8b-v1__epoch-1 \| MB-op4 \| 308 \| 35 \| 55.52 \| 55.31 \| +0.21 \|
	\| brainmed-8b-v1__epoch-1 \| MB-op5 \| 308 \| 35 \| 50.0 \| 48.72 \| +1.28 \|
	\| brainmed-8b-v1__epoch-1 \| MedXpert \| 1449 \| 15 \| 15.87 \| 15.83 \| +0.04 \|
	\| brainmed-8b-v1__epoch-2 \| MedQA \| 1273 \| 1 \| 72.58 \| 72.56 \| +0.02 \|
	\| brainmed-8b-v1__epoch-2 \| MB-op4 \| 308 \| 35 \| 56.49 \| 56.41 \| +0.08 \|
	\| brainmed-8b-v1__epoch-2 \| MB-op5 \| 308 \| 35 \| 50.65 \| 49.08 \| +1.57 \|
	\| brainmed-8b-v1__epoch-2 \| MedXpert \| 1449 \| 15 \| 17.81 \| 17.5 \| +0.31 \|
	\| brainmed-8b-v1__epoch-3 \| MedQA \| 1273 \| 1 \| 72.11 \| 72.17 \| -0.06 \|
	\| brainmed-8b-v1__epoch-3 \| MB-op4 \| 308 \| 35 \| 58.77 \| 57.14 \| +1.63 \|
	\| brainmed-8b-v1__epoch-3 \| MB-op5 \| 308 \| 35 \| 56.82 \| 57.51 \| -0.69 \|
	\| brainmed-8b-v1__epoch-3 \| MedXpert \| 1449 \| 15 \| 18.08 \| 17.78 \| +0.30 \|
	\| brainmed-8b-v1__last \| MedQA \| 1273 \| 1 \| 72.11 \| 72.17 \| -0.06 \|
	\| brainmed-8b-v1__last \| MB-op4 \| 308 \| 35 \| 58.77 \| 57.14 \| +1.63 \|
	\| brainmed-8b-v1__last \| MB-op5 \| 308 \| 35 \| 56.82 \| 57.51 \| -0.69 \|
	\| brainmed-8b-v1__last \| MedXpert \| 1449 \| 15 \| 18.08 \| 17.78 \| +0.30 \|
	\| brainmed-8b-v1__soup-a0.15 \| MedQA \| 1273 \| 1 \| 77.85 \| 77.91 \| -0.06 \|
	\| brainmed-8b-v1__soup-a0.15 \| MB-op4 \| 308 \| 35 \| 60.06 \| 60.44 \| -0.38 \|
	\| brainmed-8b-v1__soup-a0.15 \| MB-op5 \| 308 \| 35 \| 50.97 \| 50.92 \| +0.05 \|
	\| brainmed-8b-v1__soup-a0.15 \| MedXpert \| 1449 \| 15 \| 16.08 \| 15.76 \| +0.32 \|
	\| brainmed-8b-v1__soup-a0.3 \| MedQA \| 1273 \| 1 \| 76.28 \| 76.26 \| +0.02 \|
	\| brainmed-8b-v1__soup-a0.3 \| MB-op4 \| 308 \| 35 \| 59.42 \| 60.07 \| -0.65 \|
	\| brainmed-8b-v1__soup-a0.3 \| MB-op5 \| 308 \| 35 \| 54.87 \| 54.95 \| -0.08 \|
	\| brainmed-8b-v1__soup-a0.3 \| MedXpert \| 1449 \| 15 \| 16.43 \| 16.11 \| +0.32 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MedQA \| 1273 \| 1 \| 76.67 \| 76.73 \| -0.06 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MB-op4 \| 308 \| 35 \| 62.01 \| 62.64 \| -0.63 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MB-op5 \| 308 \| 35 \| 52.92 \| 52.75 \| +0.17 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MedXpert \| 1449 \| 15 \| 18.63 \| 18.34 \| +0.29 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MedQA \| 1273 \| 1 \| 76.9 \| 76.97 \| -0.07 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MB-op4 \| 308 \| 35 \| 58.77 \| 57.88 \| +0.89 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MB-op5 \| 308 \| 35 \| 54.55 \| 53.11 \| +1.44 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MedXpert \| 1449 \| 15 \| 18.91 \| 18.55 \| +0.36 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MedQA \| 1273 \| 1 \| 76.12 \| 76.18 \| -0.06 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MB-op4 \| 308 \| 35 \| 60.71 \| 59.71 \| +1.00 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MB-op5 \| 308 \| 35 \| 56.49 \| 54.58 \| +1.91 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MedXpert \| 1449 \| 15 \| 19.32 \| 19.11 \| +0.21 \|
	\| base-HuatuoGPT-o1-8B \| Average (8) \| \| \| 53.62 \| 52.95 \| +0.67 \|
	\| brainmed-8b-final \| Average (8) \| \| \| 53.97 \| 53.42 \| +0.55 \|
	\| brainmed-8b-v1 \| Average (8) \| \| \| 52.21 \| 51.49 \| +0.72 \|

	## Data integrity

	- training rows after decontamination: 44351 (removed 0, 0.0%)

	Contamination found in the source corpus and removed before training:

	\| Benchmark \| n \| Leaked items \| % \| Train rows dropped \|
	\|---\|---\|---\|---\|---\|
	\| gpqa_medical.jsonl \| 390 \| 0 \| 0.0% \| 0 \|
	\| hle_med.jsonl \| 103 \| 30 \| 29.13% \| 30 \|
	\| medbullets_op4.jsonl \| 308 \| 35 \| 11.36% \| 35 \|
	\| medbullets_op5.jsonl \| 308 \| 35 \| 11.36% \| 35 \|
	\| medmcqa_val.jsonl \| 4183 \| 0 \| 0.0% \| 0 \|
	\| medqa_4opt.jsonl \| 1273 \| 1 \| 0.08% \| 1 \|
	\| medqa_5opt.jsonl \| 1273 \| 1 \| 0.08% \| 1 \|
	\| medxpertqa.jsonl \| 1449 \| 15 \| 1.04% \| 17 \|
	\| mmlu_pro_medical.jsonl \| 1535 \| 340 \| 22.15% \| 261 \|
	\| pubmedqa_test.jsonl \| 1000 \| 0 \| 0.0% \| 0 \|

	## Answer-format compliance

	Share of outputs where the answer was read from an explicit `The answer is X.` rather
	than recovered by the scorer's fuzzy fallbacks. A low value means the accuracy above
	leans on string matching and should be treated with suspicion.

	\| Run \| Benchmark \| Strict-format % \| Empty outputs \|
	\|---\|---\|---\|---\|
	\| base-HuatuoGPT-o1-8B \| MedQA \| 99.84% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MedMCQA \| 99.81% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| PubMedQA \| 100.0% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MMLU-Pro \| 99.15% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MB-op4 \| 100.0% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MB-op5 \| 99.68% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MedXpert \| 96.27% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| HLE(med) \| 96.12% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| GPQA(med) \| 98.21% \| 0 \|
	\| base-HuatuoGPT-o1-8B \| MedQA-5opt \| 99.76% \| 0 \|
	\| brainmed-8b-final \| MedQA \| 99.14% \| 0 \|
	\| brainmed-8b-final \| MedMCQA \| 99.83% \| 0 \|
	\| brainmed-8b-final \| PubMedQA \| 99.9% \| 0 \|
	\| brainmed-8b-final \| MMLU-Pro \| 99.48% \| 0 \|
	\| brainmed-8b-final \| MB-op4 \| 99.68% \| 0 \|
	\| brainmed-8b-final \| MB-op5 \| 99.35% \| 0 \|
	\| brainmed-8b-final \| MedXpert \| 99.24% \| 0 \|
	\| brainmed-8b-final \| HLE(med) \| 97.09% \| 0 \|
	\| brainmed-8b-final \| GPQA(med) \| 98.46% \| 0 \|
	\| brainmed-8b-final \| MedQA-5opt \| 98.9% \| 0 \|
	\| brainmed-8b-v1 \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1 \| MedMCQA \| 99.88% \| 0 \|
	\| brainmed-8b-v1 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1 \| MMLU-Pro \| 99.61% \| 0 \|
	\| brainmed-8b-v1 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1 \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1 \| HLE(med) \| 95.15% \| 0 \|
	\| brainmed-8b-v1 \| GPQA(med) \| 99.74% \| 0 \|
	\| brainmed-8b-v1 \| MedQA-5opt \| 99.92% \| 0 \|
	\| brainmed-8b-v1__best \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__best \| MedMCQA \| 99.88% \| 0 \|
	\| brainmed-8b-v1__best \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__best \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__best \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__best \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| MedQA \| 99.92% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| MedMCQA \| 99.62% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| PubMedQA \| 99.9% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-1 \| MedXpert \| 99.72% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| MedMCQA \| 99.88% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-2 \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| MedMCQA \| 99.9% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__epoch-3 \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1__last \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__last \| MedMCQA \| 99.9% \| 0 \|
	\| brainmed-8b-v1__last \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__last \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__last \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__last \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| MedQA \| 89.32% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| MedMCQA \| 99.19% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| PubMedQA \| 99.6% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| MB-op4 \| 86.69% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| MB-op5 \| 87.66% \| 0 \|
	\| brainmed-8b-v1__soup-a0.15 \| MedXpert \| 92.41% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| MedQA \| 99.69% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| MedMCQA \| 99.64% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| MB-op4 \| 99.03% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| MB-op5 \| 99.68% \| 0 \|
	\| brainmed-8b-v1__soup-a0.3 \| MedXpert \| 99.59% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MedQA \| 99.14% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MedMCQA \| 99.83% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| PubMedQA \| 99.9% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MB-op4 \| 99.68% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MB-op5 \| 99.35% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.3 \| MedXpert \| 99.24% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MedQA \| 99.84% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MedMCQA \| 99.86% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.5 \| MedXpert \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MedMCQA \| 99.93% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| PubMedQA \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MB-op4 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MB-op5 \| 100.0% \| 0 \|
	\| brainmed-8b-v1__soup-last-a0.7 \| MedXpert \| 100.0% \| 0 \|