Text Generation
Transformers
Safetensors
English
qwen3
clinical
medical
pretrained
base-model
KOS-V4
from-scratch
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V4-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V4-Base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V4-Base")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V4-Base") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V4-Base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V4-Base with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V4-Base" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Base
- SGLang
How to use Kentucky-Open-Science/KOS-V4-Base with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Base" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Base", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V4-Base with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Base
Card: add radiology/clinical home-turf win (rank 1/17 per-domain BPB vs trillion-token models) + tokenizer byte-efficiency win (rank 4/17); remove forward reference
Browse files
README.md
CHANGED
|
@@ -87,6 +87,14 @@ other special tokens** β no chat / ChatML tokens; the base was not trained on
|
|
| 87 |
residual byte-fragment "sink" merges present at pretraining have been **removed (corrected tokenizer)**; the
|
| 88 |
vocabulary stays 32,000. BPB is per-byte, so this does not confound cross-model comparison.
|
| 89 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
## Pre-training
|
| 91 |
Trained **from scratch, not distilled or continued**. Pure next-token cross-entropy (**no auxiliary losses**),
|
| 92 |
full-parameter on the LlamaFactory trainer, AdamW, peak LR 3.0e-4 cosine (warmup 1%), grad-clip 1.0,
|
|
@@ -125,7 +133,7 @@ kentucky_health web).
|
|
| 125 |
license verification; they will be added once resolved.)*
|
| 126 |
|
| 127 |
**Disclosed issue:** ~35% of tokens are duplicates (a FineWeb-Edu sharding build bug + PMC repetition); a deduped
|
| 128 |
-
corpus (`v4_dedup`, 79.4 M docs, 0% dup) is ready but was **not** trained
|
| 129 |
|
| 130 |
**Governance:** includes PhysioNet-credentialed, de-identified, redistribution-restricted clinical sources (MIMIC-IV
|
| 131 |
discharge / radiology, MIMIC-CXR) β any downstream use must confirm PhysioNet DUA compliance.
|
|
@@ -149,6 +157,15 @@ final-lens accuracy **0.5715** vs BioMedLM 0.2903 β a **top-3** placement) and
|
|
| 149 |
wins, KOS-V4-Base reaches **frontier parity (best-tied) on 6 metrics** (near-zero dead-neuron rate; saturated needle
|
| 150 |
retrieval at several depths) and **top-3 on 9 more**.
|
| 151 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 152 |
**Where it is weak:**
|
| 153 |
- **Closed-book medical MCQ** (T6 β near-chance): as a 180 B-token base it does not reliably recall parametric medical
|
| 154 |
facts; it is worst on the hardest professional/college splits (mmlu_professional_medicine **0.2096**,
|
|
|
|
| 87 |
residual byte-fragment "sink" merges present at pretraining have been **removed (corrected tokenizer)**; the
|
| 88 |
vocabulary stays 32,000. BPB is per-byte, so this does not confound cross-model comparison.
|
| 89 |
|
| 90 |
+
**Where the tokenizer does well.** On raw byte-compression efficiency (tokens-per-byte **0.2591**) it ranks **4th of
|
| 91 |
+
17**, essentially tied with the most byte-efficient general tokenizers (Llama-3 0.2573) β fewer tokens per document, a
|
| 92 |
+
multiplicative saving on both training and inference. It deliberately does *not* optimize for whole-word medical
|
| 93 |
+
vocabulary, so its medical single-token rate (T1b STRR) is the lowest in the pool β but that tradeoff is **vindicated**:
|
| 94 |
+
despite the lowest single-token rate, the model still ranks **1st of 17 on held-out medical BPB and leads the pool on
|
| 95 |
+
medical entity extraction**. A high single-token rate is neither necessary for nor sufficient for strong downstream
|
| 96 |
+
medical modeling.
|
| 97 |
+
|
| 98 |
## Pre-training
|
| 99 |
Trained **from scratch, not distilled or continued**. Pure next-token cross-entropy (**no auxiliary losses**),
|
| 100 |
full-parameter on the LlamaFactory trainer, AdamW, peak LR 3.0e-4 cosine (warmup 1%), grad-clip 1.0,
|
|
|
|
| 133 |
license verification; they will be added once resolved.)*
|
| 134 |
|
| 135 |
**Disclosed issue:** ~35% of tokens are duplicates (a FineWeb-Edu sharding build bug + PMC repetition); a deduped
|
| 136 |
+
corpus (`v4_dedup`, 79.4 M docs, 0% dup) is ready but was **not** trained β this release is the original single-epoch corpus.
|
| 137 |
|
| 138 |
**Governance:** includes PhysioNet-credentialed, de-identified, redistribution-restricted clinical sources (MIMIC-IV
|
| 139 |
discharge / radiology, MIMIC-CXR) β any downstream use must confirm PhysioNet DUA compliance.
|
|
|
|
| 157 |
wins, KOS-V4-Base reaches **frontier parity (best-tied) on 6 metrics** (near-zero dead-neuron rate; saturated needle
|
| 158 |
retrieval at several depths) and **top-3 on 9 more**.
|
| 159 |
|
| 160 |
+
**Radiology and clinical text is its home turf β and there it beats every trillion-token model in the pool.** On
|
| 161 |
+
per-domain held-out BPB (lower = better) it ranks **1st of 17** on radiology reports (**0.4761** vs Qwen3-4B/36 T
|
| 162 |
+
0.7995, Llama-3-8B 0.7444, Gemma-2-9B 0.7760), chest-X-ray reports (**0.5887**), and clinical narratives (**0.3221**);
|
| 163 |
+
and **1st of 17** on radiology entity/relation extraction (RadGraph DR.1/2/3 macro-F1 **0.7665 / 0.7990 / 0.8150**). It
|
| 164 |
+
*trails* on the general axes the trillion-token models saturate β biomedical literature (rank 17/17) and textbooks
|
| 165 |
+
(rank 12/17) β but the clinical/radiology wins are decisive enough to carry the **5-domain BPB mean to rank 1/17**. A
|
| 166 |
+
3 B model on 180 B tokens out-modeling 4β9 B models trained on 8β36 T tokens, specifically on the clinical text it was
|
| 167 |
+
built for, is the payoff of a from-scratch clinical corpus.
|
| 168 |
+
|
| 169 |
**Where it is weak:**
|
| 170 |
- **Closed-book medical MCQ** (T6 β near-chance): as a 180 B-token base it does not reliably recall parametric medical
|
| 171 |
facts; it is worst on the hardest professional/college splits (mmlu_professional_medicine **0.2096**,
|