Text Generation
Transformers
Safetensors
English
qwen3
clinical
medical
instruction-following
tool-calling
function-calling
KOS-V4
from-scratch
conversational
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V4-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V4-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V4-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V4-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V4-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V4-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V4-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
Prepend shared kos-v4 family headline (data-fraction positioning)
Browse files
README.md
CHANGED
|
@@ -14,41 +14,39 @@ tags:
|
|
| 14 |
- from-scratch
|
| 15 |
---
|
| 16 |
|
| 17 |
-
## kos-v4
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
assembled
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
**
|
| 31 |
-
**
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
|
| 35 |
-
| :--
|
| 36 |
-
|
|
| 37 |
-
|
|
| 38 |
-
|
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
**
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
*Shared kos-v4 family headline — the model-specific card follows below.*
|
| 52 |
|
| 53 |
---
|
| 54 |
|
|
@@ -57,9 +55,11 @@ peer scores zero.
|
|
| 57 |
**kos-v4-instruct** is an open-weights **3B language model trained completely from scratch** by a small team. It is
|
| 58 |
a decoder-only transformer (Qwen3 architecture, bespoke 3B configuration) optimized for **instruction following and
|
| 59 |
tool / function calling** over clinical and general English text. Its instruction ability comes from **GRPO
|
| 60 |
-
reinforcement learning** on a from-scratch clinical base, reaching **IFEval AVG4 = 62.7**
|
| 61 |
-
|
| 62 |
-
**
|
|
|
|
|
|
|
| 63 |
|
| 64 |
## Core specifications
|
| 65 |
| Attribute | Detail |
|
|
@@ -70,12 +70,12 @@ Qwen2.5-3B-Instruct (65.4) — while holding strong tool-calling (BFCL 81/73/62)
|
|
| 70 |
| **Attention** | 24 query / 8 KV heads (GQA 3:1), head_dim 128, per-head QK-RMSNorm, no attention bias |
|
| 71 |
| **Feed-forward** | SwiGLU, intermediate size 8192 |
|
| 72 |
| **Embeddings** | Untied input/output |
|
| 73 |
-
| **Vocabulary** | 32,000
|
| 74 |
| **Context length** | 24,576 (`max_position_embeddings` 65,536) |
|
| 75 |
| **Position encoding** | RoPE, θ = 25,000 |
|
| 76 |
| **Precision** | bfloat16 |
|
| 77 |
| **Chat template** | ChatML (`<|im_start|>` / `<|im_end|>`, eos = `<|im_end|>`) |
|
| 78 |
-
| **Pretraining tokens** |
|
| 79 |
| **License** | *to be set by Kentucky Open Science* |
|
| 80 |
|
| 81 |
## Quickstart (Hugging Face Transformers)
|
|
@@ -88,11 +88,11 @@ tok = AutoTokenizer.from_pretrained(model_id)
|
|
| 88 |
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
|
| 89 |
|
| 90 |
messages = [
|
| 91 |
-
|
| 92 |
]
|
| 93 |
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
| 94 |
with torch.no_grad():
|
| 95 |
-
|
| 96 |
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
| 97 |
```
|
| 98 |
Serve with vLLM / TGI as a standard `Qwen3`-architecture causal LM. **Pin RoPE θ = 25000** on any GGUF/quantized export.
|
|
@@ -111,7 +111,7 @@ kos-v4-instruct uses **ChatML**:
|
|
| 111 |
tokens; pass tools via the tokenizer's chat template `tools=` argument.
|
| 112 |
|
| 113 |
## Pre-training (the kos-v4 base)
|
| 114 |
-
Trained from scratch
|
| 115 |
|
| 116 |
### Hyperparameters & schedule
|
| 117 |
| Parameter | Value |
|
|
@@ -122,81 +122,88 @@ Trained from scratch — not distilled or continued from another model.
|
|
| 122 |
| LR schedule | Cosine, warmup ratio 0.01, 1 epoch |
|
| 123 |
| Gradient clipping | 1.0 max-norm |
|
| 124 |
| Sequence length | 24,576 (whole-document neat-packing; docs > 24k dropped, never split) |
|
| 125 |
-
| Micro-batch / grad-accum | 1 / 1
|
| 126 |
| Precision / kernels | bfloat16, FlashAttention-2 |
|
| 127 |
-
| Total steps / tokens | 305,613 /
|
| 128 |
|
| 129 |
### Training data mixture
|
| 130 |
-
English-only, **medical/biomedical-first**, **49 sources /
|
| 131 |
-
(approx. tokens): PubMed Central full text
|
| 132 |
-
BlueScrubs
|
| 133 |
-
plus
|
| 134 |
-
MIMIC-CXR). Register mix (chunks): formal-academic 100 M, formal-clinical 14.5 M, patient-facing 7.8 M
|
| 135 |
technical-reference 5.9 M, conversational 0.5 M.
|
| 136 |
|
| 137 |
### Known pre-training issues (disclosed)
|
| 138 |
-
- **
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
- **38 "sink" tokens
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
- **RoPE θ drift.** A planned θ=10,000 was never applied; the model trained at **θ=25,000**
|
| 145 |
|
| 146 |
## Post-training (this model)
|
| 147 |
### 1. Instruction SFT
|
| 148 |
-
**Full-parameter** SFT (not LoRA) from the surgical-tokenizer base
|
| 149 |
-
NEFTune α=5**, whole-document packing on 8×H200. Corpus (
|
| 150 |
-
(
|
| 151 |
constraints sampled from the empirical IFEval distribution, teacher rejection-sampling kept only responses passing
|
| 152 |
-
the official checker), generic function-calling (xLAM, 20 k), and grounded cite/abstain RAG (
|
| 153 |
gate (broad perplexity ≤ 1.5× base) guards retention.
|
| 154 |
|
| 155 |
### 2. Reinforcement learning (GRPO)
|
| 156 |
-
**GRPO** via **verl** on 8×H200, with **deterministic verifiable rewards
|
| 157 |
-
primary reward runs the *official* IFEval constraint checker (dense shaping `0.7
|
| 158 |
sources reward grounded RAG (abstain-when-unanswerable) and tool-call recall. Settings: lr 1e-6, KL 0.001
|
| 159 |
(low-variance), group size n=16, 128 prompts/step, checkpoint every 40 steps (**ship the argmax**). Official IFEval
|
| 160 |
-
AVG4
|
|
|
|
| 161 |
|
| 162 |
## Evaluation & benchmarks
|
| 163 |
-
Public benchmarks only. **IFEval**
|
| 164 |
-
strict
|
|
|
|
|
|
|
| 165 |
|
| 166 |
-
| Benchmark | kos-v4-instruct |
|
| 167 |
-
| :--- | :--- | :--- | :--- |
|
| 168 |
-
| **IFEval** AVG4 | **62.7** |
|
| 169 |
-
| IFEval prompt-strict / inst-strict / prompt-loose / inst-loose | 55.8 / 67.0 / 58.4 / 69.7 |
|
| 170 |
-
| **BFCL** simple / multiple / parallel | **
|
| 171 |
-
| *(reference)* kos-v4 SFT base, pre-RL, IFEval AVG4 | 47.7 |
|
| 172 |
-
|
| 173 |
-
**Reading.** From a small team's from-scratch 3B, IFEval **62.7**
|
| 174 |
-
|
| 175 |
-
|
|
|
|
|
|
|
|
|
|
| 176 |
|
| 177 |
## Compute footprint & requirements
|
| 178 |
### Training
|
| 179 |
-
- **Pre-training:** 24× H200 (3 nodes, pure DDP),
|
| 180 |
- **SFT:** 1× 8×H200 node, 1 epoch. **RL (GRPO):** 1× 8×H200 node.
|
| 181 |
|
| 182 |
### Deployment (inference)
|
| 183 |
| Precision | Approx. VRAM | Notes |
|
| 184 |
| :--- | :--- | :--- |
|
| 185 |
-
| **bfloat16** |
|
| 186 |
-
| INT8 (approx.) |
|
| 187 |
-
| INT4 (approx.) |
|
| 188 |
|
| 189 |
## Intended use & limitations
|
| 190 |
- **Intended use:** general instruction following, structured output, and function/tool calling in clinical-adjacent
|
| 191 |
-
|
| 192 |
-
- **
|
| 193 |
-
|
| 194 |
-
- This
|
| 195 |
-
|
| 196 |
-
- IFEval 62.7
|
|
|
|
| 197 |
|
| 198 |
## The kos-v4 family
|
| 199 |
-
- **kos-v4-instruct**
|
| 200 |
-
- **[kos-v4-retriever](https://huggingface.co/Kentucky-Open-Science/kos-v4-retriever)**
|
| 201 |
-
- **[kos-v4-grounded-record](https://huggingface.co/Kentucky-Open-Science/kos-v4-grounded-record)**
|
| 202 |
-
- **[kos-v4-grounded-guideline](https://huggingface.co/Kentucky-Open-Science/kos-v4-grounded-guideline)**
|
|
|
|
| 14 |
- from-scratch
|
| 15 |
---
|
| 16 |
|
| 17 |
+
## kos-v4: a from-scratch 3B clinical suite that reaches original-ChatGPT-level instruction-following
|
| 18 |
+
|
| 19 |
+
kos-v4 is a **first release from a small team**, not a large lab. It is a family of **3B clinical models built
|
| 20 |
+
completely from scratch**: a custom 32k medical tokenizer, a from-scratch pretrained base, and task-specialized
|
| 21 |
+
post-training, assembled into a **grounded clinical RAG system** (a retriever, grounded readers that abstain when
|
| 22 |
+
the answer is absent, and an instruction and tool model). The goal is not to memorize medicine; it is to reliably
|
| 23 |
+
follow the standard of care retrieved for it.
|
| 24 |
+
|
| 25 |
+
**David and Goliath.** The base was trained on **180 billion tokens using 24 GPUs** over about six days. For
|
| 26 |
+
contrast, OpenAI built the original ChatGPT (**GPT-3.5**) on a Microsoft supercomputer of roughly **10,000 GPUs**
|
| 27 |
+
and 285,000 CPU cores, one of the largest machines in the world at the time. kos-v4 used on the order of **400 times
|
| 28 |
+
fewer GPUs**, a small fraction of the data, and trained from scratch as a small group.
|
| 29 |
+
|
| 30 |
+
**And it reaches that bar.** On **IFEval**, the standard instruction-following benchmark, kos-v4-instruct scores
|
| 31 |
+
**AVG4 62.7**, on par with and a touch above the reported **GPT-3.5-turbo (60.7)**, and within about three points of
|
| 32 |
+
a current big-company 3B, **Qwen2.5-3B-Instruct (65.4)**.
|
| 33 |
+
|
| 34 |
+
| IFEval (AVG4 of the four strict and loose metrics) | score | who built it, and how |
|
| 35 |
+
| :-- | :-- | :-- |
|
| 36 |
+
| **kos-v4-instruct (ours)** | **62.7** | a small team, from scratch, 180B tokens, 24 GPUs |
|
| 37 |
+
| GPT-3.5-turbo, the original ChatGPT (reported) | 60.7 | OpenAI, about 10,000-GPU supercomputer |
|
| 38 |
+
| Qwen2.5-3B-Instruct | 65.4 | Alibaba, about 18 trillion tokens |
|
| 39 |
+
|
| 40 |
+
**Where it genuinely leads.** On the RAG tasks it was built for, kos-v4 is competitive or best in class:
|
| 41 |
+
- **Guideline grounding** (RGB negative-rejection: refusing when the guidance does not cover the question) is
|
| 42 |
+
**45.0**, the best in our comparison set, above even the newer Qwen3-4B (40.7).
|
| 43 |
+
- **Retrieval** (BEIR SciFact, zero-shot) is **NDCG@10 = 0.7327**, inside the strong-dense band (0.70 to 0.76) and
|
| 44 |
+
above BM25 (0.665).
|
| 45 |
+
|
| 46 |
+
**Read honestly.** This is a first release from a small team on a small budget, not a frontier model. Our numbers
|
| 47 |
+
are measured on public benchmarks; the GPT-3.5-turbo figure is OpenAI's model as reported in published
|
| 48 |
+
instruction-following evaluations (53.8, 64.7, 56.6, 67.5 across the four metrics). Each model card states its own
|
| 49 |
+
results, including where it trails.
|
|
|
|
|
|
|
| 50 |
|
| 51 |
---
|
| 52 |
|
|
|
|
| 55 |
**kos-v4-instruct** is an open-weights **3B language model trained completely from scratch** by a small team. It is
|
| 56 |
a decoder-only transformer (Qwen3 architecture, bespoke 3B configuration) optimized for **instruction following and
|
| 57 |
tool / function calling** over clinical and general English text. Its instruction ability comes from **GRPO
|
| 58 |
+
reinforcement learning** on a from-scratch clinical base, reaching **IFEval AVG4 = 62.7**. That is on par with the
|
| 59 |
+
reported score for the original ChatGPT (**GPT-3.5-turbo, 60.7**) and within about three points of
|
| 60 |
+
**Qwen2.5-3B-Instruct (65.4)**, from a fraction of the data and compute. It also adds reliable **tool and function
|
| 61 |
+
calling** (BFCL 81/73/62), a capability the original ChatGPT did not have at launch. It is the flagship of the
|
| 62 |
+
**kos-v4** family (a retriever and two grounded readers ship alongside it).
|
| 63 |
|
| 64 |
## Core specifications
|
| 65 |
| Attribute | Detail |
|
|
|
|
| 70 |
| **Attention** | 24 query / 8 KV heads (GQA 3:1), head_dim 128, per-head QK-RMSNorm, no attention bias |
|
| 71 |
| **Feed-forward** | SwiGLU, intermediate size 8192 |
|
| 72 |
| **Embeddings** | Untied input/output |
|
| 73 |
+
| **Vocabulary** | 32,000, custom medical **byte-level BPE** (8% fewer tokens/medical-char than GPT-4/Llama/Qwen) |
|
| 74 |
| **Context length** | 24,576 (`max_position_embeddings` 65,536) |
|
| 75 |
| **Position encoding** | RoPE, θ = 25,000 |
|
| 76 |
| **Precision** | bfloat16 |
|
| 77 |
| **Chat template** | ChatML (`<|im_start|>` / `<|im_end|>`, eos = `<|im_end|>`) |
|
| 78 |
+
| **Pretraining tokens** | 180.3 B (English medical/biomedical + web) |
|
| 79 |
| **License** | *to be set by Kentucky Open Science* |
|
| 80 |
|
| 81 |
## Quickstart (Hugging Face Transformers)
|
|
|
|
| 88 |
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
|
| 89 |
|
| 90 |
messages = [
|
| 91 |
+
{"role": "user", "content": "List three contraindications for ibuprofen. Answer in exactly 3 bullet points."}
|
| 92 |
]
|
| 93 |
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
| 94 |
with torch.no_grad():
|
| 95 |
+
out = model.generate(inputs, max_new_tokens=256, do_sample=False)
|
| 96 |
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
| 97 |
```
|
| 98 |
Serve with vLLM / TGI as a standard `Qwen3`-architecture causal LM. **Pin RoPE θ = 25000** on any GGUF/quantized export.
|
|
|
|
| 111 |
tokens; pass tools via the tokenizer's chat template `tools=` argument.
|
| 112 |
|
| 113 |
## Pre-training (the kos-v4 base)
|
| 114 |
+
Trained from scratch, not distilled or continued from another model.
|
| 115 |
|
| 116 |
### Hyperparameters & schedule
|
| 117 |
| Parameter | Value |
|
|
|
|
| 122 |
| LR schedule | Cosine, warmup ratio 0.01, 1 epoch |
|
| 123 |
| Gradient clipping | 1.0 max-norm |
|
| 124 |
| Sequence length | 24,576 (whole-document neat-packing; docs > 24k dropped, never split) |
|
| 125 |
+
| Micro-batch / grad-accum | 1 / 1, giving a **global batch of 589,824 tokens/step** |
|
| 126 |
| Precision / kernels | bfloat16, FlashAttention-2 |
|
| 127 |
+
| Total steps / tokens | 305,613 / 180.3 B token-positions |
|
| 128 |
|
| 129 |
### Training data mixture
|
| 130 |
+
English-only, **medical/biomedical-first**, **49 sources / 130 M document chunks**. Largest contributors
|
| 131 |
+
(approx. tokens): PubMed Central full text 77 B, FineWeb-Edu (readability-filtered web) 51 B, mMedC-en 6.3 B
|
| 132 |
+
BlueScrubs 4.6 B, Hindawi 2.9 B, MedAL 2.5 B, S2ORC 1.2 B, clinical trials 0.7 B, Wikipedia 0.6 B
|
| 133 |
+
plus 30 smaller clinical / ontology / pharmacovigilance sources and de-identified clinical corpora (e.g. MIMIC-IV
|
| 134 |
+
MIMIC-CXR). Register mix (chunks): formal-academic 100 M, formal-clinical 14.5 M, patient-facing 7.8 M
|
| 135 |
technical-reference 5.9 M, conversational 0.5 M.
|
| 136 |
|
| 137 |
### Known pre-training issues (disclosed)
|
| 138 |
+
- **35 % duplicate tokens.** An audit found 35 % of training tokens were duplicate copies (a FineWeb-Edu shard/
|
| 139 |
+
filter build bug re-emitted byte-identical docs, 2.2× copies; plus 24 % in PMC). A fully deduplicated corpus was
|
| 140 |
+
built but **not** trained, so this base carries the duplication; dedup is a ready improvement for the next base.
|
| 141 |
+
- **38 "sink" tokens to surgical tokenizer.** 38 orphan BPE merges (markdown/table residue) behaved as a localized
|
| 142 |
+
attention sink; the 38 merge rules were removed (vocab stays 32,000), the ids baked as `bad_words_ids`, and a
|
| 143 |
+
fail-closed gate enforced at SFT/RL/serve.
|
| 144 |
+
- **RoPE θ drift.** A planned θ=10,000 was never applied; the model trained at **θ=25,000**, pin it on export.
|
| 145 |
|
| 146 |
## Post-training (this model)
|
| 147 |
### 1. Instruction SFT
|
| 148 |
+
**Full-parameter** SFT (not LoRA) from the surgical-tokenizer base, ChatML template, **LR 7e-5, 1 epoch
|
| 149 |
+
NEFTune α=5**, whole-document packing on 8×H200. Corpus (0.9 M+ rows): a stratified medical+tool+general majority
|
| 150 |
+
(615 k), a **23-family instruction-following taxonomy** (200 k, generated with a **verifier-in-the-loop** loop
|
| 151 |
constraints sampled from the empirical IFEval distribution, teacher rejection-sampling kept only responses passing
|
| 152 |
+
the official checker), generic function-calling (xLAM, 20 k), and grounded cite/abstain RAG (79 k). A forgetting
|
| 153 |
gate (broad perplexity ≤ 1.5× base) guards retention.
|
| 154 |
|
| 155 |
### 2. Reinforcement learning (GRPO)
|
| 156 |
+
**GRPO** via **verl** on 8×H200, with **deterministic verifiable rewards, no reward model, no LLM judge.** The
|
| 157 |
+
primary reward runs the *official* IFEval constraint checker (dense shaping `0.7*frac + 0.3*all_pass`); other data
|
| 158 |
sources reward grounded RAG (abstain-when-unanswerable) and tool-call recall. Settings: lr 1e-6, KL 0.001
|
| 159 |
(low-variance), group size n=16, 128 prompts/step, checkpoint every 40 steps (**ship the argmax**). Official IFEval
|
| 160 |
+
AVG4 rose across GRPO steps from **47.7 (SFT base)** to **62.7 at step 1120 (shipped)**, a gain of 15.0 points
|
| 161 |
+
(intermediate checkpoints scored 58.6, 59.4, 59.7, 61.5, then the 62.7 peak, then 61.8).
|
| 162 |
|
| 163 |
## Evaluation & benchmarks
|
| 164 |
+
Public benchmarks only. **IFEval** is the EleutherAI lm-evaluation-harness with `--apply_chat_template`, AVG4 of the
|
| 165 |
+
four strict and loose metrics. **BFCL** is the Berkeley Function-Calling AST checker. We measured kos-v4 and
|
| 166 |
+
Qwen2.5-3B ourselves on this harness. The GPT-3.5-turbo column is OpenAI's model **as reported** in published
|
| 167 |
+
instruction-following evaluations, included as the original-ChatGPT reference point rather than run by us.
|
| 168 |
|
| 169 |
+
| Benchmark | kos-v4-instruct (ours) | GPT-3.5-turbo (original ChatGPT, reported) | Qwen2.5-3B-Instruct |
|
| 170 |
+
| :--- | :--- | :--- | :--- |
|
| 171 |
+
| **IFEval** AVG4 | **62.7** | 60.7 | 65.4 |
|
| 172 |
+
| IFEval prompt-strict / inst-strict / prompt-loose / inst-loose | 55.8 / 67.0 / 58.4 / 69.7 | 53.8 / 64.7 / 56.6 / 67.5 | 59.5 / 68.5 / 62.1 / 71.3 |
|
| 173 |
+
| **BFCL** simple / multiple / parallel | **81 / 73 / 62** | none at launch | 95 / 89 / 75 |
|
| 174 |
+
| *(reference)* kos-v4 SFT base, pre-RL, IFEval AVG4 | 47.7 | | |
|
| 175 |
+
|
| 176 |
+
**Reading.** From a small team's from-scratch 3B, IFEval **62.7** lands right on the original ChatGPT
|
| 177 |
+
(GPT-3.5-turbo, a reported 60.7) and within about three points of Qwen2.5-3B-Instruct (65.4), while being
|
| 178 |
+
**15.0 points above our own SFT base** (47.7). On tool-calling, kos-v4 is functional (81/73/62), a capability the
|
| 179 |
+
original ChatGPT lacked at launch; general instruct models trained on far more tool data score higher (Qwen2.5-3B
|
| 180 |
+
at 95/89/75). Newer, more heavily distilled small models trained on 50 to 200 times more data also score higher on
|
| 181 |
+
IFEval; kos-v4's result is notable for its data and compute budget, not a claim to lead the current frontier.
|
| 182 |
|
| 183 |
## Compute footprint & requirements
|
| 184 |
### Training
|
| 185 |
+
- **Pre-training:** 24× H200 (3 nodes, pure DDP), 5.73 days, **about 3,300 H200-GPU-hours**.
|
| 186 |
- **SFT:** 1× 8×H200 node, 1 epoch. **RL (GRPO):** 1× 8×H200 node.
|
| 187 |
|
| 188 |
### Deployment (inference)
|
| 189 |
| Precision | Approx. VRAM | Notes |
|
| 190 |
| :--- | :--- | :--- |
|
| 191 |
+
| **bfloat16** | 7 GB | native weights (6.03 GB) + activations; 1× 16 GB GPU is comfortable |
|
| 192 |
+
| INT8 (approx.) | 4 GB | not shipped; typical for a 3 B model |
|
| 193 |
+
| INT4 (approx.) | 2.5 GB | not shipped; runs on consumer/edge hardware |
|
| 194 |
|
| 195 |
## Intended use & limitations
|
| 196 |
- **Intended use:** general instruction following, structured output, and function/tool calling in clinical-adjacent
|
| 197 |
+
workflows. For grounded record/guideline reading with abstention use **kos-v4-grounded-record** / **kos-v4-grounded-guideline**; for retrieval use **kos-v4-retriever**.
|
| 198 |
+
- **Not a medical-knowledge QA model.** It follows instructions and calls tools; it does not reliably recall
|
| 199 |
+
parametric medical facts and is not intended for medical multiple-choice exams. **Ground it with retrieval instead.**
|
| 200 |
+
- This instruction-tuned checkpoint trades away grounded abstention (its RGB negative-rejection is near zero), so for
|
| 201 |
+
non-fabrication on unanswerable questions use **kos-v4-grounded-record** or **kos-v4-grounded-guideline**.
|
| 202 |
+
- IFEval 62.7 is on the original-ChatGPT (GPT-3.5-turbo) level and near Qwen2.5-3B (65.4); newer small models trained
|
| 203 |
+
on far more data score higher. English only. Strong public-benchmark numbers are not validation on real clinical data.
|
| 204 |
|
| 205 |
## The kos-v4 family
|
| 206 |
+
- **kos-v4-instruct**, this model (general instruction + tool calling).
|
| 207 |
+
- **[kos-v4-retriever](https://huggingface.co/Kentucky-Open-Science/kos-v4-retriever)**, clinical dense retriever (BEIR SciFact 0.7327 zero-shot).
|
| 208 |
+
- **[kos-v4-grounded-record](https://huggingface.co/Kentucky-Open-Science/kos-v4-grounded-record)**, patient-record grounding specialist (RGB neg-rejection 35.0).
|
| 209 |
+
- **[kos-v4-grounded-guideline](https://huggingface.co/Kentucky-Open-Science/kos-v4-grounded-guideline)**, clinical-guideline grounding specialist (RGB neg-rejection 45.0).
|