Text Generation
Transformers
Safetensors
English
qwen3
clinical
medical
instruction-following
tool-calling
function-calling
KOS-V4
from-scratch
conversational
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V4-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V4-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V4-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V4-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V4-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V4-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V4-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
Manual update
Browse files
README.md
CHANGED
|
@@ -34,29 +34,19 @@ Kentucky College of Medicine team ([Office for Research](https://medicine.uky.ed
|
|
| 34 |
(Qwen3 architecture, bespoke 3B config) optimized for **instruction following and tool / function calling**. Its
|
| 35 |
instruction ability comes from **GRPO reinforcement learning** on a from-scratch clinical base.
|
| 36 |
|
| 37 |
-
**
|
| 38 |
-
— the exact metric the Hugging Face Open LLM Leaderboard publishes as "IFEval."
|
| 39 |
-
four-metric average anywhere.
|
| 40 |
-
|
| 41 |
-
**David and Goliath.** KOS-V4 was trained on **180 billion tokens using 24 GPUs**. OpenAI trained the original
|
| 42 |
-
ChatGPT (**GPT-3.5**) on a ~10,000-GPU supercomputer, and today's open models train on **trillions** of tokens
|
| 43 |
-
(Qwen2.5 ~18T, Llama-3 ~15T). KOS-V4's 180B tokens is **~100× less data**. From that budget:
|
| 44 |
|
| 45 |
| IFEval **strict-avg** | model | who built it, and how |
|
| 46 |
| --: | :-- | :-- |
|
|
|
|
| 47 |
| **61.6** | **KOS-V4-Instruct (ours)** | a small team, from scratch, 180B tokens, 24 GPUs |
|
| 48 |
| 55.9 | GPT-3.5-turbo-1106 (the original ChatGPT) | OpenAI, ~10,000-GPU supercomputer |
|
| 49 |
-
| 64.7 | Qwen2.5-3B-Instruct | Alibaba, ~18 trillion tokens |
|
| 50 |
|
| 51 |
KOS-V4-Instruct **clears the original GPT-3.5-turbo generation (55.9)** and lands within ~3 points of the
|
| 52 |
commercially trained Qwen2.5-3B (64.7). It also adds real **tool / function calling** (official BFCL 72.75/73/60.5),
|
| 53 |
which the original ChatGPT lacked at launch — though modern small models score higher there.
|
| 54 |
|
| 55 |
-
**Benchmarks are official-suite only:** IFEval via the EleutherAI lm-evaluation-harness (`--apply_chat_template`),
|
| 56 |
-
BFCL via the official `bfcl_eval`. Peer strict-avg values are the Open LLM Leaderboard's "IFEval" (strict). Our 61.6
|
| 57 |
-
is measured on our own copy of that harness; it calibrates cleanly (we measure Qwen2.5-3B at 64.0 vs the leaderboard's
|
| 58 |
-
64.7, a 0.7-pt gap).
|
| 59 |
-
|
| 60 |
## Core specifications
|
| 61 |
| Attribute | Detail |
|
| 62 |
| :--- | :--- |
|
|
@@ -121,20 +111,10 @@ the *official* IFEval checker. GRPO lifted the **official IFEval strict-avg from
|
|
| 121 |
|
| 122 |
## Evaluation & benchmarks
|
| 123 |
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
| instruction-level strict | 67.1 | 61.2 | 68.5 | 79.3 | 87.6 |
|
| 129 |
-
| **strict-avg (headline)** | **61.6** | **55.9** | **64.0** | **75.2** | **85.0** |
|
| 130 |
-
|
| 131 |
-
*(These four peers are measured on our own harness for exact apples-to-apples; the "in context" table below uses the
|
| 132 |
-
public Open LLM Leaderboard strict number for a wider set.)*
|
| 133 |
-
|
| 134 |
-
### BFCL (official `bfcl_eval`, function-calling mode; simple / multiple / parallel)
|
| 135 |
-
| BFCL (official FC) | KOS-V4-Instruct | Qwen2.5-3B | Llama-3.2-3B |
|
| 136 |
-
| :-- | :-- | :-- | :-- |
|
| 137 |
-
| simple / multiple / parallel | **72.75 / 73.00 / 60.50** | 95.00 / 92.00 / 74.50 | 91.75 / 92.50 / 88.50 |
|
| 138 |
|
| 139 |
## IFEval in context (strict-avg, models our size or larger)
|
| 140 |
One metric for everyone: **strict-avg**. Open-model values are the **Open LLM Leaderboard "IFEval"** (which *is*
|
|
@@ -195,6 +175,11 @@ claim to lead the current field.
|
|
| 195 |
(arXiv:2403.17297) and Open LLM Leaderboard raw results. `~ *` rows are estimates from published AVG4 (strict runs a
|
| 196 |
few points lower), clearly labeled.
|
| 197 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 198 |
## Compute footprint
|
| 199 |
Pre-training: 24× H200 (3 nodes, DDP), 5.73 days, ~3,300 H200-GPU-hours. SFT + RL (GRPO): a single 8-GPU node each.
|
| 200 |
|
|
|
|
| 34 |
(Qwen3 architecture, bespoke 3B config) optimized for **instruction following and tool / function calling**. Its
|
| 35 |
instruction ability comes from **GRPO reinforcement learning** on a from-scratch clinical base.
|
| 36 |
|
| 37 |
+
**IFEval reported as strict-avg** = `(prompt-level strict + instruction-level strict) / 2`
|
| 38 |
+
— the exact metric the Hugging Face Open LLM Leaderboard publishes as "IFEval."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
| IFEval **strict-avg** | model | who built it, and how |
|
| 41 |
| --: | :-- | :-- |
|
| 42 |
+
| 64.7 | Qwen2.5-3B-Instruct | Alibaba, ~18 trillion tokens |
|
| 43 |
| **61.6** | **KOS-V4-Instruct (ours)** | a small team, from scratch, 180B tokens, 24 GPUs |
|
| 44 |
| 55.9 | GPT-3.5-turbo-1106 (the original ChatGPT) | OpenAI, ~10,000-GPU supercomputer |
|
|
|
|
| 45 |
|
| 46 |
KOS-V4-Instruct **clears the original GPT-3.5-turbo generation (55.9)** and lands within ~3 points of the
|
| 47 |
commercially trained Qwen2.5-3B (64.7). It also adds real **tool / function calling** (official BFCL 72.75/73/60.5),
|
| 48 |
which the original ChatGPT lacked at launch — though modern small models score higher there.
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
## Core specifications
|
| 51 |
| Attribute | Detail |
|
| 52 |
| :--- | :--- |
|
|
|
|
| 111 |
|
| 112 |
## Evaluation & benchmarks
|
| 113 |
|
| 114 |
+
**Benchmarks are official-suite only:** IFEval via the EleutherAI lm-evaluation-harness (`--apply_chat_template`),
|
| 115 |
+
BFCL via the official `bfcl_eval`. Peer strict-avg values are the Open LLM Leaderboard's "IFEval" (strict). Our 61.6
|
| 116 |
+
is measured on our own copy of that harness; it calibrates cleanly (we measure Qwen2.5-3B at 64.0 vs the leaderboard's
|
| 117 |
+
64.7, a 0.7-pt gap).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
|
| 119 |
## IFEval in context (strict-avg, models our size or larger)
|
| 120 |
One metric for everyone: **strict-avg**. Open-model values are the **Open LLM Leaderboard "IFEval"** (which *is*
|
|
|
|
| 175 |
(arXiv:2403.17297) and Open LLM Leaderboard raw results. `~ *` rows are estimates from published AVG4 (strict runs a
|
| 176 |
few points lower), clearly labeled.
|
| 177 |
|
| 178 |
+
### BFCL (official `bfcl_eval`, function-calling mode; simple / multiple / parallel)
|
| 179 |
+
| BFCL (official FC) | KOS-V4-Instruct | Qwen2.5-3B | Llama-3.2-3B |
|
| 180 |
+
| :-- | :-- | :-- | :-- |
|
| 181 |
+
| simple / multiple / parallel | **72.75 / 73.00 / 60.50** | 95.00 / 92.00 / 74.50 | 91.75 / 92.50 / 88.50 |
|
| 182 |
+
|
| 183 |
## Compute footprint
|
| 184 |
Pre-training: 24× H200 (3 nodes, DDP), 5.73 days, ~3,300 H200-GPU-hours. SFT + RL (GRPO): a single 8-GPU node each.
|
| 185 |
|