Text Generation
Transformers
Safetensors
English
qwen3
clinical
medical
instruction-following
tool-calling
function-calling
KOS-V4
from-scratch
conversational
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V4-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V4-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V4-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V4-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V4-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V4-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V4-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V4-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V4-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V4-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V4-Instruct
Rebrand to KOS-V4; David-vs-Goliath GPT-3.5 framing; complete peer tables
Browse files
README.md
CHANGED
|
@@ -65,7 +65,8 @@ tool / function calling** over clinical and general English text. Its instructio
|
|
| 65 |
reinforcement learning** on a from-scratch clinical base, reaching **IFEval AVG4 = 62.7**. That is on par with the
|
| 66 |
reported score for the original ChatGPT (**GPT-3.5-turbo, 60.7**) and within about three points of
|
| 67 |
**Qwen2.5-3B-Instruct (65.4)**, from a fraction of the data and compute. It also adds reliable **tool and function
|
| 68 |
-
calling** (BFCL
|
|
|
|
| 69 |
**KOS-V4** family (a retriever and two grounded readers ship alongside it).
|
| 70 |
|
| 71 |
## Core specifications
|
|
@@ -169,23 +170,31 @@ AVG4 rose across GRPO steps from **47.7 (SFT base)** to **62.7 at step 1120 (shi
|
|
| 169 |
|
| 170 |
## Evaluation & benchmarks
|
| 171 |
Public benchmarks only. **IFEval** is the EleutherAI lm-evaluation-harness with `--apply_chat_template`, AVG4 of the
|
| 172 |
-
four strict and loose metrics. **BFCL** is the Berkeley Function-Calling
|
| 173 |
-
|
| 174 |
-
|
|
|
|
| 175 |
|
| 176 |
-
|
|
| 177 |
| :--- | :--- | :--- | :--- |
|
| 178 |
-
| **
|
| 179 |
-
|
|
| 180 |
-
| **
|
| 181 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
|
| 183 |
**Reading.** From a small team's from-scratch 3B, IFEval **62.7** lands right on the original ChatGPT
|
| 184 |
(GPT-3.5-turbo, a reported 60.7) and within about three points of Qwen2.5-3B-Instruct (65.4), while being
|
| 185 |
-
**15.0 points above our own SFT base** (47.7). On tool-calling, KOS-V4
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
|
|
|
|
|
|
| 189 |
|
| 190 |
## Compute footprint & requirements
|
| 191 |
### Training
|
|
|
|
| 65 |
reinforcement learning** on a from-scratch clinical base, reaching **IFEval AVG4 = 62.7**. That is on par with the
|
| 66 |
reported score for the original ChatGPT (**GPT-3.5-turbo, 60.7**) and within about three points of
|
| 67 |
**Qwen2.5-3B-Instruct (65.4)**, from a fraction of the data and compute. It also adds reliable **tool and function
|
| 68 |
+
calling** (official BFCL 73/73/61), a capability the original ChatGPT lacked at launch, though modern small models
|
| 69 |
+
score higher here. It is the flagship of the
|
| 70 |
**KOS-V4** family (a retriever and two grounded readers ship alongside it).
|
| 71 |
|
| 72 |
## Core specifications
|
|
|
|
| 170 |
|
| 171 |
## Evaluation & benchmarks
|
| 172 |
Public benchmarks only. **IFEval** is the EleutherAI lm-evaluation-harness with `--apply_chat_template`, AVG4 of the
|
| 173 |
+
four strict and loose metrics. **BFCL** is the official Berkeley Function-Calling Leaderboard package (`bfcl_eval`),
|
| 174 |
+
run with its **official per-model native handlers** in function-calling mode: each model is prompted in its own tool
|
| 175 |
+
format and scored by the official AST checker. The GPT-3.5-turbo IFEval column is OpenAI's model **as reported** in
|
| 176 |
+
published instruction-following evaluations, included as the original-ChatGPT reference point rather than run by us.
|
| 177 |
|
| 178 |
+
| IFEval | KOS-V4-Instruct (ours) | GPT-3.5-turbo (original ChatGPT, reported) | Qwen2.5-3B-Instruct |
|
| 179 |
| :--- | :--- | :--- | :--- |
|
| 180 |
+
| **AVG4** | **62.7** | 60.7 | 65.4 |
|
| 181 |
+
| prompt-strict / inst-strict / prompt-loose / inst-loose | 55.8 / 67.0 / 58.4 / 69.7 | 53.8 / 64.7 / 56.6 / 67.5 | 59.5 / 68.5 / 62.1 / 71.3 |
|
| 182 |
+
| *(reference)* KOS-V4 SFT base, pre-RL AVG4 | 47.7 | | |
|
| 183 |
+
|
| 184 |
+
Tool-calling, **official BFCL** (function-calling mode; simple / multiple / parallel accuracy, higher is better):
|
| 185 |
+
|
| 186 |
+
| BFCL (official FC) | KOS-V4-Instruct (ours) | Qwen2.5-3B-Instruct | Llama-3.2-3B-Instruct |
|
| 187 |
+
| :--- | :--- | :--- | :--- |
|
| 188 |
+
| simple / multiple / parallel | **72.75 / 73.00 / 60.50** | 95.00 / 92.00 / 74.50 | 91.75 / 92.50 / 88.50 |
|
| 189 |
|
| 190 |
**Reading.** From a small team's from-scratch 3B, IFEval **62.7** lands right on the original ChatGPT
|
| 191 |
(GPT-3.5-turbo, a reported 60.7) and within about three points of Qwen2.5-3B-Instruct (65.4), while being
|
| 192 |
+
**15.0 points above our own SFT base** (47.7). On tool-calling, KOS-V4 works (official BFCL 72.75 / 73.00 / 60.50)
|
| 193 |
+
but clearly **trails the modern small instruct models**: both Qwen2.5-3B and Llama-3.2-3B score around 90 or above on
|
| 194 |
+
simple and multiple, and far higher on parallel. Function-calling is a real capability here, one the original ChatGPT
|
| 195 |
+
did not have at launch, but it is **not a strength relative to today's small models**, which were trained on far more
|
| 196 |
+
tool-use data. The same holds on IFEval, where newer, more heavily distilled small models score higher; KOS-V4's
|
| 197 |
+
results are notable for its data and compute budget, not a claim to lead the current frontier.
|
| 198 |
|
| 199 |
## Compute footprint & requirements
|
| 200 |
### Training
|