codybum commited on
Commit
792be18
·
verified ·
1 Parent(s): 2d5aa65

Commercial IFEval AVG4 baselines: tiered Verified/Reported/Estimate list with citations

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -227,7 +227,7 @@ than KOS-V4 are excluded. Calibration: our harness measures Qwen2.5-3B at AVG4 6
227
  | **KOS-V4-Instruct (ours)** | **Univ. of Kentucky** | **2025** | **3.0B** | **62.7** |
228
  | Yi-1.5-9B-Chat | 01.AI | May 2024 | 8.8B | 62.7 |
229
  | OpenChat-3.5-1210 | OpenChat | Dec 2023 | 7.2B | 62.2 |
230
- | *GPT-3.5-turbo* (commercial ref) | OpenAI | 2023 | undisclosed | 60.7 |
231
  | Phi-3.5-mini-instruct | Microsoft | Aug 2024 | 3.8B | 60.5 |
232
  | Mistral-7B-Instruct-v0.2 | Mistral AI | Dec 2023 | 7.2B | 57.5 |
233
  | Phi-3-mini-4k-instruct | Microsoft | Apr 2024 | 3.8B | 57.1 |
@@ -239,6 +239,10 @@ than KOS-V4 are excluded. Calibration: our harness measures Qwen2.5-3B at AVG4 6
239
  | Qwen1.5-7B-Chat | Alibaba | Jan 2024 | 7.7B | 46.1 |
240
  | Qwen1.5-4B-Chat | Alibaba | Jan 2024 | 4.0B | 33.2 |
241
 
 
 
 
 
242
  **Reading.** At **3B**, KOS-V4 (62.7) **edges the original commercial ChatGPT (GPT-3.5-turbo, 60.7)**, ties the 8.8B
243
  Yi-1.5, and outscores every 7B-14B open instruct model below it, including both Mistral-7B-Instruct releases,
244
  **Meta's Llama-3.1-8B**, Llama-2-13b-chat, gemma-1.1-7b-it, Zephyr-7B and Qwen1.5-7B. **The newest major model it
 
227
  | **KOS-V4-Instruct (ours)** | **Univ. of Kentucky** | **2025** | **3.0B** | **62.7** |
228
  | Yi-1.5-9B-Chat | 01.AI | May 2024 | 8.8B | 62.7 |
229
  | OpenChat-3.5-1210 | OpenChat | Dec 2023 | 7.2B | 62.2 |
230
+ | *GPT-3.5-turbo* (commercial ref) | OpenAI | 2023 | 20B (est.) | 60.7 |
231
  | Phi-3.5-mini-instruct | Microsoft | Aug 2024 | 3.8B | 60.5 |
232
  | Mistral-7B-Instruct-v0.2 | Mistral AI | Dec 2023 | 7.2B | 57.5 |
233
  | Phi-3-mini-4k-instruct | Microsoft | Apr 2024 | 3.8B | 57.1 |
 
239
  | Qwen1.5-7B-Chat | Alibaba | Jan 2024 | 7.7B | 46.1 |
240
  | Qwen1.5-4B-Chat | Alibaba | Jan 2024 | 4.0B | 33.2 |
241
 
242
+ **Note on GPT-3.5-turbo size.** The 20B parameter figure is an **estimate**, not an official number. OpenAI has never
243
+ disclosed the parameter count of GPT-3.5-turbo; the roughly 20B consensus comes from leaked research documents and
244
+ industry estimates.
245
+
246
  **Reading.** At **3B**, KOS-V4 (62.7) **edges the original commercial ChatGPT (GPT-3.5-turbo, 60.7)**, ties the 8.8B
247
  Yi-1.5, and outscores every 7B-14B open instruct model below it, including both Mistral-7B-Instruct releases,
248
  **Meta's Llama-3.1-8B**, Llama-2-13b-chat, gemma-1.1-7b-it, Zephyr-7B and Qwen1.5-7B. **The newest major model it