KartiOS commited on
Commit
03884af
·
1 Parent(s): 9b981c1

Pin and measure v0, the reference baseline

Browse files

v0 is the starting line: the pinned base at BF16, measured on our own hardware
so every later version has an exact thing to beat. Records throughput, the
vision check, the tool-call parse and both canaries.

The throughput is the notable result - at BF16 this 4B runs at about the same
single-stream rate as the 27B NVFP4 on the same box, which is a memory-bandwidth
finding and the case for quantizing the deployment lane.

Files changed (2) hide show
  1. README.md +45 -5
  2. recipe.json +34 -1
README.md CHANGED
@@ -17,9 +17,9 @@ tags:
17
 
18
  # Karti-Small-VL-4B
19
 
20
- **Status: in training. No checkpoint exists yet.** This page is being published
21
- from the scaffolding stage on purpose, and it will be updated as runs happen —
22
- including the ones that fail.
23
 
24
  Karti-Small-VL-4B is Lumbridge's small **vision + tool-calling** model: a 4B
25
  model meant to run locally and do the two things a household agent actually
@@ -151,12 +151,52 @@ Karti-Small-VL-4B-2026.08.29-r007-a1b2c3d
151
 
152
  A **release** is a checkpoint the promotion gate accepted, and only then does it
153
  get a short name — `Karti-Small-VL-4B-v1`. Versions are immutable and are never
154
- reassigned, so a version number means something passed a gate rather than merely
155
- finished training.
 
 
 
 
 
156
 
157
  This replaces a week-anchored scheme that could not express iterating more than
158
  once a week without inventing a date.
159
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
160
  ## What it is being trained to do
161
 
162
  The tool surface it inherits from the 3B program, now with images in the
 
17
 
18
  # Karti-Small-VL-4B
19
 
20
+ **Status: v0 baseline pinned and measured. Training not yet started.** This page
21
+ is published from the scaffolding stage on purpose, and it will be updated as
22
+ runs happen — including the ones that fail.
23
 
24
  Karti-Small-VL-4B is Lumbridge's small **vision + tool-calling** model: a 4B
25
  model meant to run locally and do the two things a household agent actually
 
151
 
152
  A **release** is a checkpoint the promotion gate accepted, and only then does it
153
  get a short name — `Karti-Small-VL-4B-v1`. Versions are immutable and are never
154
+ reassigned, so from v1 onward a version number means something passed a gate
155
+ rather than merely finished training.
156
+
157
+ `v0` is the exception that makes the rule usable: it is the **reference
158
+ baseline** — the pinned base, measured on our own hardware, before any training.
159
+ It is what every later version has to beat, so it is numbered rather than left
160
+ floating.
161
 
162
  This replaces a week-anchored scheme that could not express iterating more than
163
  once a week without inventing a date.
164
 
165
+ ## Versions
166
+
167
+ ### `Karti-Small-VL-4B-v0` — reference baseline
168
+
169
+ The starting line, pinned and measured so later versions have something exact to
170
+ be compared against.
171
+
172
+ | | |
173
+ |---|---|
174
+ | weights | [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) @ `851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a` |
175
+ | precision | BF16 |
176
+ | parameters | 4,659,865,088 |
177
+ | training applied | none — this is the baseline |
178
+ | serving alias | `karti-vl` · `karti-vl-4b-base` |
179
+
180
+ Measured on a DGX Spark (GB10, 121.7 GB unified), vLLM 0.27.1, BF16,
181
+ `max_model_len` 32768, KV cache FP8, temperature 0, single stream:
182
+
183
+ | check | result |
184
+ |---|---|
185
+ | generation, 128 tokens | **18.24 tok/s** |
186
+ | generation, 512 tokens | **17.74 tok/s** |
187
+ | vision — shapes and colours off a synthetic image | correct |
188
+ | tool call via `qwen3_xml` parser | parsed — `cron(job="driveway-clear")` |
189
+ | arithmetic canary `19 × 23` | 437 |
190
+ | empty-content canary at `max_tokens: 80` | non-empty, `finish_reason: stop` |
191
+
192
+ The throughput number is the interesting one: at BF16 this 4B runs at
193
+ approximately the same single-stream rate as the 27B NVFP4 brain on the same
194
+ box. That is a memory-bandwidth result, not a capability one, and it is the
195
+ argument for quantizing the deployment lane rather than shipping BF16.
196
+
197
+ Weights are not mirrored here — v0 is byte-identical to the pinned Qwen revision
198
+ above, so that revision is the artifact and this table is its measurement.
199
+
200
  ## What it is being trained to do
201
 
202
  The tool surface it inherits from the 3B program, now with images in the
recipe.json CHANGED
@@ -2,7 +2,7 @@
2
  "schema": "karti.model-recipe/v2",
3
  "model_name": "Karti-Small-VL-4B",
4
  "serving_alias": "karti-vl",
5
- "status": "scaffolding-complete-no-run-executed",
6
  "weights_published": false,
7
  "private_training_rows_published": false,
8
  "base_model": {
@@ -95,5 +95,38 @@
95
  "mitigation": "enable_thinking:false pinned for every run and serving config",
96
  "required_canary": "a tight-max_tokens request must return non-empty content before any result from this model is believed"
97
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
98
  }
99
  }
 
2
  "schema": "karti.model-recipe/v2",
3
  "model_name": "Karti-Small-VL-4B",
4
  "serving_alias": "karti-vl",
5
+ "status": "v0-baseline-pinned-and-measured-no-training-run-executed",
6
  "weights_published": false,
7
  "private_training_rows_published": false,
8
  "base_model": {
 
95
  "mitigation": "enable_thinking:false pinned for every run and serving config",
96
  "required_canary": "a tight-max_tokens request must return non-empty content before any result from this model is believed"
97
  }
98
+ },
99
+ "releases": {
100
+ "v0": {
101
+ "role": "reference baseline; the pinned base measured on our hardware before any training",
102
+ "weights_repository": "Qwen/Qwen3.5-4B",
103
+ "weights_revision": "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a",
104
+ "weights_mirrored_here": false,
105
+ "weights_mirror_note": "byte-identical to the pinned upstream revision; that revision is the artifact",
106
+ "precision": "BF16",
107
+ "training_applied": "none",
108
+ "gate_passed": "n/a - a baseline is not gated; v1 onward are",
109
+ "measured_on": "DGX Spark GB10, 121.7 GB unified, vLLM 0.27.1, max_model_len 32768, kv-cache fp8, temperature 0, single stream",
110
+ "measured_2026_08_29": {
111
+ "tok_per_s_128": 18.24,
112
+ "tok_per_s_512": 17.74,
113
+ "vision": "correct on synthetic shape/colour image",
114
+ "tool_call": "parsed via qwen3_xml",
115
+ "canary_19x23": 437,
116
+ "canary_empty_content_at_max_tokens_80": "non-empty, finish_reason=stop"
117
+ },
118
+ "observation": "at BF16 this 4B runs at roughly the same single-stream rate as the 27B NVFP4 on the same box - a memory-bandwidth result, and the argument for quantizing the deployment lane"
119
+ }
120
+ },
121
+ "serving": {
122
+ "aliases": [
123
+ "karti-vl",
124
+ "karti-vl-4b-base"
125
+ ],
126
+ "port": 8002,
127
+ "runtime": "vLLM 0.27.1",
128
+ "tool_call_parser": "qwen3_xml",
129
+ "chat_template": "the model's own; never overridden - train/serve parity depends on it",
130
+ "enable_thinking": false
131
  }
132
  }