Update card: blockwise int4 + GSM8K parity table

README.md

@@ -14,7 +14,7 @@ pipeline_tag: text-generation
 library_name: litert-lm
 ---
 
-# Falcon3-3B-Instruct — LiteRT-LM (mixed int4)
+# Falcon3-3B-Instruct — LiteRT-LM (blockwise int4)
 
 [tiiuae/Falcon3-3B-Instruct](https://huggingface.co/tiiuae/Falcon3-3B-Instruct)
 converted to the **LiteRT-LM** (`.litertlm`) format for on-device inference with
@@ -25,12 +25,12 @@ Text-only conversion (the Falcon3 decoder; no vision/audio towers).
 
 | | |
 |---|---|
-| **File** | `model.litertlm` (~1.7 GB) |
-| **Quantization** | mixed int4 — weights INT4 (channelwise, symmetric), embeddings INT8 |
+| **File** | `model.litertlm` (~1.86 GB) |
+| **Quantization** | int4 weights — **blockwise (block 32)**, symmetric; embeddings INT8 |
 | **Compute** | integer |
+| **Context (KV cache)** | 4096 |
 | **Base model** | tiiuae/Falcon3-3B-Instruct |
-| **Decode speed** | ~144 tok/s (Apple Silicon, LiteRT-LM) · ~33 tok/s cold on iPhone 17 Pro GPU |
-| **Context** | 32K |
+| **Decode speed** | ~101 tok/s (Apple Silicon, CPU, LiteRT-LM, greedy) |
 
 ## Usage
 
@@ -48,38 +48,31 @@ The `.litertlm` bundle carries the tokenizer and the prompt template (Falcon3's
 native `<|user|>` / `<|assistant|>` format, stop token `<|endoftext|>`), so no
 separate tokenizer files are needed.
 
-## Quality
+## Quality — GSM8K parity
 
-Scored with an 8-question local quality gate (correctness + degeneracy, the same
-checks as the iOS LLM benchmark), each asked separately on Apple Silicon:
+Measured on GSM8K (n=100, greedy, 0-shot chain-of-thought asking for `#### <n>`,
+identical prompt and answer-extraction for every row). The 4-bit MLX build is the
+known-good 4-bit control:
 
-| Check | Result |
+| Configuration | GSM8K |
 |---|---|
-| 17 + 25 = 42 | ✅ |
-| capital of Japan = Tokyo | ✅ |
-| opposite of "hot" = cold | ✅ |
-| days in a week = 7 | ✅ |
-| "thank you" in French = merci | ✅ |
-| 8 × 7 = 56 | ✅ |
-| larger of 0.9 / 0.11 | ✅ |
-| "roses are red, violets are…" = blue | ✅ |
+| bf16 (reference) | 75% |
+| MLX 4-bit (control) | 76% |
+| **This model — LiteRT int4** | **79%** |
 
-**8 / 8 correct, non-degenerate**, and it terminates cleanly at `<|endoftext|>`.
-This is a direct-answering instruct model (no `<think>` block). As expected for a
-3B model at int4, long free-form explanations can show occasional minor artifacts
-(a stray word or brief language mix); short-form answers and reasoning are clean.
-
-## On-device
-
-Validated on **iPhone 17 Pro** via the LiteRT-LM Swift sample app: loads in ~5 s,
-generates coherent text, ~33 tok/s decode (cold) on the GPU backend.
+LiteRT int4 is fully at parity — it matches or slightly exceeds both the 4-bit
+control and bf16 here (the small spread is sampling noise at n=100). This is a
+direct-answering instruct model (no `<think>` block) and terminates cleanly at
+`<|endoftext|>`.
 
 ## Conversion
 
 Converted with [`litert-torch`](https://github.com/google-ai-edge/litert) using a
-mixed-int4 recipe (INT4 weights channelwise + INT8 embeddings) and Falcon3's
-native chat template. Falcon3-3B is a standard `LlamaForCausalLM` architecture, so
-it rides the existing converter and runtime with no custom code.
+**blockwise int4** recipe (INT4 weights, block size 32, symmetric) with embeddings
+kept at INT8, KV cache 4096, and Falcon3's native chat template. Falcon3-3B is a
+standard `LlamaForCausalLM` architecture, so it rides the existing converter and
+runtime with no custom code. Blockwise (not channelwise) int4 is what preserves
+reasoning accuracy.
 
 ## License