README.md

---
license: other
license_name: webai-non-commercial-license-ver.-1.0
license_link: LICENSE.md
base_model: webAI-Official/TwIL-LM3
tags:
  - litert
  - litert-lm
  - litertlm
  - on-device
  - edge
pipeline_tag: text-generation
library_name: litert-lm
---

# TwIL-LM3 — LiteRT-LM

[webAI-Official/TwIL-LM3](https://huggingface.co/webAI-Official/TwIL-LM3) converted to the **LiteRT-LM** (`.litertlm`) format for on-device inference with Google's [LiteRT-LM](https://github.com/google-ai-edge/litert-lm) runtime. **Requires litert-lm ≥ 0.16.**

**⚠ Non-commercial.** The upstream model ships under the webAI Non-Commercial License v1.0, and this conversion inherits it unchanged: **non-commercial research and educational use only**. The full license text is in `LICENSE.md` in this repository — §3.1 requires that any redistribution carries it, so keep it with the file.

| File | Recipe | Size |
|---|---|---|
| `TwIL-LM3_int4.litertlm` | int4 blockwise-32 + OCTAV on linears, int8 embedding (externalised) | 2.00 GB |

## Correctness

**8-question sanity gate: 8/8 on every backend measured** — Mac GPU, Mac CPU, and on an iPhone 17 Pro on both Metal and CPU. Non-degenerate in all four. A 3B model answering all eight on a phone at the bundle's own 4096-token context, with no ladder reduction, is the useful part.

Graded with `--max-tokens 2048`: this is a reasoning model and emits a `<think>` block before answering, so a smaller budget scores it as wrong when it has simply not finished thinking.

### The formal-logic claim: measured against the base, and no difference detected

Upstream describes TwIL-LM3 as specialised for formal logic. The eight-question gate is general knowledge and cannot see that claim in either direction, so it was tested separately: **12 items with one defensible answer each** (syllogisms, modus tollens, affirming the consequent, contrapositive, De Morgan, quantifier negation, disjunctive syllogism, transitivity, necessary vs sufficient, biconditional, vacuous truth), run against **this model and its base [SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B) on identical items, identical flags, identical runtime, and identically-converted bundles** (same rail, same recipe, both 2.00 GB, both duplication-free).

| | TwIL-LM3 | SmolLM3-3B (base) |
|---|---|---|
| score | 10 / 12 | 9 / 12 |

**Read this as "no difference detected", not as a win.** The two models disagree on only **3 of the 12 items**, splitting 2–1 — TwIL-LM3 is right on the valid syllogism and the contrapositive where the base is wrong, the base is right on De Morgan where TwIL-LM3 is wrong, and both fail the vacuous-truth item. A 2–1 split among three discordant items is indistinguishable from chance, and a 12-item instrument can only detect large effects in any case. The honest summary is that **this test was not sensitive enough to confirm or refute the specialisation**, and the one-point total should not be quoted as evidence for it.

The harness is `scripts/eval_logic.py` in the [converter repo](https://github.com/john-rocky/hf-to-litertlm), so the comparison can be repeated or extended.

## Performance

`litert-lm benchmark` (litert-lm 0.16.0), Apple M4 Max, `-p 256 -d 256 --cache no`, quiet machine, serialized, measured twice (run-to-run spread under 1%):

| Backend | Prefill (256) | Decode | TTFT | Init |
|---|---|---|---|---|
| **GPU (Metal)** | **1341 tok/s** | **93.0 tok/s** | 0.206 s | 2.50 s |
| CPU | 141.5 tok/s | 25.2 tok/s | 2.23 s | 1.97 s |

iPhone 17 Pro, same file, own harness driving the LiteRT-LM Swift API:

| Backend | 8Q | Context | Init |
|---|---|---|---|
| **GPU (Metal)** | **8/8** | bundle default (4096) | 6.34 s |
| CPU | **8/8** | bundle default (4096) | 3.52 s |

The phone harness measures correctness and load time only — **it does not measure throughput, so no phone tok/s figure is quoted here** rather than one being estimated from the desktop.

## Usage

```bash
litert-lm run ./TwIL-LM3_int4.litertlm --prompt "If it is raining then the ground is wet. The ground is not wet. Is it raining?"

# GPU
litert-lm run ./TwIL-LM3_int4.litertlm --backend gpu --cache no --prompt "..."
```

The bundle carries the tokenizer and the SmolLM3 chat template with its thinking controls (`/think`, `/no_think`) — not plain ChatML. A 4096-token KV budget.

## Conversion notes

Converted with [`litert-torch`](https://github.com/google-ai-edge/litert-torch) 0.9.3 / litert-converter 0.3.1 / ai-edge-quantizer 0.8.0 / litert-lm-builder 0.16.0 — a pristine released stack, no patched checkout.

- **Same rail as SmolLM3-3B, and the file size proves it.** TwIL-LM3 is a SmolLM3-3B finetune, so it converts with the identical one-command recipe. The result is **2,002,241,456 bytes** against the shipped SmolLM3-3B's 2,002,257,840 — a 16 KB difference on the same architecture and recipe, which is the strongest single check that the rail applied correctly.
- **`EXTERNALIZE_EMBEDDER=1` is required, not cosmetic.** The model ties its embedding and `lm_head`, so a recipe asking for int4 linears and an int8 embedder describes one tensor two ways; the quantizer resolves it by copying the 128256-row table once per prefill signature. Externalising the table removes the conflict. Verified duplication-free after export (0.65 bytes/param).
- **No `start_token` in the bundle**, which is correct here: upstream sets `bos_token: None`. A bundle that prepends a BOS the model was never trained to see is a quiet quality killer, so it is worth confirming rather than assuming.
- **Quantization**: int4 blockwise-32 with OCTAV clipping on the linears, int8 on the embedding.

## License and changes

Distributed under the **webAI Non-Commercial License v1.0**, inherited unchanged from the base model; the complete license text is included as `LICENSE.md` as its §3.1 requires. **Use is limited to non-commercial research and educational purposes (§3.3).** 

**Changes from the original work:** weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged into the `.litertlm` bundle. No weights were retrained or altered in value beyond quantization.

This repository is a community conversion and is not affiliated with webAI.