---
license: apache-2.0
base_model:
  - Qwen/Qwen3-0.6B
  - HuggingFaceTB/SmolLM2-135M-Instruct
  - google/gemma-3-1b-it
tags:
  - razor
  - on-device
  - quantized
  - int8
---

# Razor weights

int8 weights for the Razor inference engine, which runs language models on
phone and laptop CPUs. These files are a flat binary that Razor memory-maps
directly; they are not loadable by transformers or llama.cpp.

| File | Base model | Size | Notes |
| --- | --- | --- | --- |
| `razor_gemma3_1b_int8.bin` | Gemma 3 1B IT | 1003 MB | Sliding-window attention, tied int8 embedding |
| `razor_qwen3_06b_int8.bin` | Qwen3 0.6B | 598 MB | Tied int8 embedding |
| `razor_smollm2_135m_int8.bin` | SmolLM2 135M Instruct | 249 MB | fp32 embedding, int8 tied head |

All three store linear weights `[out, in]` and carry the layout marker below.

Tokenizers are not included. Use the tokenizer.json from the base model.

## Accuracy

Each file was checked against its fp32 HuggingFace reference on five prompts.
The argmax token matched on all five for every model, and the logits over the
reference's top 50 tokens correlate as follows.

| Model | top-1 | top-50 overlap | logit correlation |
| --- | --- | --- | --- |
| Gemma 3 1B | 5/5 | 0.84 | 0.950 |
| Qwen3 0.6B | 5/5 | 0.88 | 0.976 |

Gemma was additionally checked at 354 and 1064 prompt tokens, either side of
its 512-token attention window, since a window bug is invisible on the short
prompts a parity suite usually uses.

## Layout marker

Files exported after 2026-08-13 contain a `__layout_out_in` tensor, which tells
Razor the linear weights are stored `[out, in]`. Files without it hold
`[in, out]` and are transposed at load. The marker exists because the layout
cannot be inferred from the dimensions: on Qwen3 the K and V projections are
square, so both layouts look identical and a wrong guess produces fluent
nonsense rather than an error.

## Licence

Apache-2.0, inherited from the base models. Qwen3 is by Alibaba Cloud,
SmolLM2 is by Hugging Face, and Gemma 3 is by Google and additionally subject
to the Gemma Terms of Use.