File size: 18,505 Bytes
7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 7a703c9 04996a1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 | ---
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
base_model: thinkingmachines/Inkling-2-Mini
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
- strata
- inkling-2-mini
- moe
- quantization
- looped-transformer
- dspark
---
<p>
<a href="https://github.com/Niko1221/Strata"><img class="dark:hidden" src="./assets/masthead-light.svg" width="100%" alt="Strata on GitHub"><img class="hidden dark:block" src="./assets/masthead-dark.svg" width="100%" alt="Strata on GitHub"></a>
</p>
<h1 align="center">Inkling 2 Mini for Strata</h1>
<p align="center"><b>Run Thinking Machines Lab's Inkling 2 Mini on your own computer.</b> A 4 GiB graphics card is enough to start, with packs made for 12, 16 and 24 GiB cards. Free and open source. No account needed.</p>
<!-- Demo slot. The Hy4-Flash card shows a GIF here. To add one for Inkling 2 Mini, put the file in assets/ and move the
three lines below out of this comment.
<p align="center">
<img src="./assets/demo.gif" width="560" height="315" alt="[FILL: what the demo shows]">
</p>
-->
<p align="center">
<a href="#pick-your-pack"><b>Pick your pack</b></a> · <a href="#get-started">Get started</a> · <a href="#how-close-is-it-to-the-original">Quality</a> · <a href="#performance-with-strata">Speed</a> · <a href="#support-strata">Support Strata</a>
</p>
<p>
<img class="dark:hidden" src="./assets/hero-light.svg" width="100%" alt="26.4B parameters, 3.4B active per token. A 4 GiB graphics card is enough to start. Up to 1M tokens of context, with text and images in.">
<img class="hidden dark:block" src="./assets/hero-dark.svg" width="100%" alt="26.4B parameters, 3.4B active per token. A 4 GiB graphics card is enough to start. Up to 1M tokens of context, with text and images in.">
</p>
**Inkling 2 Mini** is Thinking Machines Lab's open-weights model made to run on local hardware. It is a looped transformer: the middle layers run three times for every token. It has 26.4B parameters and uses 3.4B per token (5.4B when the repeated passes are counted). It reads text and images, writes text, and supports up to 1M tokens of context (1,048,576).
## Pick your pack
**Pick by your graphics card's memory.** Each pack has a recommended card size. Smaller cards work too, from 4 GiB.
<p>
<img class="dark:hidden" src="./assets/packs-light.svg" width="100%" alt="Three packs. Recommended GPU 12 GiB: SQ-10, a 10.47 GB download, matches the original 92.35% of the time, the smallest download. Recommended GPU 16 GiB: SQ-15, a 14.35 GB download, matches the original 96.68% of the time. Recommended GPU 24 GiB: SQ-20, a 19.35 GB download, matches the original 100% of the time, identical to the original.">
<img class="hidden dark:block" src="./assets/packs-dark.svg" width="100%" alt="Three packs. Recommended GPU 12 GiB: SQ-10, a 10.47 GB download, matches the original 92.35% of the time, the smallest download. Recommended GPU 16 GiB: SQ-15, a 14.35 GB download, matches the original 96.68% of the time. Recommended GPU 24 GiB: SQ-20, a 19.35 GB download, matches the original 100% of the time, identical to the original.">
</p>
| Recommended GPU | Pack file | Download |
| ---: | --- | ---: |
| **12 GiB** | `Inkling-2-Mini-SQ-10.strata` | 10.47 GB |
| **16 GiB** | `Inkling-2-Mini-SQ-15.strata` | 14.35 GB |
| **24 GiB** | `Inkling-2-Mini-SQ-20.strata` | 19.35 GB |
"Recommended GPU" is the graphics card memory the pack is made for. The number in a pack's name is its approximate size in GB, not the card it needs. To read images, add `Inkling-2-Mini-Vision.strata` (0.64 GB). [All files](#files) are listed below.
- **A 4 GiB graphics card is enough to start.** Strata keeps the part of the model that does not fit on the card in system RAM. This works because Inkling 2 Mini is a Mixture-of-Experts model: it uses only 3.4B of its 26.4B parameters for each token.
- **12 GiB?** SQ-10 fits entirely in the card's own memory.
- **16 GiB?** SQ-15 fits entirely in the card's own memory, and picks the same next word as the original 96.68% of the time.
- **24 GiB?** SQ-20 matched the original exactly in every test.
## Get started
1. **Install Strata.** [Download the app for Windows, Linux or Mac](https://github.com/Niko1221/Strata/releases/latest). It needs no account.
2. **Choose Inkling 2 Mini** and the pack that matches your graphics card.
3. **Start chatting.**
<details>
<summary><b>Prefer the terminal?</b></summary>
Linux or Mac:
```sh
curl -fsSL --connect-timeout 15 --max-time 120 https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
```
Windows PowerShell:
```powershell
irm -TimeoutSec 120 https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
```
On Windows, open a new terminal afterwards. Then run `strata launch` and follow the prompts.
</details>
<!-- VERIFY before publishing: the Hy4-Flash card also gives the direct command (strata launch, then the family and model options). No Inkling family name was in the public Strata repository on 22 November 2026, so only the plain command is given above. Add the direct command to the last sentence once the names are final. -->
## How close is it to the original?
**SQ-20 is identical to the original model in every test.** SQ-15 and SQ-10 give up a little closeness to fit on smaller cards.
<p>
<img class="dark:hidden" src="./assets/closeness-light.svg" width="100%" alt="How often each pack picks the same next word as the original model, higher is better. Test prompts: SQ-20 100%, SQ-15 96.68%, SQ-10 92.35%. WikiText-2: SQ-20 100%, SQ-15 95.97%, SQ-10 89.79%. SQ-20 is identical to the original on both.">
<img class="hidden dark:block" src="./assets/closeness-dark.svg" width="100%" alt="How often each pack picks the same next word as the original model, higher is better. Test prompts: SQ-20 100%, SQ-15 96.68%, SQ-10 92.35%. WikiText-2: SQ-20 100%, SQ-15 95.97%, SQ-10 89.79%. SQ-20 is identical to the original on both.">
</p>
**How to read this.** Each pack's next-word predictions are compared with the original model's, on a set of test prompts and on WikiText-2. "Same word" is how often the pack's first choice for the next word is the same as the original's, so higher is better. "Drift" is the average gap between the two sets of predictions, called KL divergence. Zero means identical, so lower is better.
| Pack | Same word, test | Same word, WikiText-2 | Drift, test | Drift, WikiText-2 |
| --- | ---: | ---: | ---: | ---: |
| **SQ-20** | **100.00%** | **100.00%** | **0.00000** | **0.00000** |
| SQ-15 | 96.68% | 95.97% | 0.00889 | 0.01071 |
| SQ-10 | 92.35% | 89.79% | 0.05017 | 0.07199 |
<details>
<summary>How this was measured</summary>
The reference is the original model with its own weights. The test prompts were held out from tuning, and are also scored in two parts: short prompts and long prompts of 2K tokens. "Dev" is the set of development prompts. WikiText-2 was run at 2K tokens of context, where the original scores a perplexity of 6.141 (lower is better).
| Pack | Drift, dev | Drift, short test | Drift, 2K test | WikiText-2 perplexity | DSpark AR α |
| --- | ---: | ---: | ---: | ---: | ---: |
| **SQ-20** | 0.00000 | 0.00000 | 0.00000 | 6.141 (+0.000) | 2.701 |
| SQ-15 | 0.00880 | 0.00968 | 0.00575 | 6.177 (+0.036) | 2.583 |
| SQ-10 | 0.04340 | 0.05645 | 0.02508 | 6.461 (+0.320) | 2.701 |
Every pack has a DSpark draft model built in. The last column is its AR α. The part that reads images, `Inkling-2-Mini-Vision.strata`, is stored without loss in BF16: 422 tensors, 318.3M parameters.
</details>
<!-- VERIFY before publishing: the measurement paragraph above says only what the results table shows. Add how many prompts and tokens each set has, as the Hy4-Flash card does, and check the sentence "The test prompts were held out from tuning": it is inferred from the names of the Dev and Test columns. -->
## Performance with Strata
**Up to 250 tokens a second for one request, and over 4,000 in total with 64 requests at once.** Both on an RTX 5090. Speeds measured with Strata on 20 machines, from integrated graphics and Macs to an RTX 5090.
<p>
<img class="dark:hidden" src="./assets/speed-light.svg" width="100%" alt="Output speed with Strata for one request, in tokens per second, with prompt speed beside it, fastest first. RTX 5090: 176.5 to 250.3, prompt 11,306. RTX PRO 5000: 154.4 to 214.5, prompt 6,840+. RTX 5070 Ti: 54.2 to 155.5, prompt 4,237. 2x RTX PRO 6000: 131.6, prompt 7,980. Olares One: 80.5 to 115+, prompt 4,367. Apple M5 Max: 100.0 to 114.5, prompt 3,063. Radeon AI PRO R9700: 99.0 to 106.5, prompt 3,982. RTX 5070: 93.9 to 101.4, prompt 2,216. DGX Spark: 46.1 to 74.3, prompt 2,197. Tesla V100: 70.5, prompt 2,287. 2x RTX A4000: 68.4, prompt 1,890+. Radeon RX 7900 XT: 54.7+, prompt 2,685. Strix Halo 128 GB: 47.3 to 48.0, prompt 1,466. RTX 2080 Ti: 42.8, prompt 1,304. Tesla P100: 35.0, prompt 223. Apple M4 Mac mini: 27.6, prompt 312.3. GTX 1080 Ti: 24.6, prompt 244. Radeon RX 6700 XT: 17.8, prompt 68.5. Radeon 780M: 11.1, prompt 344.2. Intel Arc A750: 6.1, prompt 67.9. A plus sign means at least that much.">
<img class="hidden dark:block" src="./assets/speed-dark.svg" width="100%" alt="Output speed with Strata for one request, in tokens per second, with prompt speed beside it, fastest first. RTX 5090: 176.5 to 250.3, prompt 11,306. RTX PRO 5000: 154.4 to 214.5, prompt 6,840+. RTX 5070 Ti: 54.2 to 155.5, prompt 4,237. 2x RTX PRO 6000: 131.6, prompt 7,980. Olares One: 80.5 to 115+, prompt 4,367. Apple M5 Max: 100.0 to 114.5, prompt 3,063. Radeon AI PRO R9700: 99.0 to 106.5, prompt 3,982. RTX 5070: 93.9 to 101.4, prompt 2,216. DGX Spark: 46.1 to 74.3, prompt 2,197. Tesla V100: 70.5, prompt 2,287. 2x RTX A4000: 68.4, prompt 1,890+. Radeon RX 7900 XT: 54.7+, prompt 2,685. Strix Halo 128 GB: 47.3 to 48.0, prompt 1,466. RTX 2080 Ti: 42.8, prompt 1,304. Tesla P100: 35.0, prompt 223. Apple M4 Mac mini: 27.6, prompt 312.3. GTX 1080 Ti: 24.6, prompt 244. Radeon RX 6700 XT: 17.8, prompt 68.5. Radeon 780M: 11.1, prompt 344.2. Intel Arc A750: 6.1, prompt 67.9. A plus sign means at least that much.">
</p>
Each bar is one machine, from its lowest to its highest measured speed across packs and kinds of prompt. The RTX 5090's top figure is with SQ-10. With SQ-20 it writes 176.5 to 198.9 tokens a second. Prompt speed is the highest measured on that machine.
<details>
<summary>How speeds were measured, with every figure</summary>
Measured with thinking off. All speeds are in tokens per second.
- **Prompt speed** is how fast the input is read.
- **One request** is how fast the answer is written when one request runs alone, with Strata's built-in draft decoding on (W=4 Beam / DSpark) unless a figure is marked plain.
- **Many at once** is the total written across that many requests running together.
- **Hybrid** means part of the model stays in system RAM because the pack does not fit in the card's memory.
- Where a cell has several figures separated by a slash, they follow the order of the packs in that row. "Code", "math" and "chat" are kinds of prompt. A plus sign means at least that much.
| Machine | Pack | Prompt speed | One request | Many at once |
| --- | --- | ---: | ---: | ---: |
| RTX 5090 32 GB | SQ-20 | 11,306 (8,722 at 2K) | 198.9 code, 176.5 math | 2,094.1 with 32, 4,089.7 with 64 |
| RTX 5090 32 GB | SQ-10 | 10,452 (7,645 at 2K) | 250.3 code, 238.7 math | 1,492.9 with 32, 3,010.4 with 64 |
| 2x RTX PRO 6000 96 GB | SQ-20 on two cards | 7,980 | 131.6 | 2,081.1 with 32 |
| RTX PRO 5000 72 GB | SQ-10 | 6,840+ | 192.2 code, 214.5 math, 154.4 chat | 1,850+ with 64 |
| Olares One (RTX 5090 Laptop 24 GB) | SQ-20 | 4,367 (3,951 at 2K) | 80.5 plain, 115+ with Beam | 502.0 with 32 |
| RTX 5070 Ti 16 GB | SQ-15 / SQ-10 / SQ-20 hybrid | 4,237 / 4,136 / 2,205 | 88.0 / 155.5 / 54.2 | 980+ with 32 |
| RTX 5070 12 GB (Windows 11) | SQ-10 | 2,216 | 93.9 code, 101.4 math | 120.0 with 64, hybrid |
| DGX Spark GB10 128 GB | SQ-20 / SQ-10 | 2,197 / 2,070 | 46.1 / 74.3 | 1,145.1 / 805.8 with 64 |
| 2x RTX A4000 16 GB | Official safetensors and GGUF files, on two cards | 1,890+ | 68.4 | 410+ with 32 |
| Tesla V100 32 GB | SQ-20 / SQ-10 | 2,287 / 2,265 | 70.5 | 990.1 / 676.0 with 64 |
| RTX 2080 Ti 11 GB | SQ-10 hybrid | 1,304 | 42.8 | 145+ with 16 |
| GTX 1080 Ti 11 GB | SQ-10 hybrid | 244 | 24.6 | 78+ with 8 |
| Tesla P100 16 GB | SQ-10 | 223 | 35.0 | 136.9 with 32 |
| Radeon AI PRO R9700 32 GB | SQ-10 | 3,982 (4,308 with 64 at once) | 99.0 code, 106.5 math | 291.5 with 8, 587.8 with 64 |
| Radeon RX 7900 XT 20 GB (over OCuLink) | SQ-10 | 2,685 (2,873 with 8 at once) | 54.7+, about 78+ with Beam | 352.3 with 64 |
| Ryzen AI Max+ Strix Halo 128 GB | SQ-20 / SQ-10 | 1,466 (1,654 with 64 at once) | 47.3 / 48.0 | 172.8 with 8, 374.9 with 64 |
| Radeon 780M integrated graphics (60 GB shared memory) | SQ-10 | 344.2 | 11.1 | 51.1 with 32 |
| Radeon RX 6700 XT 12 GB | SQ-10 hybrid | 68.5 | 17.8 | 33.4 with 4 |
| Apple M5 Max 128 GB | SQ-20 / SQ-15 / SQ-10 | 3,063 / 3,023 / 2,997 | 108.9 / 100.0 / 114.5 | 597.2 / 542.5 with 64 |
| Apple M4 Mac mini 32 GB | SQ-10 | 312.3 | 27.6 code | 73.1 with 32 |
| Intel Arc A750 8 GB | SQ-10 hybrid | 67.9 | 6.1 | 10.8 with 8 |
Left out of the chart and the headline: two peak readings (222.3 with SQ-20 on the RTX 5090, 28.3 on the M4 Mac mini) and one micro-benchmark reading (306.2 with SQ-10 on the RTX 5090).
</details>
<!-- VERIFY before publishing: (1) Add the Strata version and the context sizes, as the Hy4-Flash card does. (2) Check the reading of "Hybrid" and of "W=4 Beam / DSpark" in the list above, and that the Arc A750 row is right to say hybrid. Note: the supplied speed table's Intel row also has figures from a second card whose name was cut off. They are left out of this card on purpose. -->
## Recommended settings
| Setting | Value |
| --- | --- |
| `temperature` | 1.0 |
This is the default in the original model's own settings file. Two more points from Thinking Machines Lab's model card:
- **Set a limit on output tokens**, and on agent turns if you run agents. The model can get stuck in long reasoning loops.
- **Reasoning cannot be switched off fully.** The lowest reasoning effort still reasons a little.
## Files
| File | Size |
| --- | ---: |
| `Inkling-2-Mini-SQ-10.strata` | 10.47 GB (9.75 GiB) |
| `Inkling-2-Mini-SQ-15.strata` | 14.35 GB (13.36 GiB) |
| `Inkling-2-Mini-SQ-20.strata` | 19.35 GB (18.02 GiB) |
| `Inkling-2-Mini-Vision.strata`, for reading images | 0.64 GB (0.59 GiB) |
<!-- VERIFY before publishing: the Hy4-Flash card lists SHA-256 checksums here. To add them, rename the heading to
"Files and checksums", change the "All files" link in "Pick your pack" to #files-and-checksums, and move the block
below out of this comment with the real values.
<details>
<summary>Pack file checksums (SHA-256)</summary>
```text
[FILL] Inkling-2-Mini-SQ-20.strata
[FILL] Inkling-2-Mini-SQ-15.strata
[FILL] Inkling-2-Mini-SQ-10.strata
[FILL] Inkling-2-Mini-Vision.strata
```
</details>
-->
## Support Strata
<p align="center">
<b>Help keep local AI fast, independent, and open to everyone.</b><br>
<sub>Strata is built independently and released free under the MIT license. Sponsoring on GitHub or buying a coffee directly funds new GPU kernels, model quantizations, and hardware testing.</sub>
</p>
<p align="center">
<a href="https://github.com/sponsors/Niko1221"><img src="./assets/btn-sponsor.svg" alt="Sponsor Strata on GitHub" height="44"></a>
<a href="https://buymeacoffee.com/strataengine"><img src="./assets/btn-coffee.svg" alt="Buy Strata a coffee" height="44"></a>
</p>
## Licence
- **The model:** Inkling 2 Mini is made by Thinking Machines Lab and released under the [Apache 2.0 licence](https://www.apache.org/licenses/LICENSE-2.0), with a [Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/). The files here are builds of [thinkingmachines/Inkling-2-Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) packed for Strata.
- **The Strata engine:** [MIT](https://github.com/Niko1221/Strata/blob/main/LICENSE).
<!-- Source figures, as supplied on 22 November 2026
A record of the results table the card was written from. The chart tool does not read this block: it reads the tables
readers see ("Pick your pack" and "How close is it to the original?").
| Deliverable | Size (GB / GiB) | Target GPU | Dev KLD | Test KLD (Short / 2K Long) | WikiText-2 2K KLD / PPL (Native 6.141) | Top-1 (Test / WT2) | Embedded DSpark AR alpha |
| --- | ---: | --- | ---: | ---: | ---: | ---: | ---: |
| Inkling-2-Mini-SQ-20.strata | 19.347 GB (18.02 GiB) | 24 GiB | 0.00000 | 0.00000 (0.00000 / 0.00000) | 0.00000 / 6.141 (+0.000) | 100.00% / 100.00% | 2.701 |
| Inkling-2-Mini-SQ-15.strata | 14.348 GB (13.36 GiB) | 16 GiB (100% VRAM) | 0.00880 | 0.00889 (0.00968 / 0.00575) | 0.01071 / 6.177 (+0.036) | 96.68% / 95.97% | 2.583 |
| Inkling-2-Mini-SQ-10.strata | 10.472 GB (9.75 GiB) | 12 GiB (100% VRAM) | 0.04340 | 0.05017 (0.05645 / 0.02508) | 0.07199 / 6.461 (+0.320) | 92.35% / 89.79% | 2.701 |
| Inkling-2-Mini-Vision.strata | 0.637 GB (0.59 GiB) | Vision shard | Lossless BF16 (422 model.visual.* tensors, 318.3M params) | | | | |
-->
<!-- Speed chart data
The speed chart is drawn from this table: one row per machine, with the highest prompt speed measured on it and its
output speed for one request, as one number or as lowest–highest across packs and kinds of prompt. Peak and micro
readings are left out. Every figure is also in the table under "How speeds were measured". Edit a row, then run
tools/make_charts.py.
| Machine | Prompt speed | Output speed |
| --- | ---: | ---: |
| RTX 5090 | 11,306 | 176.5–250.3 |
| RTX PRO 5000 | 6,840+ | 154.4–214.5 |
| RTX 5070 Ti | 4,237 | 54.2–155.5 |
| 2x RTX PRO 6000 | 7,980 | 131.6 |
| Olares One | 4,367 | 80.5–115+ |
| Apple M5 Max | 3,063 | 100.0–114.5 |
| Radeon AI PRO R9700 | 3,982 | 99.0–106.5 |
| RTX 5070 | 2,216 | 93.9–101.4 |
| DGX Spark | 2,197 | 46.1–74.3 |
| Tesla V100 | 2,287 | 70.5 |
| 2x RTX A4000 | 1,890+ | 68.4 |
| Radeon RX 7900 XT | 2,685 | 54.7+ |
| Strix Halo 128 GB | 1,466 | 47.3–48.0 |
| RTX 2080 Ti | 1,304 | 42.8 |
| Tesla P100 | 223 | 35.0 |
| Apple M4 Mac mini | 312.3 | 27.6 |
| GTX 1080 Ti | 244 | 24.6 |
| Radeon RX 6700 XT | 68.5 | 17.8 |
| Radeon 780M | 344.2 | 11.1 |
| Intel Arc A750 | 67.9 | 6.1 |
-->
|