q-ternary-bonsai-1.7b: Ternary Bonsai 1.7B as one streamable .holo
Created using Bonsai by Prism ML.
This is prism-ml/Ternary-Bonsai-1.7B, a Qwen3-1.7B with ternary weights {-1, 0, +1} and one scale per 128 weights, re-laid for the Hologram Q engine so it runs in a browser on WebGPU. Nothing is re-quantized: the trained codes pass through exactly, and the f16 group scales are widened to f32.
Experimental. It needs a patched Q engine. The stock engine has no ternary-per-128 kernel and does not apply YaRN position scaling, so it will not run this file correctly. The two changes are described below.
File
| File | What it is |
|---|---|
q-ternary-bonsai-1.7b.v1.holo |
One file, 424,013,749 bytes: header, embedded tokenizer (the source GGUF header), then gzip'd tensor blocks in the order the engine reads them, each named by the BLAKE3 of its stored bytes. |
- sha256 of the file:
acf0b5ad96b2a9b5b6a72b47cdddf5ec7ae4d15d5c50d0b5dea1ab245288ba99 - manifest root:
sha256:8932967dbd30cbabcd5bd446dc3e553991c07088760ba49e6c812c39aba419e6 - source:
Ternary-Bonsai-1.7B-PQ2_0.gguf(group 128)
The output layer is tied to the embedding, so both directory entries point at the same stored bytes.
Format
The header manifest is the 1-bit Bonsai manifest (mode: "q1", bits: 1) with two additions:
"tern": true. Each weight block is[sign plane N*K/8 bytes][nonzero plane N*K/8 bytes][f32 scale per 128 weights]. A weight isscale * (+1 if its sign bit is set, else -1)when its nonzero bit is set, and 0 otherwise. Bitg = n*K + k, least significant bit first."yarn": {"factor": 4.0, "orig": 8192}. The source GGUF declares YaRN rotary scaling. Rotary pairs below 17 keep their frequency, pairs above 34 run at one quarter, with a linear ramp between; the rotated vector is scaled by0.1 * ln(4) + 1. This matches transformers and llama.cpp.
Measured
One laptop (Radeon 8050S integrated GPU, Chromium, WebGPU), file served locally, greedy decoding, a 436-token prompt:
- 85 to 114 tokens per second.
- 0.31 to 0.45 s to first token once the fixed system prompt's state is cached; about 3.3 s to compute that state the first time.
- 4.1 to 5.6 s from page load to engine ready with the file already on disk.
- GPU memory 1,176 MB at a 3,000-token context, 809 MB at 1,400.
- Four of five outputs byte-identical to llama.cpp's WebGPU build on the same prompts; the fifth matches for 1,350 of 1,867 characters.
Not measured: phones, Safari, Firefox, and download time from this host.
Licence
Apache-2.0, as the source model. See LICENSE and NOTICE.txt, both copied unchanged from the source repo.
Model tree for HOLOGRAMTECH/q-ternary-bonsai-1.7b
Base model
prism-ml/Ternary-Bonsai-1.7B-unpacked