q-ternary-bonsai-1.7b: Ternary Bonsai 1.7B as one streamable .holo

Created using Bonsai by Prism ML.

This is prism-ml/Ternary-Bonsai-1.7B, a Qwen3-1.7B with ternary weights {-1, 0, +1} and one scale per 128 weights, re-laid for the Hologram Q engine so it runs in a browser on WebGPU. Nothing is re-quantized: the trained codes pass through exactly, and the f16 group scales are widened to f32.

Experimental. It needs a patched Q engine. The stock engine has no ternary-per-128 kernel and does not apply YaRN position scaling, so it will not run this file correctly. The two changes are described below.

File

File What it is
q-ternary-bonsai-1.7b.v1.holo One file, 424,013,749 bytes: header, embedded tokenizer (the source GGUF header), then gzip'd tensor blocks in the order the engine reads them, each named by the BLAKE3 of its stored bytes.
  • sha256 of the file: acf0b5ad96b2a9b5b6a72b47cdddf5ec7ae4d15d5c50d0b5dea1ab245288ba99
  • manifest root: sha256:8932967dbd30cbabcd5bd446dc3e553991c07088760ba49e6c812c39aba419e6
  • source: Ternary-Bonsai-1.7B-PQ2_0.gguf (group 128)

The output layer is tied to the embedding, so both directory entries point at the same stored bytes.

Format

The header manifest is the 1-bit Bonsai manifest (mode: "q1", bits: 1) with two additions:

  • "tern": true. Each weight block is [sign plane N*K/8 bytes][nonzero plane N*K/8 bytes][f32 scale per 128 weights]. A weight is scale * (+1 if its sign bit is set, else -1) when its nonzero bit is set, and 0 otherwise. Bit g = n*K + k, least significant bit first.
  • "yarn": {"factor": 4.0, "orig": 8192}. The source GGUF declares YaRN rotary scaling. Rotary pairs below 17 keep their frequency, pairs above 34 run at one quarter, with a linear ramp between; the rotated vector is scaled by 0.1 * ln(4) + 1. This matches transformers and llama.cpp.

Measured

One laptop (Radeon 8050S integrated GPU, Chromium, WebGPU), file served locally, greedy decoding, a 436-token prompt:

  • 85 to 114 tokens per second.
  • 0.31 to 0.45 s to first token once the fixed system prompt's state is cached; about 3.3 s to compute that state the first time.
  • 4.1 to 5.6 s from page load to engine ready with the file already on disk.
  • GPU memory 1,176 MB at a 3,000-token context, 809 MB at 1,400.
  • Four of five outputs byte-identical to llama.cpp's WebGPU build on the same prompts; the fifth matches for 1,350 of 1,867 characters.

Not measured: phones, Safari, Firefox, and download time from this host.

Licence

Apache-2.0, as the source model. See LICENSE and NOTICE.txt, both copied unchanged from the source repo.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for HOLOGRAMTECH/q-ternary-bonsai-1.7b