README.md

---

license: apache-2.0
library_name: onnx
tags:
  - onnx
  - onnxruntime
  - text-generation
  - automatic-speech-recognition
  - text-to-speech
  - image-feature-extraction
  - spike-whale
---


# ONNX-Library

ONNX exports of the **Quazim0t0 model family** — the SpikeWhale-DNA language models (Byrne / Escarda),
the Byrne ASR and TTS models, and the Byrne-VLM vision encoder. Every graph was exported from the
original PyTorch weights with the legacy TorchScript exporter (opset 17) and **parity-verified**
against the source model.

Each model lives in **its own folder**. The `.onnx` graphs contain the neural network only —
tokenizers, text frontends, and CTC/beam decoders stay in the original source repos (linked below),
exactly as they do at inference time.

## Contents

### Language models — `SpikeWhaleLM` (13)
`input_ids [B,T] (int64) → logits [B,T,16512]` · dynamic batch & sequence · file `model.onnx` (~383 MB each)

| Folder | Family trait | Source |
|---|---|---|
| `Byrne-86M` | HRM | [Byrne-86M](https://huggingface.co/Quazim0t0/Byrne-86M) |
| `Byrne-86M-Base` | HRM (base) | [Byrne-86M-Base](https://huggingface.co/Quazim0t0/Byrne-86M-Base) |
| `Byrne-86M-Base-JL` | HRM (base, JL) | [Byrne-86M-Base-JL](https://huggingface.co/Quazim0t0/Byrne-86M-Base-JL) |
| `Byrne-86M-JL` | HRM (JL) | [Byrne-86M-JL](https://huggingface.co/Quazim0t0/Byrne-86M-JL) |
| `Byrne-TriAtn-86M` | HRM (tri-attention) | [Byrne-TriAtn-86M](https://huggingface.co/Quazim0t0/Byrne-TriAtn-86M) |
| `Byrne-TriAtn-86M-JL` | HRM (tri-attn, JL) | [Byrne-TriAtn-86M-JL](https://huggingface.co/Quazim0t0/Byrne-TriAtn-86M-JL) |
| `Escarda-86M` | HRM + JEPA | [Escarda-86M](https://huggingface.co/Quazim0t0/Escarda-86M) |
| `Escarda-86M-Base` | HRM+JEPA (base) | [Escarda-86M-Base](https://huggingface.co/Quazim0t0/Escarda-86M-Base) |
| `Escarda-86M-Base-JL` | HRM+JEPA (base, JL) | [Escarda-86M-Base-JL](https://huggingface.co/Quazim0t0/Escarda-86M-Base-JL) |
| `Escarda-86M-Identity` | HRM+JEPA (identity) | [Escarda-86M-Identity](https://huggingface.co/Quazim0t0/Escarda-86M-Identity) |
| `Escarda-86M-JL` | HRM+JEPA (JL) | [Escarda-86M-JL](https://huggingface.co/Quazim0t0/Escarda-86M-JL) |
| `Escarda-TriAtn-86M` | HRM+JEPA (tri-attn) | [Escarda-TriAtn-86M](https://huggingface.co/Quazim0t0/Escarda-TriAtn-86M) |
| `Escarda-TriAtn-86M-JL` | HRM+JEPA (tri-attn, JL) | [Escarda-TriAtn-86M-JL](https://huggingface.co/Quazim0t0/Escarda-TriAtn-86M-JL) |

Tokenizer (`tokenizer.json` + `spike_tokenizer.py`) is in each source repo. Verified **100 % argmax-token

agreement** with PyTorch (the deep custom ops add ~1e-1 fp noise to the wide logits — harmless; base
variants are near-exact).

### Speech recognition — `Byrne-ASR-English/model.onnx` (~50 MB)
`mel [B,80,T] (float32) → logits [B,T',29]` (CTC, dynamic frames). Mel frontend params: sample_rate 24000,

n_fft 1024, hop 256, n_mels 80, log-mel. Vocab: `<blank>`, space, a–z, `'`. The lexicon / bigram / ARPA

beam-search decode lives in the source repo ([Byrne-ASR-English](https://huggingface.co/Quazim0t0/Byrne-ASR-English)).

Parity 7e-6.



### Vision encoder — `Byrne-VLM-131M/vision.onnx` (~167 MB)

`image [B,3,448,448] (float32, [-1,1]) → pooled [B,512] + tokens [B,784,512]`. ViT-style, patch 16, native

**448×448** (28×28 patch grid), 2D axial RoPE. Source: [Byrne-VLM-131M](https://huggingface.co/Quazim0t0/Byrne-VLM-131M).

Parity 1e-6. *(The multimodal LM half is not included here.)*



### Text-to-speech — `Byrne-Speech/` (2-stage, ~49 MB)

1. `acoustic.onnx` — FastSpeech2: `ids [B,Tp] (int64) + plen [B] → mel [B,80,Tm]` (variable length; length

   regulator generalizes across text lengths).

2. `vocoder.onnx` — HiFi-GAN: `mel [B,80,Tm] → wav [B,1,Tm*256]` (24 kHz, hop 256). `weight_norm` folded.

Char text frontend (`text_to_char_sequence`) is in the source repo ([Byrne-Speech](https://huggingface.co/Quazim0t0/Byrne-Speech)).
Chain: `text → acoustic.onnx → mel → vocoder.onnx → wav`. Parity 7e-7.

### Tools (4)

| Folder | Task | I/O contract | Source |
|---|---|---|---|
| `Escarda-Rewrite/model.onnx` | text rewriting (causal LM) | `input_ids[B,T] -> logits[B,T,16512]` | [Escarda-Rewrite](https://huggingface.co/Quazim0t0/Escarda-Rewrite) |
| `Byrne-Embed/model.onnx` | text embeddings | `input_ids[B,T] -> embedding[B,768]` (pooled sentence vector) | [Byrne-Embed](https://huggingface.co/Quazim0t0/Byrne-Embed) |
| `Byrne-Anon/model.onnx` | PII tagging (BIOES) | `input_ids[B,T] -> pii_logits[B,T,33]` (labels in source pii_labels.json) | [Byrne-Anon](https://huggingface.co/Quazim0t0/Byrne-Anon) |

| `Byrne-Docling-131M/vision.onnx` | document VLM vision encoder | `image[B,3,448,448] -> pooled[B,512] + tokens[B,784,512]` | [Byrne-Docling-131M](https://huggingface.co/Quazim0t0/Byrne-Docling-131M) |



## Vision-language generation (full pipeline)



`Byrne-VLM-131M` (captioning) and `Byrne-Docling-131M` (document -> DocTags) are **generative** — each

ships THREE files for real image->text generation (the single `vision.onnx` is the encoder only):



- `vision_connector.onnx` : `image[1,3,448,448] -> image_embeds[1,784,640]` (vision encoder + projector)

- `lm_decode.onnx`        : `inputs_embeds[1,T,640] -> logits[1,T,V]` (LoRA-applied LM)

- `embed_tokens.npy`      : `[V,640]` token-embedding table (for generated text tokens)

**Generate:** encode the image once, feed `image_embeds` as the prefix, then autoregressively append
`embed_tokens[next_token]` and re-run `lm_decode`:
```python

import numpy as np, onnxruntime as ort

va=ort.InferenceSession("Byrne-VLM-131M/vision_connector.onnx")

lm=ort.InferenceSession("Byrne-VLM-131M/lm_decode.onnx")

emb=np.load("Byrne-VLM-131M/embed_tokens.npy")

ie=va.run(["image_embeds"],{"image":img})[0]          # img: [1,3,448,448] float32 in [-1,1]

out=[]

for _ in range(48):

    x = ie if not out else np.concatenate([ie, emb[out][None]], 1)

    nxt = int(lm.run(["logits"],{"inputs_embeds":x})[0][0,-1].argmax())

    if nxt==EOS: break

    out.append(nxt)                                    # decode with the model's tokenizer

```
Verified to produce PyTorch-identical greedy output (Byrne-VLM: *"A group of people standing on the

ground."*). Byrne-VLM decodes with the shared LM tokenizer; Byrne-Docling with its source
`tokenizer_doctags.json`.

## Usage (ONNX Runtime)

**Language model** (needs the tokenizer from the source repo):
```python

import onnxruntime as ort, numpy as np

sess = ort.InferenceSession("Byrne-86M/model.onnx", providers=["CPUExecutionProvider"])

input_ids = np.array([[1, 23, 45, 6]], dtype=np.int64)          # from the SpikeWhale tokenizer

logits = sess.run(["logits"], {"input_ids": input_ids})[0]      # [1, T, 16512]

next_id = logits[0, -1].argmax()                                # greedy next token

```

**TTS** (chain the two graphs):
```python

import onnxruntime as ort, numpy as np

ac = ort.InferenceSession("Byrne-Speech/acoustic.onnx", providers=["CPUExecutionProvider"])

vo = ort.InferenceSession("Byrne-Speech/vocoder.onnx",  providers=["CPUExecutionProvider"])

ids  = np.array([[...]], dtype=np.int64)                         # text_to_char_sequence(text)

plen = np.array([ids.shape[1]], dtype=np.int64)

mel = ac.run(["mel"], {"ids": ids, "plen": plen})[0]            # [1,80,Tm]

wav = vo.run(["wav"], {"mel": mel})[0]                          # [1,1,Tm*256] @ 24 kHz

```

**Vision encoder:**
```python

pooled, tokens = sess.run(["pooled","tokens"], {"images": img})  # img [B,3,448,448] float32 in [-1,1]

```

All graphs use dynamic batch (and dynamic sequence/frames where noted), so batching works out of the box.

## Provenance

Exported 1:1 from the PyTorch checkpoints in the linked source repos (opset 17, `dynamo=False`), each
verified against its original model. License: Apache-2.0.