Release MatterWaveAI/grt-30m-tokenizer (BPE tokenizer + 19.57M shared polar phase-angle embedding bank)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,84 @@
+---
+license: apache-2.0
+library_name: transformers
+tags:
+  - grt
+  - geometric-rotation-transformer
+  - tokenizer
+  - polar-embedding
+  - phase-angle-embedding
+  - rotational-transformer
+  - qwen3
+  - matterwave
+base_model: MatterWaveAI/grt-30m
+---
+
+# GRT-30M-Tokenizer (`MatterWaveAI/grt-30m-tokenizer`)
+
+Canonical **BPE Tokenizer (`151,669` tokens)** and shared **Polar Phase-Angle Token Embedding Bank (`19,565,302` parameters, `0 nn.Linear` layers)** for the **GRT-30M Research Suite**:
+- [`MatterWaveAI/grt-30m`](https://huggingface.co/MatterWaveAI/grt-30m) (Base Causal LM)
+- [`MatterWaveAI/grt-30m-chat`](https://huggingface.co/MatterWaveAI/grt-30m-chat) (ChatML Conversational SFT)
+- [`MatterWaveAI/grt-30m-mor`](https://huggingface.co/MatterWaveAI/grt-30m-mor) (4-Expert Mixture-of-Rotations Causal LM)
+- [`MatterWaveAI/grt-30m-embedding`](https://huggingface.co/MatterWaveAI/grt-30m-embedding) (Circular-Pooled Bi-Encoder)
+- [`MatterWaveAI/grt-30m-reranker`](https://huggingface.co/MatterWaveAI/grt-30m-reranker) (Rotational Cross-Encoder Reranker)
+- [`MatterWaveAI/grt-30m-vl`](https://huggingface.co/MatterWaveAI/grt-30m-vl) (100% Rotational Vision-Language Model)
+
+---
+
+## What Is Included
+
+| File | Size | Contents |
+| :--- | :---: | :--- |
+| `tokenizer.json` | `11.4 MB` | Rust `PreTrainedTokenizerFast` BPE vocabulary (`151,669` tokens, Qwen 3 compatible + ChatML special tokens) |
+| `tokenizer_config.json` | `1.2 KB` | Tokenizer configuration including the ChatML `chat_template` (`<|im_start|>` / `<|im_end|>`) |
+| `polar_embedding.safetensors` | `78.3 MB` | Shared polar phase-angle embedding table & tied cosine head scale (`19,565,302` FP32 parameters) |
+| `config.json` | `310 B` | Polar embedding geometry metadata (`vocab_size = 151669`, `hidden_size = 256`, `n_pairs = 128`) |
+
+### Polar Embedding Tensors (`polar_embedding.safetensors`)
+
+In the Geometric Rotation Transformer (`GRT`), token embeddings live in polar coordinates on `128` orthogonal `2D` planes (`hidden_size = 256`) rather than an unstructured Euclidean matrix:
+```text
+e(token_id)[2p]   = exp(log_magnitude[token_id]) · cos(phase_angles[token_id, p])
+e(token_id)[2p+1] = exp(log_magnitude[token_id]) · sin(phase_angles[token_id, p])
+```
+
+| Tensor Name | Shape | Parameters | Role |
+| :--- | :---: | :---: | :--- |
+| `embedding.phase_angles` | `[151669, 128]` | `19,413,632` | Per-token phase angles `θ` on the 128-torus `(S¹)^128` (tied between input embedding and output `GRTCosineHead`) |
+| `embedding.log_magnitude` | `[151669]` | `151,669` | Per-token radial log-magnitude offset |
+| `lm_head.output_scale` | `[]` (scalar) | `1` | Raw softplus temperature scale for the tied polar cosine language modeling head |
+| **Total** | — | **`19,565,302`** | **`66.8%` of `grt-30m` total parameters (shared identically across all 6 `grt-30m*` models)** |
+
+---
+
+## Quickstart
+
+### 1. Load the Tokenizer (`transformers` / `vLLM`)
+
+```python
+from transformers import AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("MatterWaveAI/grt-30m-tokenizer")
+tokens = tokenizer("The Givens Rotational Transformer replaces dense matrices with")
+print(tokens["input_ids"])
+```
+
+### 2. Load the Shared Polar Embedding Table (`safetensors`)
+
+```python
+import torch
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+
+path = hf_hub_download("MatterWaveAI/grt-30m-tokenizer", "polar_embedding.safetensors")
+polar_bank = load_file(path)
+
+phase_angles = polar_bank["embedding.phase_angles"]   # [151669, 128]
+log_magnitude = polar_bank["embedding.log_magnitude"] # [151669]
+output_scale = polar_bank["lm_head.output_scale"]     # []
+print(phase_angles.shape, log_magnitude.shape, output_scale.shape)
+```
+
+### 3. Harmonic Upcycling to Larger `d_model` (`256 → 512` or `1024`)
+
+When scaling a new GRT backbone to a wider `d_model = 256 · k` (`n_pairs = 128 · k`), tile `embedding.phase_angles` by `k` along dim `-1` (`phase_angles.repeat(1, k)`) to preserve all pairwise token cosine similarities at initialization.

SHA256SUMS

@@ -0,0 +1,5 @@
+4c7ed24fcbaf5b1c62ec160296e682eb7b700ea19d88f0a8e01e363b125d505f  config.json
+ffb11d3b6b961df2d916e158137d6f0aa93d6ae8b6c98630f781ec9a6fa9877b  polar_embedding.safetensors
+393ae1ae75bce22d1d3fce5b51ac74232b8893a1f4e8dc873d46a3a6052c4c6b  README.md
+be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506  tokenizer.json
+b0b05688d53eec97348d23f9a203772fa28f12bdc8a0f6f5401f41568ec6ca31  tokenizer_config.json

config.json

@@ -0,0 +1,12 @@
+{
+  "model_type": "grt",
+  "vocab_size": 151669,
+  "hidden_size": 256,
+  "n_pairs": 128,
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "pad_token_id": 151643,
+  "output_scale_init": 30.0,
+  "polar_embedding_parameters": 19565302,
+  "polar_embedding_file": "polar_embedding.safetensors"
+}

polar_embedding.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffb11d3b6b961df2d916e158137d6f0aa93d6ae8b6c98630f781ec9a6fa9877b
+size 78261488

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}