Text Generation
Transformers
Safetensors
GGUF
English
grt
geometric-rotation-transformer
causal-lm
rotational
knowledge-distillation
qwen3
Instructions to use MatterWaveAI/grt-30m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MatterWaveAI/grt-30m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MatterWaveAI/grt-30m")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("MatterWaveAI/grt-30m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use MatterWaveAI/grt-30m with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf MatterWaveAI/grt-30m:F32 # Run inference directly in the terminal: llama cli -hf MatterWaveAI/grt-30m:F32
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf MatterWaveAI/grt-30m:F32 # Run inference directly in the terminal: llama cli -hf MatterWaveAI/grt-30m:F32
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf MatterWaveAI/grt-30m:F32 # Run inference directly in the terminal: ./build/bin/llama cli -hf MatterWaveAI/grt-30m:F32
Use Docker
docker model run hf.co/MatterWaveAI/grt-30m:F32
- Llama new
- LM Studio
- Jan
- vLLM
How to use MatterWaveAI/grt-30m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MatterWaveAI/grt-30m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MatterWaveAI/grt-30m:F32
- SGLang
How to use MatterWaveAI/grt-30m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MatterWaveAI/grt-30m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MatterWaveAI/grt-30m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MatterWaveAI/grt-30m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Ollama
How to use MatterWaveAI/grt-30m with Ollama:
ollama run hf.co/MatterWaveAI/grt-30m:F32
- Unsloth Desktop
- Docker Model Runner
How to use MatterWaveAI/grt-30m with Docker Model Runner:
docker model run hf.co/MatterWaveAI/grt-30m:F32
- Lemonade
How to use MatterWaveAI/grt-30m with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull MatterWaveAI/grt-30m:F32
Run and chat with the model
lemonade run user.grt-30m-F32
List all available models
lemonade list
- Atomic Chat
docs(model-card): replace raw LaTeX with clean Unicode and code formatting
Browse files- README.md +10 -7
- SHA256SUMS +1 -1
README.md
CHANGED
|
@@ -22,7 +22,7 @@ pipeline_tag: text-generation
|
|
| 22 |
|
| 23 |
# GRT-30M (24-Hour Qwen 3 Distillation Preview)
|
| 24 |
|
| 25 |
-
A **Geometric Rotation Transformer (`GRTForCausalLM`)** distilled from **Qwen 3** (sharing the `151,669`-token Qwen 3 BPE vocabulary) in a single **24-hour distillation run** (`10.79B` tokens at `~123.9k tok/s` across `8x NVIDIA A100-SXM4-80GB`). GRT replaces every dense linear projection (`nn.Linear`) in attention (`Q, K, V, O`) and feed-forward (`expand, contract`) with cascades of content-modulated
|
| 26 |
|
| 27 |
**This is a base research checkpoint distilled from Qwen 3, not an instruction-tuned assistant.** It demonstrates that a rotation-only student backbone with just **`9.74M` non-embedding parameters** (`29.31M` total stored parameters with the 151,669-token Qwen 3 vocabulary):
|
| 28 |
1. Distills stably from Qwen 3 at **`123,900 tok/s`** to **`val_loss = 4.4776`** (`val_ppl = 88.02`, **`27.2%` top-1 token accuracy** over a `151,669`-token vocabulary).
|
|
@@ -123,7 +123,7 @@ Evaluated zero-shot (no task fine-tuning) via teacher-forced length-normalized l
|
|
| 123 |
|
| 124 |
#### PyTorch ROCm (`gfx1151`, `BF16`, Fused HIP Kernels)
|
| 125 |
|
| 126 |
-
| Workload | Batch
|
| 127 |
|---|---|---:|---:|
|
| 128 |
| **Prefill (`B=1, T=512`)** | `1 x 512` | `129.36` | `3,957.9` |
|
| 129 |
| **Prefill (`B=1, T=2048`)** | `1 x 2048` | `357.89` | **`5,722.5`** |
|
|
@@ -177,16 +177,19 @@ sha256sum --ignore-missing -c SHA256SUMS
|
|
| 177 |
|
| 178 |
## Architecture Overview
|
| 179 |
|
| 180 |
-
GRT replaces dense weight matrices (`nn.Linear`) in both attention and FFN blocks with cascades of learned
|
| 181 |
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
|
|
|
|
|
|
|
|
|
| 185 |
|
| 186 |
- **`GRTEmbedding` & `GRTCosineHead`** — Token embeddings are stored in polar form (`phase_angles` + `log_magnitude`) and tied to a scaled cosine-similarity language modeling head (`lm_head.output_scale = 40.23`).
|
| 187 |
- **`GRTAttention`** — 11-stage content-modulated Givens butterfly cascades for `Q, K, V, O`, paired with standard scaled dot-product attention and learnable RoPE frequencies.
|
| 188 |
- **`GRTMixer` & `GRTFFN`** — Cross-head Givens mixing stage followed by 21-stage rotational FFN expansion/contraction and phase-preserving `GRTRadialGELU`.
|
| 189 |
-
- **`GRTPearlNorm`** — Pair-aware RMS normalization that normalizes pair radii
|
| 190 |
|
| 191 |
---
|
| 192 |
|
|
|
|
| 22 |
|
| 23 |
# GRT-30M (24-Hour Qwen 3 Distillation Preview)
|
| 24 |
|
| 25 |
+
A **Geometric Rotation Transformer (`GRTForCausalLM`)** distilled from **Qwen 3** (sharing the `151,669`-token Qwen 3 BPE vocabulary) in a single **24-hour distillation run** (`10.79B` tokens at `~123.9k tok/s` across `8x NVIDIA A100-SXM4-80GB`). GRT replaces every dense linear projection (`nn.Linear`) in attention (`Q, K, V, O`) and feed-forward (`expand, contract`) with cascades of content-modulated `2×2` Givens rotations on `SO(D)` interleaved with butterfly permutations.
|
| 26 |
|
| 27 |
**This is a base research checkpoint distilled from Qwen 3, not an instruction-tuned assistant.** It demonstrates that a rotation-only student backbone with just **`9.74M` non-embedding parameters** (`29.31M` total stored parameters with the 151,669-token Qwen 3 vocabulary):
|
| 28 |
1. Distills stably from Qwen 3 at **`123,900 tok/s`** to **`val_loss = 4.4776`** (`val_ppl = 88.02`, **`27.2%` top-1 token accuracy** over a `151,669`-token vocabulary).
|
|
|
|
| 123 |
|
| 124 |
#### PyTorch ROCm (`gfx1151`, `BF16`, Fused HIP Kernels)
|
| 125 |
|
| 126 |
+
| Workload | Batch × Seq (`B × T`) | Latency (`ms`) | Throughput (`tok/s`) |
|
| 127 |
|---|---|---:|---:|
|
| 128 |
| **Prefill (`B=1, T=512`)** | `1 x 512` | `129.36` | `3,957.9` |
|
| 129 |
| **Prefill (`B=1, T=2048`)** | `1 x 2048` | `357.89` | **`5,722.5`** |
|
|
|
|
| 177 |
|
| 178 |
## Architecture Overview
|
| 179 |
|
| 180 |
+
GRT replaces dense weight matrices (`nn.Linear`) in both attention and FFN blocks with cascades of learned `2×2` Givens rotations arranged in a butterfly topology. At each stage `k`, pairs of coordinates `(x0, x1)` are rotated by an angle `theta_k` modulated by 4-component `Cl(2,0)` multivector features:
|
| 181 |
|
| 182 |
+
```text
|
| 183 |
+
feat(x0, x1) = [x0^2 + x1^2, x0, x1, x0 * x1_next - x1 * x0_next]
|
| 184 |
+
theta_k = alpha_k + tanh(norm(feat(x0, x1)) @ C_k)
|
| 185 |
+
r0 = x0 * cos(theta_k) - x1 * sin(theta_k)
|
| 186 |
+
r1 = x0 * sin(theta_k) + x1 * cos(theta_k)
|
| 187 |
+
```
|
| 188 |
|
| 189 |
- **`GRTEmbedding` & `GRTCosineHead`** — Token embeddings are stored in polar form (`phase_angles` + `log_magnitude`) and tied to a scaled cosine-similarity language modeling head (`lm_head.output_scale = 40.23`).
|
| 190 |
- **`GRTAttention`** — 11-stage content-modulated Givens butterfly cascades for `Q, K, V, O`, paired with standard scaled dot-product attention and learnable RoPE frequencies.
|
| 191 |
- **`GRTMixer` & `GRTFFN`** — Cross-head Givens mixing stage followed by 21-stage rotational FFN expansion/contraction and phase-preserving `GRTRadialGELU`.
|
| 192 |
+
- **`GRTPearlNorm`** — Pair-aware RMS normalization that normalizes pair radii `sqrt(x_{2i}^2 + x_{2i+1}^2)` while preserving phase angles.
|
| 193 |
|
| 194 |
---
|
| 195 |
|
SHA256SUMS
CHANGED
|
@@ -2,6 +2,6 @@
|
|
| 2 |
5dbb8b936d1e2411f30d1ecf072a4d7c2f2df6e299b0d11c7ef5cbaf72ebd5bc generation_config.json
|
| 3 |
65bda04ea57b1799e7b0c6d09102475a81c3247c2e29fdb5cbdcdf6d52a21af2 grt-30m-f32.gguf
|
| 4 |
205f76f3ed59fee0ed54bb8133ea1b1a64527994c68776066057b4707f1b2456 model.safetensors
|
| 5 |
-
|
| 6 |
be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506 tokenizer.json
|
| 7 |
5f007d04324afa93876e2a484190e868136c69dfe0e1a29a1063eae583df9d32 tokenizer_config.json
|
|
|
|
| 2 |
5dbb8b936d1e2411f30d1ecf072a4d7c2f2df6e299b0d11c7ef5cbaf72ebd5bc generation_config.json
|
| 3 |
65bda04ea57b1799e7b0c6d09102475a81c3247c2e29fdb5cbdcdf6d52a21af2 grt-30m-f32.gguf
|
| 4 |
205f76f3ed59fee0ed54bb8133ea1b1a64527994c68776066057b4707f1b2456 model.safetensors
|
| 5 |
+
025b49eeb0c4dab8e792d4931140b7334191bbde8f0d9795b091017cbdfff14c README.md
|
| 6 |
be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506 tokenizer.json
|
| 7 |
5f007d04324afa93876e2a484190e868136c69dfe0e1a29a1063eae583df9d32 tokenizer_config.json
|