Spaces:
Configuration error
Configuration error
Update README.md
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@
|
|
| 15 |
### 🌟 About Model Rampage
|
| 16 |
Model Rampage is an independent AI research organization focused on breaking the **custom kernel lock-in** and **long-context memory wall** in modern language models.
|
| 17 |
|
| 18 |
-
While existing sub-quadratic architectures (such as Mamba-2, RWKV, and Gated DeltaNet) rely on hardware-specific CUDA or
|
| 19 |
|
| 20 |
Because every chip on earth already accelerates matrix multiplication natively, BareTorch delivers sub-quadratic execution scaling and fixed-size recurrent state updates **out-of-the-box across Apple Silicon (MLX), NVIDIA GPUs (CUDA), AMD/Intel GPUs, WebGPU, and Mobile NPUs.**
|
| 21 |
|
|
@@ -49,28 +49,28 @@ Trained on **75 Billion tokens**, `BareTorch-2.5B-Base` outperforms foundational
|
|
| 49 |
|
| 50 |
By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-rank associative states (`rank = 16`, `chunk_size = 32`), **BareTorch-2.5B-Base** eliminates memory bandwidth bottlenecks on both edge laptops and local workstations:
|
| 51 |
|
| 52 |
-
- **
|
| 53 |
-
- **
|
| 54 |
-
- **44% Faster Workstation Decoding (NVIDIA CUDA):** Generates tokens at **65.4 tok/s** across all context windows compared to **45.6 tok/s** for `Qwen3.5-2B-Base`.
|
| 55 |
|
| 56 |
-
#### 1. Apple Silicon Edge Benchmark (MacBook Pro M1 — Native MLX
|
| 57 |
|
| 58 |
| Context Length | Model (Backend) | TTFT (ms) | Prefill Throughput | Decode Speed | Peak RAM |
|
| 59 |
| :---: | :--- | :---: | :---: | :---: | :---: |
|
| 60 |
-
| **512** | **BareTorch-2.5B (MLX
|
| 61 |
-
| 512 | Qwen3.5-2B (
|
| 62 |
-
| **1,024** | **BareTorch-2.5B (MLX
|
| 63 |
-
| 1,024 | Qwen3.5-2B (
|
| 64 |
-
| **2,048** | **BareTorch-2.5B (MLX
|
| 65 |
-
| 2,048 | Qwen3.5-2B (
|
| 66 |
-
| **4,096** | **BareTorch-2.5B (MLX
|
| 67 |
-
| 4,096 | Qwen3.5-2B (
|
| 68 |
-
| **8,192** | **BareTorch-2.5B (MLX
|
| 69 |
-
| 8,192 | Qwen3.5-2B (
|
| 70 |
-
| **16,384** | **BareTorch-2.5B (MLX
|
| 71 |
-
| 16,384 | Qwen3.5-2B (
|
| 72 |
-
| **32,768** | **BareTorch-2.5B (MLX
|
| 73 |
-
| 32,768 | Qwen3.5-2B (
|
| 74 |
|
| 75 |
#### 2. Local NVIDIA GPU Workstation Benchmark (CUDA — BF16)
|
| 76 |
|
|
@@ -105,5 +105,4 @@ By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-r
|
|
| 105 |
journal={BareTorch Framework Laboratory Technical Report},
|
| 106 |
year={2026},
|
| 107 |
url={[https://github.com/martin-kbcc/baretorch](https://github.com/martin-kbcc/baretorch)}
|
| 108 |
-
}
|
| 109 |
-
```
|
|
|
|
| 15 |
### 🌟 About Model Rampage
|
| 16 |
Model Rampage is an independent AI research organization focused on breaking the **custom kernel lock-in** and **long-context memory wall** in modern language models.
|
| 17 |
|
| 18 |
+
While existing sub-quadratic architectures (such as Mamba-2, RWKV, and Gated DeltaNet) rely on hardware-specific CUDA, Triton, or platform-specific Metal kernels that fail to port cleanly across consumer edge devices, our flagship framework—**BareTorch**—formulates sub-quadratic sequence mixers (like **CS-LRAD**) using **100% standard matrix multiplications (GEMMs)**.
|
| 19 |
|
| 20 |
Because every chip on earth already accelerates matrix multiplication natively, BareTorch delivers sub-quadratic execution scaling and fixed-size recurrent state updates **out-of-the-box across Apple Silicon (MLX), NVIDIA GPUs (CUDA), AMD/Intel GPUs, WebGPU, and Mobile NPUs.**
|
| 21 |
|
|
|
|
| 49 |
|
| 50 |
By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-rank associative states (`rank = 16`, `chunk_size = 32`), **BareTorch-2.5B-Base** eliminates memory bandwidth bottlenecks on both edge laptops and local workstations:
|
| 51 |
|
| 52 |
+
- **~34% Faster Edge Prefill on Apple Silicon (M1 MacBook Pro — Native MLX 4-Bit):** Under strict apples-to-apples 4-bit MLX quantization (`group_size=64`), BareTorch processes 32,768 tokens in **31.9 seconds (1,028.0 tok/s)** vs **42.6 seconds (769.8 tok/s)** for `Qwen3.5-2B` (MLX 4-bit), peaking at **1,204.1 tok/s** at 2,048 context with zero custom Metal shaders.
|
| 53 |
+
- **High-Throughput Edge Decoding & 30% Lower 32k RAM (M1 MacBook Pro — Native MLX 4-Bit):** Generates tokens at **99.27 tok/s** (1,024 context) and sustains **75.54 tok/s** at 32,768 context (vs **71.28 tok/s** for `Qwen3.5-2B`) while cutting 32k peak unified memory by **1.48 GB** (**3.52 GB** vs **5.00 GB**).
|
| 54 |
+
- **44% Faster Workstation Decoding (NVIDIA CUDA — BF16):** Generates tokens at **65.4 tok/s** across all context windows compared to **45.6 tok/s** for `Qwen3.5-2B-Base`.
|
| 55 |
|
| 56 |
+
#### 1. Apple Silicon Edge Benchmark (MacBook Pro M1 — Native MLX 4-Bit)
|
| 57 |
|
| 58 |
| Context Length | Model (Backend) | TTFT (ms) | Prefill Throughput | Decode Speed | Peak RAM |
|
| 59 |
| :---: | :--- | :---: | :---: | :---: | :---: |
|
| 60 |
+
| **512** | **BareTorch-2.5B (MLX 4-bit)** | **437.9 ms** | **1,169.3 tok/s** | **99.06 tok/s** | **1.99 GB** |
|
| 61 |
+
| 512 | Qwen3.5-2B (MLX 4-bit) | 576.4 ms | 888.2 tok/s | 98.82 tok/s | 1.84 GB |
|
| 62 |
+
| **1,024** | **BareTorch-2.5B (MLX 4-bit)** | **855.5 ms** | **1,197.0 tok/s** | **99.27 tok/s** | **2.28 GB** |
|
| 63 |
+
| 1,024 | Qwen3.5-2B (MLX 4-bit) | 1,140.1 ms | 898.2 tok/s | 99.07 tok/s | 2.10 GB |
|
| 64 |
+
| **2,048** | **BareTorch-2.5B (MLX 4-bit)** | **1,700.8 ms** | **1,204.1 tok/s** | **95.85 tok/s** | **2.41 GB** |
|
| 65 |
+
| 2,048 | Qwen3.5-2B (MLX 4-bit) | 2,271.7 ms | 901.5 tok/s | 96.63 tok/s | 2.32 GB |
|
| 66 |
+
| **4,096** | **BareTorch-2.5B (MLX 4-bit)** | **3,438.4 ms** | **1,191.2 tok/s** | **93.63 tok/s** | **2.69 GB** |
|
| 67 |
+
| 4,096 | Qwen3.5-2B (MLX 4-bit) | 4,617.7 ms | 887.0 tok/s | 94.39 tok/s | 2.76 GB |
|
| 68 |
+
| **8,192** | **BareTorch-2.5B (MLX 4-bit)** | **6,963.2 ms** | **1,176.5 tok/s** | **90.85 tok/s** | **2.69 GB** |
|
| 69 |
+
| 8,192 | Qwen3.5-2B (MLX 4-bit) | 9,386.1 ms | 872.8 tok/s | 90.65 tok/s | 3.18 GB |
|
| 70 |
+
| **16,384** | **BareTorch-2.5B (MLX 4-bit)** | **14,549.3 ms** | **1,126.1 tok/s** | **85.05 tok/s** | **3.33 GB** |
|
| 71 |
+
| 16,384 | Qwen3.5-2B (MLX 4-bit) | 19,503.7 ms | 840.0 tok/s | 83.17 tok/s | 3.75 GB |
|
| 72 |
+
| **32,768** | **BareTorch-2.5B (MLX 4-bit)** | **31,876.9 ms** | **1,028.0 tok/s** | **75.54 tok/s** | **3.52 GB** |
|
| 73 |
+
| 32,768 | Qwen3.5-2B (MLX 4-bit) | 42,567.7 ms | 769.8 tok/s | 71.28 tok/s | 5.00 GB |
|
| 74 |
|
| 75 |
#### 2. Local NVIDIA GPU Workstation Benchmark (CUDA — BF16)
|
| 76 |
|
|
|
|
| 105 |
journal={BareTorch Framework Laboratory Technical Report},
|
| 106 |
year={2026},
|
| 107 |
url={[https://github.com/martin-kbcc/baretorch](https://github.com/martin-kbcc/baretorch)}
|
| 108 |
+
}
|
|
|