martin-kb-rampage commited on
Commit
77bb38c
·
verified ·
1 Parent(s): 12d06e0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -21
README.md CHANGED
@@ -15,7 +15,7 @@
15
  ### 🌟 About Model Rampage
16
  Model Rampage is an independent AI research organization focused on breaking the **custom kernel lock-in** and **long-context memory wall** in modern language models.
17
 
18
- While existing sub-quadratic architectures (such as Mamba-2, RWKV, and Gated DeltaNet) rely on hardware-specific CUDA or Triton kernels that fail to port cleanly to consumer edge devices, our flagship framework—**BareTorch**—formulates sub-quadratic sequence mixers (like **CS-LRAD**) using **100% standard matrix multiplications (GEMMs)**.
19
 
20
  Because every chip on earth already accelerates matrix multiplication natively, BareTorch delivers sub-quadratic execution scaling and fixed-size recurrent state updates **out-of-the-box across Apple Silicon (MLX), NVIDIA GPUs (CUDA), AMD/Intel GPUs, WebGPU, and Mobile NPUs.**
21
 
@@ -49,28 +49,28 @@ Trained on **75 Billion tokens**, `BareTorch-2.5B-Base` outperforms foundational
49
 
50
  By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-rank associative states (`rank = 16`, `chunk_size = 32`), **BareTorch-2.5B-Base** eliminates memory bandwidth bottlenecks on both edge laptops and local workstations:
51
 
52
- - **3.60x Faster Edge Prefill on Apple Silicon (M1 MacBook Pro - Native MLX):** Processes 32,768 tokens in **30.6 seconds (1,070.3 tok/s)** vs **110.2 seconds (297.5 tok/s)** for `Qwen3.5-2B` on PyTorch MPS, peaking at **1,326.7 tok/s** at 4,096 context with zero custom Metal shaders.
53
- - **1.36x Faster Edge Decoding on Apple Silicon (M1 MacBook Pro):** Sustains **26.40 tok/s** at 32k context (and **35.78 tok/s** at 512 context) within a compact **5.93 GB RAM** footprint.
54
- - **44% Faster Workstation Decoding (NVIDIA CUDA):** Generates tokens at **65.4 tok/s** across all context windows compared to **45.6 tok/s** for `Qwen3.5-2B-Base`.
55
 
56
- #### 1. Apple Silicon Edge Benchmark (MacBook Pro M1 — Native MLX FP16)
57
 
58
  | Context Length | Model (Backend) | TTFT (ms) | Prefill Throughput | Decode Speed | Peak RAM |
59
  | :---: | :--- | :---: | :---: | :---: | :---: |
60
- | **512** | **BareTorch-2.5B (MLX Native)** | **416.39 ms** | **1,229.62 tok/s** | **35.78 tok/s** | 4.80 GB |
61
- | 512 | Qwen3.5-2B (PyTorch MPS) | 1,192.24 ms | 429.44 tok/s | 27.05 tok/s | 3.54 GB |
62
- | **1,024** | **BareTorch-2.5B (MLX Native)** | **773.78 ms** | **1,323.37 tok/s** | **35.67 tok/s** | 4.87 GB |
63
- | 1,024 | Qwen3.5-2B (PyTorch MPS) | 2,384.76 ms | 429.39 tok/s | 26.76 tok/s | 3.55 GB |
64
- | **2,048** | **BareTorch-2.5B (MLX Native)** | **1,549.67 ms** | **1,321.58 tok/s** | **34.51 tok/s** | 5.03 GB |
65
- | 2,048 | Qwen3.5-2B (PyTorch MPS) | 4,760.31 ms | 430.22 tok/s | 26.57 tok/s | 3.55 GB |
66
- | **4,096** | **BareTorch-2.5B (MLX Native)** | **3,087.39 ms** | **1,326.69 tok/s** | **32.78 tok/s** | 5.31 GB |
67
- | 4,096 | Qwen3.5-2B (PyTorch MPS) | 9,630.56 ms | 425.31 tok/s | 26.24 tok/s | 3.58 GB |
68
- | **8,192** | **BareTorch-2.5B (MLX Native)** | **6,360.21 ms** | **1,288.01 tok/s** | **32.56 tok/s** | 5.42 GB |
69
- | 8,192 | Qwen3.5-2B (PyTorch MPS) | 19,937.32 ms | 410.89 tok/s | 25.71 tok/s | 3.63 GB |
70
- | **16,384** | **BareTorch-2.5B (MLX Native)** | **13,603.16 ms** | **1,204.43 tok/s** | **30.54 tok/s** | 5.59 GB |
71
- | 16,384 | Qwen3.5-2B (PyTorch MPS) | 43,497.17 ms | 376.67 tok/s | 22.39 tok/s | 3.72 GB |
72
- | **32,768** | **BareTorch-2.5B (MLX Native)** | **30,615.66 ms** | **1,070.30 tok/s** | **26.40 tok/s** | 5.93 GB |
73
- | 32,768 | Qwen3.5-2B (PyTorch MPS) | 110,152.64 ms | 297.48 tok/s | 19.45 tok/s | 3.92 GB |
74
 
75
  #### 2. Local NVIDIA GPU Workstation Benchmark (CUDA — BF16)
76
 
@@ -105,5 +105,4 @@ By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-r
105
  journal={BareTorch Framework Laboratory Technical Report},
106
  year={2026},
107
  url={[https://github.com/martin-kbcc/baretorch](https://github.com/martin-kbcc/baretorch)}
108
- }
109
- ```
 
15
  ### 🌟 About Model Rampage
16
  Model Rampage is an independent AI research organization focused on breaking the **custom kernel lock-in** and **long-context memory wall** in modern language models.
17
 
18
+ While existing sub-quadratic architectures (such as Mamba-2, RWKV, and Gated DeltaNet) rely on hardware-specific CUDA, Triton, or platform-specific Metal kernels that fail to port cleanly across consumer edge devices, our flagship framework—**BareTorch**—formulates sub-quadratic sequence mixers (like **CS-LRAD**) using **100% standard matrix multiplications (GEMMs)**.
19
 
20
  Because every chip on earth already accelerates matrix multiplication natively, BareTorch delivers sub-quadratic execution scaling and fixed-size recurrent state updates **out-of-the-box across Apple Silicon (MLX), NVIDIA GPUs (CUDA), AMD/Intel GPUs, WebGPU, and Mobile NPUs.**
21
 
 
49
 
50
  By replacing 75% of growing Transformer KV-caches with compact, fixed-size low-rank associative states (`rank = 16`, `chunk_size = 32`), **BareTorch-2.5B-Base** eliminates memory bandwidth bottlenecks on both edge laptops and local workstations:
51
 
52
+ - **~34% Faster Edge Prefill on Apple Silicon (M1 MacBook Pro — Native MLX 4-Bit):** Under strict apples-to-apples 4-bit MLX quantization (`group_size=64`), BareTorch processes 32,768 tokens in **31.9 seconds (1,028.0 tok/s)** vs **42.6 seconds (769.8 tok/s)** for `Qwen3.5-2B` (MLX 4-bit), peaking at **1,204.1 tok/s** at 2,048 context with zero custom Metal shaders.
53
+ - **High-Throughput Edge Decoding & 30% Lower 32k RAM (M1 MacBook Pro — Native MLX 4-Bit):** Generates tokens at **99.27 tok/s** (1,024 context) and sustains **75.54 tok/s** at 32,768 context (vs **71.28 tok/s** for `Qwen3.5-2B`) while cutting 32k peak unified memory by **1.48 GB** (**3.52 GB** vs **5.00 GB**).
54
+ - **44% Faster Workstation Decoding (NVIDIA CUDA — BF16):** Generates tokens at **65.4 tok/s** across all context windows compared to **45.6 tok/s** for `Qwen3.5-2B-Base`.
55
 
56
+ #### 1. Apple Silicon Edge Benchmark (MacBook Pro M1 — Native MLX 4-Bit)
57
 
58
  | Context Length | Model (Backend) | TTFT (ms) | Prefill Throughput | Decode Speed | Peak RAM |
59
  | :---: | :--- | :---: | :---: | :---: | :---: |
60
+ | **512** | **BareTorch-2.5B (MLX 4-bit)** | **437.9 ms** | **1,169.3 tok/s** | **99.06 tok/s** | **1.99 GB** |
61
+ | 512 | Qwen3.5-2B (MLX 4-bit) | 576.4 ms | 888.2 tok/s | 98.82 tok/s | 1.84 GB |
62
+ | **1,024** | **BareTorch-2.5B (MLX 4-bit)** | **855.5 ms** | **1,197.0 tok/s** | **99.27 tok/s** | **2.28 GB** |
63
+ | 1,024 | Qwen3.5-2B (MLX 4-bit) | 1,140.1 ms | 898.2 tok/s | 99.07 tok/s | 2.10 GB |
64
+ | **2,048** | **BareTorch-2.5B (MLX 4-bit)** | **1,700.8 ms** | **1,204.1 tok/s** | **95.85 tok/s** | **2.41 GB** |
65
+ | 2,048 | Qwen3.5-2B (MLX 4-bit) | 2,271.7 ms | 901.5 tok/s | 96.63 tok/s | 2.32 GB |
66
+ | **4,096** | **BareTorch-2.5B (MLX 4-bit)** | **3,438.4 ms** | **1,191.2 tok/s** | **93.63 tok/s** | **2.69 GB** |
67
+ | 4,096 | Qwen3.5-2B (MLX 4-bit) | 4,617.7 ms | 887.0 tok/s | 94.39 tok/s | 2.76 GB |
68
+ | **8,192** | **BareTorch-2.5B (MLX 4-bit)** | **6,963.2 ms** | **1,176.5 tok/s** | **90.85 tok/s** | **2.69 GB** |
69
+ | 8,192 | Qwen3.5-2B (MLX 4-bit) | 9,386.1 ms | 872.8 tok/s | 90.65 tok/s | 3.18 GB |
70
+ | **16,384** | **BareTorch-2.5B (MLX 4-bit)** | **14,549.3 ms** | **1,126.1 tok/s** | **85.05 tok/s** | **3.33 GB** |
71
+ | 16,384 | Qwen3.5-2B (MLX 4-bit) | 19,503.7 ms | 840.0 tok/s | 83.17 tok/s | 3.75 GB |
72
+ | **32,768** | **BareTorch-2.5B (MLX 4-bit)** | **31,876.9 ms** | **1,028.0 tok/s** | **75.54 tok/s** | **3.52 GB** |
73
+ | 32,768 | Qwen3.5-2B (MLX 4-bit) | 42,567.7 ms | 769.8 tok/s | 71.28 tok/s | 5.00 GB |
74
 
75
  #### 2. Local NVIDIA GPU Workstation Benchmark (CUDA — BF16)
76
 
 
105
  journal={BareTorch Framework Laboratory Technical Report},
106
  year={2026},
107
  url={[https://github.com/martin-kbcc/baretorch](https://github.com/martin-kbcc/baretorch)}
108
+ }