CooLLaMACEO commited on
Commit
e894826
·
verified ·
1 Parent(s): c516e70

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -1
README.md CHANGED
@@ -4,6 +4,41 @@ language:
4
  - en
5
  tags:
6
  - agent
 
 
 
 
 
7
  ---
8
 
9
- # Coming Soon.. (:<
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - en
5
  tags:
6
  - agent
7
+ - ternary
8
+ - 1.5-bit
9
+ - overflow
10
+ - large-scale
11
+ - efficiency
12
  ---
13
 
14
+ # 🌊 Overflow-1T
15
+
16
+ **Overflow-1T** is a next-generation, **1.03 Trillion parameter** Large Language Model built on a custom **1.5-bit Ternary ({-1, 0, 1}) architecture**.
17
+
18
+ By utilizing the **Overflow** architecture, this model achieves massive scale reasoning while remaining computationally efficient, designed specifically to run on consumer-grade hardware through advanced weight packing and specialized C++ inference kernels.
19
+
20
+ ## 🚀 Key Specifications
21
+ * **Parameters:** 1,000,000,000,000 (1T)
22
+ * **Precision:** 1.5-bit Ternary (packed 5-weights-per-byte)
23
+ * **Architecture:** OverflowForCausalLM
24
+ * **Layers:** 128
25
+ * **Hidden Size:** 16,384
26
+ * **Attention:** Grouped Query Attention (GQA) with 16 KV heads
27
+ * **Format:** `.safetensors` / `.bbuf` (Optimized for 1TSumerGPU)
28
+
29
+
30
+
31
+ ## 🛠 Project Status: Initial Sharding
32
+ We are currently in the process of sharding the 1.5-bit weights to the Hugging Face Hub.
33
+ - **Progress:** Shard 1 of 10 currently uploading.
34
+ - **Estimated Completion:** March 2026.
35
+
36
+ ## 🧠 Why 1.5-bit?
37
+ Unlike standard 1-bit models, Overflow-1T utilizes a **0-state** (Neutral weight). This allows the model to effectively "silence" noise across its 1T parameter space, leading to significantly higher stability in Chain-of-Thought (CoT) reasoning and logic tasks compared to binary 1-bit models.
38
+
39
+ ## 💻 Inference
40
+ This model is designed to be served using the **1TSumerGPU** engine, a custom C++ and CUDA-based inference framework optimized for NVIDIA RTX 40-series GPUs.
41
+
42
+ ---
43
+ **Created by CooLLaMACEO**
44
+ *Part of the Kwen Foundation initiatives.*