Upload invention_description.md

invention_description.md

@@ -0,0 +1,84 @@
+# Technical Description of the Invention  
+
+**Invention Title:**  
+Method for Ternary-Quantized Transformer Optimization with Buffered Routing Embedding (BRE) and SwiGLU-Attention (SWA) Fusion for Low-VRAM Inference on Non-NVIDIA Hardware  
+
+**Inventor:** Konstantin Vladimirovich Grabko  
+**Contact:** grabko@cmsmanhattan.com | +1 (516) 777-0945  
+**Date of Conception:** December 2025  
+**Field of Invention:** Neural network architectures and optimization for efficient inference on non-NVIDIA hardware  
+**Confidentiality Notice:** Proprietary invention – not for public disclosure without a signed NDA.  
+
+---
+
+## 1. Background of the Invention  
+
+Conventional Large Language Models (LLMs) rely on high-precision floating-point formats (FP16/BF16), which demand significant memory bandwidth and VRAM, typically necessitating expensive NVIDIA H100/A100 hardware. Existing quantization methods (4-bit/8-bit) often introduce perplexity loss or latency due to complex dequantization steps.  
+
+This invention addresses bottlenecks for non-NVIDIA hardware (AMD ROCm) by reimagining the transformer architecture at the weight, routing, and kernel levels.  
+
+---
+
+## 2. Summary of the Invention  
+
+The invention consists of a three-tier optimization stack:  
+- **Ternary Quantization:** Mapping weights to $\{-1, 0, +1\}$.  
+- **Buffered Routing Embedding (BRE):** Optimizing how tokens access memory.  
+- **SwiGLU-Attention (SWA) Fusion:** Combining compute-heavy layers into a single hardware kernel.  
+
+---
+
+## 3. Detailed Method Steps  
+
+### Tier 1: Ternary Weight Optimization  
+
+The model weights $W$ are constrained to a ternary set using a learnable scaling factor $\gamma$:  
+
+$$W_{quant} = \gamma \cdot \text{sign}(\text{clip}(W, -1, 1))$$  
+
+- **Process:** During training, a Straight-Through Estimator (STE) is used to pass gradients through the non-differentiable quantization function.  
+- **Benefit:** Reduces weight storage by $\approx 70\%$, allowing a 3B parameter model to fit into 20GB VRAM.  
+
+---
+
+### Tier 2: Buffered Routing Embedding (BRE)  
+
+Unlike standard embeddings that load full tables into active memory, BRE implements a dynamic routing mechanism:  
+- **Step A:** Tokens are analyzed for frequency and importance.  
+- **Step B:** High-frequency embeddings are cached in a dedicated HBM buffer.  
+- **Step C:** A routing logic directs the attention mechanism to the buffer, minimizing global memory fetches (HBM-to-Cache).  
+
+---
+
+### Tier 3: SwiGLU-Attention (SWA) Fusion  
+
+In standard transformers, the Multi-Head Attention (MHA) and Feed-Forward Network (FFN) are separate operations. This invention fuses them:  
+- **Mechanism:** The SwiGLU activation logic is integrated directly into the attention computation cycle.  
+- **Hardware Target:** Optimized for AMD’s CDNA/RDNA architectures using HIP kernels.  
+- **Result:** Thermal stability is maintained below $80^\circ\text{C}$ by reducing redundant register write-backs.  
+
+---
+
+## 4. Technical Advantages  
+
+| **Feature**              | **Traditional Transformers** | **CMS Manhattan JiRack**          |
+|---------------------------|-----------------------------|-------------------------------------|
+| **VRAM Usage (3B Model)** | ~45-60 GB (FP16)            | ~20 GB (Ternary)                   |
+| **Hardware Requirement**  | NVIDIA Proprietary (CUDA)   | Hardware Agnostic (ROCm/HIP)       |
+| **Operating Temp**        | 85°C - 95°C                 | < 80°C                             |
+| **Memory Bottleneck**     | High (Global Memory Fetches)| Low (BRE-Buffered)                 |  
+
+---
+
+## 5. Claims (Summary)  
+
+1. A method for Ternary Quantization using learnable scaling $\gamma$ for transformer weights.  
+2. The architecture of Buffered Routing Embedding (BRE) for HBM memory management.  
+3. The fusion of SwiGLU and Attention layers into a single hardware-optimized kernel.  
+4. The application of these methods specifically for non-NVIDIA/ROCm inference pipelines.  
+
+---
+
+## 6. Conclusion  
+
+This invention represents a significant leap in "democratizing" AI, allowing state-of-the-art model performance on cost-effective, non-proprietary hardware without the traditional trade-offs in speed or accuracy.