JiRack Dense: Scalable Transformer Series (7B, 13B, 70B)
Author: Konstantin Vladimirovich Grabko
Organization: CMS Manhattan
Status: PATENT PENDING / PROPRIETARY TECHNOLOGY
Invention Class: High-Resolution Dense Architecture (V.1.2)
JiRack GPT 5 class
🚀 The Scalable Frontier
The JiRack Dense series provides a unified architectural framework for state-of-the-art language modeling. By utilizing SWA Fusion and Buffered Routing, these models achieve significantly higher throughput than standard Llama-based architectures.
Available Configurations
| Model | Parameters | Target Hardware | Optimization |
|---|---|---|---|
| JiRack 7B | 7.2 Billion | 1x RTX 3090/4090 | High-speed Edge Reasoning |
| JiRack 13B | 13.5 Billion | 1x A100 (40GB) | Advanced Logical Synthesis |
| JiRack 70B | 70.8 Billion | 4x - 8x H100 | Enterprise Flagship Performance |
🛠 Proprietary Core Technologies
1. SWA Fusion (SwiGLU-Attention)
The core of JiRack's speed. By fusing the Attention and SwiGLU FFN layers into a single computational kernel, we eliminate redundant memory R/W cycles.
- Benefit: 30% reduction in VRAM latency.
- Architecture: Integrated compute graph for AMD ROCm and NVIDIA CUDA.
2. BRE (Buffered Routing Embedding)
A hardware-aware embedding system designed for HBM3/4. BRE pre-fetches token weights into a local ring buffer based on predictive token sequencing.
- Benefit: Zero-latency embedding lookups even at maximum sequence lengths.
3. GQA Scaling
Optimized Grouped-Query Attention ratios (4:1 for 7B, 8:1 for 70B) to ensure the KV-cache remains manageable during long-context operations without degrading reasoning quality.
⚖️ Legal & Licensing Notice
NOTICE: PATENT PENDING
This repository contains proprietary technology owned by Konstantin Vladimirovich Grabko. Access is granted under the following conditions:
- Commercial Use: Requires a 5% Net Revenue royalty agreement.
- IP Protection: No reverse engineering of SWA kernels or BRE routing logic is permitted.
- No "Patent-Around": Licensees agree not to file IP claims based on the methods described herein.
- Attribution: Any derivative work must cite: "Powered by CMS Manhattan JiRack Technology. Inventor: Konstantin Vladimirovich Grabko."
Refer to license_dense.md for full legal documentation.
📦 Setup & Training
Environment
- Framework: PyTorch 2.3+
- Accelerator: AMD ROCm 6.0+ or NVIDIA CUDA 12.1+
- Distributed: Integrated support for DeepSpeed ZeRO-2/3.
Quick Start
To initialize a model from the factory:
from JiRack_Dense_Factory import get_jirack_config, JiRackPyTorch
# Initialize 13B configuration
config = get_jirack_config("13b")
model = JiRackPyTorch(config)