Create README.md

README.md

@@ -0,0 +1,53 @@
+---
+language: multilingual
+tags:
+- ternary
+- robotics
+- multimodal
+- pretraining
+- jirack
+- ternarytransformer
+license: apache-2.0
+datasets:
+- CMSManhattan/JiRack-Pretrain-Dataset
+inference: false
+---
+
+# JiRack Robotics - TernaryTransformer3B (Pre-training Phase)
+
+**JiRack Robotics** has officially kicked off the **multi-shard pre-training phase** for its latest **3.3B parameter robotics model**.
+
+Running under the **JiRackTrain** pipeline on the enterprise infrastructure cluster (`root@jirack1`), the training initializes the next-generation **TernaryTransformer3B** architecture, tightly coupled with the advanced **JiRack Pro Tokenizer**.
+
+## Model Details
+
+- **Model Name**: TernaryTransformer3B (3.3 Billion Parameters)
+- **Architecture**: TernaryTransformer (custom ternary bit-response logic with GQA + MoE scaling foundations)
+- **Tokenizer**: [JiRack Pro Tokenizer (128K)](https://huggingface.co/CMSManhattan/JiRack-Pro-Tokenizer-128K)
+  - 347 active language editions of Wikipedia
+  - Specialized robotic action tokens + traditional text/vision embeddings
+- **Pre-training Dataset**: [JiRack-Pretrain-Dataset](https://huggingface.co/datasets/CMSManhattan/JiRack-Pretrain-Dataset)
+- **Training Objective**: From-scratch pre-training using the exact same multimodal corpus used to optimize the JiRack Pro Tokenizer (perfect vocabulary alignment)
+
+## Why This Alignment Matters
+
+By reusing the tokenizer's own training corpus for model pre-training, JiRack Robotics eliminates standard vocabulary bias found in most open-source LLMs. The ternary neural paths learn directly from a data distribution that maps perfectly to the token vocabulary boundaries. This is expected to deliver significantly higher deterministic accuracy for real-time robotic control policies, edge processing, and low-latency physical maneuver loops.
+
+## Training Setup & Technical Specifications
+
+The training leverages the custom `train_jirack_accelerate.py` framework with the following key paradigms:
+
+- **Memory Optimization**: Active **Gradient Checkpointing** across all TernaryTransformer3B layers (drastically reduced VRAM footprint)
+- **Data Layout**: Sequential pipeline distributed across **7 cloud shards** (`jirack_pretrain_chunk_0.pt` through `jirack_pretrain_chunk_6.pt`)
+- **Infrastructure**: Enterprise cluster with network-mounted storage (`/mnt/nfs_clientshare/JiRackTrain`)
+- **Framework**: PyTorch + Accelerate (Python 3.12)
+
+**Current Training Log Snapshot** (Shard 1/7):
+
+```bash
+Shards found for processing: 7
+Initializing JiRack architecture (3.3B) from your class...
+-> Gradient Checkpointing successfully activated on TernaryTransformer3B.
+Starting sequential training loop across shards...
+
+[Shard 1/7] Loading over network: jirack_pretrain_chunk_0.pt