| --- |
| language: multilingual |
| tags: |
| - ternary |
| - robotics |
| - multimodal |
| - pretraining |
| - jirack |
| - ternarytransformer |
| license: apache-2.0 |
| datasets: |
| - CMSManhattan/JiRack-Pretrain-Dataset |
| inference: false |
| --- |
| |
| # JiRack Robotics - TernaryTransformer3B (Pre-training Phase) |
|
|
| **JiRack Robotics** has officially kicked off the **multi-shard pre-training phase** for its latest **3B parameter robotics model**. |
|
|
| Running under the **JiRackTrain** pipeline on the enterprise infrastructure cluster (`root@jirack1`), the training initializes the next-generation **TernaryTransformer3B** architecture, tightly coupled with the advanced **JiRack Pro Tokenizer**. |
|
|
| ## Model Details |
|
|
| - **Model Name**: TernaryTransformer3B (3 Billion Parameters) |
| - **Architecture**: TernaryTransformer (custom ternary bit-response logic) |
| - **Tokenizer**: [JiRack Pro Tokenizer (128K)](https://huggingface.co/CMSManhattan/JiRack-Pro-Tokenizer-128K) — 347 active language editions of Wikipedia + specialized robotic action tokens |
| - **Pre-training Dataset**: [JiRack-Pretrain-Dataset](https://huggingface.co/datasets/CMSManhattan/JiRack-Pretrain-Dataset) |
|
|
| ## Key Metrics (Step ~838/1000 on Shard 7) on Blackwell 96 Gb VRAM |
|
|
| - **Iteration Speed**: 2.36s/it (network tensor loading) |
| - **Current Loss**: 1.2665 |
| - **Moving Average Loss**: 1.6637 |
| - **Perplexity (PPL)**: ppl=5.3 |
|
|
| ```bash |
| [Shard 8/100] jirack_pretrain_chunk_7.pt |
| Training jirack_pretrain_chunk_7.pt: 83%|█████▊ | 208/250 [08:09<01:38, 2.36s/it, loss=1.2665, avg_loss=1.6637, ppl=5.3, lr=2.00e-04] |
| [0] 0:python3* "mc [root@809e0ca1a59e" 19:45 04-Jul-26 |
| ``` |
|
|
| Engineers anticipate a **steep drop** in the perplexity curve within the first 250 iterations of Shard 1 as the ternary weights align with the tokenizer's token distributions. |
|
|
| ## Intended Use |
|
|
| This foundational model is being developed specifically for **robotics applications**: |
|
|
| - Real-time control policies |
| - Multimodal reasoning (text + vision + action tokens) |
| - Edge deployment with ternary efficiency |
| - Low-latency physical interaction loops |
|
|
| ## Monitoring & Updates |
|
|
| Training is actively progressing across all **7 shards**. Follow this repository or the linked tokenizer/dataset cards for checkpoint releases, evaluation results, and fine-tuned robotic variants. |
|
|
| --- |
|
|
| **Stay tuned** — the ternary robotics revolution is just getting started! 🚀 |
