language: multilingual
tags:
- ternary
- robotics
- multimodal
- pretraining
- jirack
- ternarytransformer
license: apache-2.0
datasets:
- CMSManhattan/JiRack-Pretrain-Dataset
inference: false
JiRack Robotics - TernaryTransformer3B (Pre-training Phase)
JiRack Robotics has officially kicked off the multi-shard pre-training phase for its latest 3B parameter robotics model.
Running under the JiRackTrain pipeline on the enterprise infrastructure cluster (root@jirack1), the training initializes the next-generation TernaryTransformer3B architecture, tightly coupled with the advanced JiRack Pro Tokenizer.
Model Details
- Model Name: TernaryTransformer3B (3 Billion Parameters)
- Architecture: TernaryTransformer (custom ternary bit-response logic)
- Tokenizer: JiRack Pro Tokenizer (128K) — 347 active language editions of Wikipedia + specialized robotic action tokens
- Pre-training Dataset: JiRack-Pretrain-Dataset
Key Metrics (Step ~838/1000 on Shard 7) on Blackwell 96 Gb VRAM
- Iteration Speed: 2.36s/it (network tensor loading)
- Current Loss: 1.2665
- Moving Average Loss: 1.6637
- Perplexity (PPL): ppl=5.3
[Shard 8/100] jirack_pretrain_chunk_7.pt
Training jirack_pretrain_chunk_7.pt: 83%|█████▊ | 208/250 [08:09<01:38, 2.36s/it, loss=1.2665, avg_loss=1.6637, ppl=5.3, lr=2.00e-04]
[0] 0:python3* "mc [root@809e0ca1a59e" 19:45 04-Jul-26
Engineers anticipate a steep drop in the perplexity curve within the first 250 iterations of Shard 1 as the ternary weights align with the tokenizer's token distributions.
Intended Use
This foundational model is being developed specifically for robotics applications:
- Real-time control policies
- Multimodal reasoning (text + vision + action tokens)
- Edge deployment with ternary efficiency
- Low-latency physical interaction loops
Monitoring & Updates
Training is actively progressing across all 7 shards. Follow this repository or the linked tokenizer/dataset cards for checkpoint releases, evaluation results, and fine-tuned robotic variants.
Stay tuned — the ternary robotics revolution is just getting started! 🚀