ViuAI commited on
Commit
6fe4289
·
verified ·
1 Parent(s): a726761

Update RTX 5090 Blackwell runner with 2-step setup instructions

Browse files
runners/RTX_5090_BLACKWELL_PRETRAIN.txt CHANGED
@@ -1,21 +1,15 @@
1
  # ==============================================================================
2
  # 🚀 ViuAI Sarus-500M — NVIDIA RTX 5090 Blackwell Turbo Pretraining (32GB VRAM)
3
  # ==============================================================================
4
- # Instructions:
5
- # 1. Start your RTX 5090 instance on Race Engineering (Delhi Datacenter).
6
- # 2. Open JupyterLab, paste this entire cell into a fresh notebook, and run (Shift + Enter).
7
- # 3. Performance Targets:
8
- # • Expected Speed: ~36,000 - 45,000 tokens/sec.
9
- # • 1 Step Time: ~12 - 14 seconds (524,288 tokens/step).
10
- # • 1,000 Steps (~524M tokens): ~3.6 - 4.0 Hours.
11
- # • Auto-resumes from Step 40,250 checkpoint on Hugging Face (ViuAI/ViuAI-500M).
12
- # • Streams 200 shards (10B tokens) from ViuAI/viuai-500m-data.
13
- # • Auto-uploads ckpt_latest.pt to Hugging Face in the background every 250 steps.
14
- # ==============================================================================
15
-
16
- # Ensure PyTorch with CUDA 12.8 (sm_120 Blackwell support) is installed
17
- !pip install -q --upgrade --index-url https://download.pytorch.org/whl/cu128 torch
18
- !pip install -q transformers datasets tokenizers huggingface_hub numpy hf_transfer
19
 
20
  import os, sys, shutil
21
  import torch
 
1
  # ==============================================================================
2
  # 🚀 ViuAI Sarus-500M — NVIDIA RTX 5090 Blackwell Turbo Pretraining (32GB VRAM)
3
  # ==============================================================================
4
+ # Instructions for JupyterLab on RTX 5090 (Blackwell sm_120):
5
+ #
6
+ # 📌 CELL 1: Run once to install CUDA 12.8 PyTorch with native sm_120 Blackwell support.
7
+ # ------------------------------------------------------------------------------
8
+ # !pip install --force-reinstall --no-deps -q torch --index-url https://download.pytorch.org/whl/cu128
9
+ # !pip install -q transformers datasets tokenizers huggingface_hub numpy hf_transfer
10
+ #
11
+ # 📌 CELL 2: Run to launch continuation pretraining to 60,000 steps!
12
+ # ------------------------------------------------------------------------------
 
 
 
 
 
 
13
 
14
  import os, sys, shutil
15
  import torch