=== [VON CLOUD TRAINING START] === Shutdown scheduled for Sat 2026-09-19 22:50:42 UTC, use 'shutdown -c' to cancel. downloading uv 0.12.17 x86_64-unknown-linux-gnu installing to /root/.local/bin uv uvx everything's installed! To add /root/.local/bin to your PATH, either restart your shell or run: source /root/.local/bin/env (sh, bash, zsh) source /root/.local/bin/env.fish (fish) Cloning into '/opt/von'... Downloading cpython-3.12.14-linux-x86_64-gnu (download) (32.6MiB) Downloaded cpython-3.12.14-linux-x86_64-gnu (download) Using CPython 3.12.14 Creating virtual environment at: .venv Activate with: source .venv/bin/activate Resolved 26 packages in 720ms Downloading networkx (2.0MiB) Downloading numpy (15.9MiB) Downloading sympy (5.9MiB) Downloading setuptools (1.2MiB) Downloading triton (199.8MiB) Downloading torch (744.2MiB) Downloading torchvision (6.9MiB) Downloading nvidia-nvjitlink-cu12 (37.9MiB) Downloading pillow (6.6MiB) Downloading nvidia-cudnn-cu12 (634.0MiB) Downloaded setuptools Downloading nvidia-cublas-cu12 (391.6MiB) Downloading nvidia-cusolver-cu12 (118.4MiB) Downloaded networkx Downloaded torchvision Downloading nvidia-cusparse-cu12 (186.9MiB) Downloaded pillow Downloading nvidia-cuda-nvrtc-cu12 (22.6MiB) Downloading nvidia-cufft-cu12 (116.0MiB) Downloading nvidia-cuda-cupti-cu12 (13.5MiB) Downloading nvidia-curand-cu12 (53.9MiB) Downloading nvidia-nccl-cu12 (179.9MiB) Downloaded numpy Downloaded sympy Downloaded nvidia-cuda-cupti-cu12 Downloaded nvidia-nvjitlink-cu12 Downloaded nvidia-cusolver-cu12 Downloaded nvidia-cuda-nvrtc-cu12 Downloaded nvidia-curand-cu12 Downloaded triton Downloaded nvidia-cufft-cu12 Downloaded nvidia-cusparse-cu12 Downloaded nvidia-nccl-cu12 Downloaded nvidia-cublas-cu12 Downloaded nvidia-cudnn-cu12 Downloaded torch Prepared 26 packages in 18.28s Installed 26 packages in 327ms + filelock==3.32.3 + fsspec==2026.7.0 + jinja2==3.1.6 + markupsafe==3.0.3 + mpmath==1.3.0 + networkx==3.6.1 + numpy==2.5.2 + nvidia-cublas-cu12==12.1.3.1 + nvidia-cuda-cupti-cu12==12.1.105 + nvidia-cuda-nvrtc-cu12==12.1.105 + nvidia-cuda-runtime-cu12==12.1.105 + nvidia-cudnn-cu12==9.1.0.70 + nvidia-cufft-cu12==11.0.2.54 + nvidia-curand-cu12==10.3.2.106 + nvidia-cusolver-cu12==11.4.5.107 + nvidia-cusparse-cu12==12.1.0.106 + nvidia-nccl-cu12==2.21.5 + nvidia-nvjitlink-cu12==12.9.86 + nvidia-nvtx-cu12==12.1.105 + pillow==12.3.0 + setuptools==78.1.0 + sympy==1.13.1 + torch==2.5.1+cu121 + torchvision==0.20.1+cu121 + triton==3.1.0 + typing-extensions==4.16.0 Resolved 77 packages in 562ms Downloading awscli (19.3MiB) Downloading transformers (11.7MiB) Downloading tokenizers (3.2MiB) Downloading pyarrow (47.8MiB) Downloading pygments (1.2MiB) Downloading scipy (33.7MiB) Downloading tiktoken (1.1MiB) Downloading aiohttp (1.7MiB) Downloading hf-xet (4.3MiB) Downloading pandas (10.3MiB) Downloading sentencepiece (1.3MiB) Downloaded sentencepiece Downloaded pygments Downloaded tiktoken Downloaded aiohttp Downloaded tokenizers Downloaded hf-xet Downloaded pandas Downloaded transformers Downloaded awscli Downloaded scipy Downloaded pyarrow Prepared 54 packages in 1.51s Uninstalled 1 package in 0.82ms Installed 54 packages in 489ms + accelerate==1.15.0 + aiohappyeyeballs==2.7.1 + aiohttp==3.14.3 + aiosignal==1.4.0 + annotated-doc==0.0.5 + anyio==4.15.1 + attrs==26.1.0 + awscli==1.46.1 + certifi==2026.7.22 + charset-normalizer==3.5.1 + click==8.5.0 + colorama==0.4.6 + datasets==5.0.1 + dill==0.4.1 + docutils==0.19 + frozenlist==1.8.0 - fsspec==2026.7.0 + fsspec==2026.6.0 + h11==0.16.0 + hf-xet==1.6.0 + httpcore==1.0.9 + httpx==0.28.1 + huggingface-hub==1.32.0 + idna==3.20 + jmespath==1.1.0 + markdown-it-py==4.2.0 + mdurl==0.1.2 + multidict==6.9.0 + multiprocess==0.70.19 + packaging==26.3 + pandas==3.0.6 + propcache==0.5.4 + psutil==7.2.2 + pyarrow==25.0.1 + pyasn1==0.6.4 + pygments==2.21.0 + python-dateutil==2.9.0.post0 + pyyaml==6.0.3 + regex==2026.9.10 + requests==2.34.2 + rich==15.0.0 + rsa==4.7.2 + safetensors==0.8.0 + scipy==1.18.1 + sentencepiece==0.2.2 + shellingham==1.5.4 + six==1.17.0 + tiktoken==0.14.0 + tokenizers==0.23.2 + tqdm==4.70.1 + transformers==5.17.0 + typer==0.27.2 + urllib3==2.8.0 + xxhash==4.0.1 + yarl==1.25.1 Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. ============================================================= Building Von-1.1 Operational Decision Dataset ============================================================= Synthesized 16000 Noul verification cases Synthesized 9998 Score ordinal cases Fetching mteb/banking77... Generating train split: 0%| | 0/9993 [00:00 Processed 9993 Banking77 Choice records Fetching dair-ai/emotion... Generating train split: 0%| | 0/16000 [00:00 Processed 10000 Emotion records Curated 19993 Choice routing cases Fetching ANLI & WANLI for reasoning core... Generating train_r1 split: 0%| | 0/16946 [00:00 Retained 20000 Adversarial Core reasoning pairs Total collected operational decision samples: 65,991 Final Train Set: 62,991 records Final Val Set: 3,000 records Successfully written to data_decision/train.jsonl and data_decision/val.jsonl! Detected 4 GPUs. Starting PyTorch DDP training... W0919 21:36:44.907000 2548 torch/distributed/run.py:793] W0919 21:36:44.907000 2548 torch/distributed/run.py:793] ***************************************** W0919 21:36:44.907000 2548 torch/distributed/run.py:793] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. W0919 21:36:44.907000 2548 torch/distributed/run.py:793] ***************************************** [W919 21:36:51.227088194 CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) [W919 21:36:51.228794801 CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) [W919 21:36:51.229604614 CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) [W919 21:36:51.229932051 CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator()) Device: cuda:0 (World Size: 4, DDP: True) Loading tokenizer & model: wfzyx/von-1.0... Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. Loading weights: 0%| | 0/174 [00:00 Train Samples: 62,991 -> Val Samples: 3,000 -> Micro Batch: 4 -> Grad Accum: 4 (Effective: 64) -> Epochs: 1 -> Total Steps: 985 Epoch [1/1] Step [100/3937] Loss: 0.3824 (CE: 0.3047) Acc: 100.0% Elapsed: 70.3s Epoch [1/1] Step [200/3937] Loss: 0.0024 (CE: 0.0024) Acc: 100.0% Elapsed: 140.1s Epoch [1/1] Step [300/3937] Loss: 0.2411 (CE: 0.1823) Acc: 100.0% Elapsed: 209.9s Epoch [1/1] Step [400/3937] Loss: 0.0003 (CE: 0.0003) Acc: 100.0% Elapsed: 279.1s Epoch [1/1] Step [500/3937] Loss: 1.4282 (CE: 1.2409) Acc: 75.0% Elapsed: 348.3s Epoch [1/1] Step [600/3937] Loss: 0.0255 (CE: 0.0242) Acc: 100.0% Elapsed: 418.0s Epoch [1/1] Step [700/3937] Loss: 0.0624 (CE: 0.0565) Acc: 100.0% Elapsed: 488.2s Epoch [1/1] Step [800/3937] Loss: 0.0044 (CE: 0.0043) Acc: 100.0% Elapsed: 557.6s Epoch [1/1] Step [900/3937] Loss: 0.0003 (CE: 0.0003) Acc: 100.0% Elapsed: 626.9s Epoch [1/1] Step [1000/3937] Loss: 0.0029 (CE: 0.0029) Acc: 100.0% Elapsed: 696.5s Epoch [1/1] Step [1100/3937] Loss: 0.6323 (CE: 0.4765) Acc: 75.0% Elapsed: 765.9s Epoch [1/1] Step [1200/3937] Loss: 0.0026 (CE: 0.0025) Acc: 100.0% Elapsed: 836.1s Epoch [1/1] Step [1300/3937] Loss: 0.0014 (CE: 0.0014) Acc: 100.0% Elapsed: 905.7s Epoch [1/1] Step [1400/3937] Loss: 0.0067 (CE: 0.0066) Acc: 100.0% Elapsed: 975.1s Epoch [1/1] Step [1500/3937] Loss: 0.0312 (CE: 0.0285) Acc: 100.0% Elapsed: 1045.6s Epoch [1/1] Step [1600/3937] Loss: 0.0048 (CE: 0.0048) Acc: 100.0% Elapsed: 1115.0s Epoch [1/1] Step [1700/3937] Loss: 0.5514 (CE: 0.4154) Acc: 75.0% Elapsed: 1184.2s Epoch [1/1] Step [1800/3937] Loss: 0.0008 (CE: 0.0008) Acc: 100.0% Elapsed: 1254.8s Epoch [1/1] Step [1900/3937] Loss: 0.0158 (CE: 0.0152) Acc: 100.0% Elapsed: 1324.9s Epoch [1/1] Step [2000/3937] Loss: 0.0000 (CE: 0.0000) Acc: 100.0% Elapsed: 1395.0s Epoch [1/1] Step [2100/3937] Loss: 1.6323 (CE: 1.2543) Acc: 50.0% Elapsed: 1465.1s Epoch [1/1] Step [2200/3937] Loss: 0.0026 (CE: 0.0026) Acc: 100.0% Elapsed: 1535.1s Epoch [1/1] Step [2300/3937] Loss: 0.0025 (CE: 0.0025) Acc: 100.0% Elapsed: 1604.7s Epoch [1/1] Step [2400/3937] Loss: 0.0017 (CE: 0.0017) Acc: 100.0% Elapsed: 1674.0s Epoch [1/1] Step [2500/3937] Loss: 1.4374 (CE: 1.1931) Acc: 75.0% Elapsed: 1744.3s Epoch [1/1] Step [2600/3937] Loss: 0.0169 (CE: 0.0160) Acc: 100.0% Elapsed: 1815.1s Epoch [1/1] Step [2700/3937] Loss: 0.0594 (CE: 0.0524) Acc: 100.0% Elapsed: 1885.7s Epoch [1/1] Step [2800/3937] Loss: 0.0001 (CE: 0.0001) Acc: 100.0% Elapsed: 1955.1s Epoch [1/1] Step [2900/3937] Loss: 0.0145 (CE: 0.0139) Acc: 100.0% Elapsed: 2025.5s Epoch [1/1] Step [3000/3937] Loss: 0.3584 (CE: 0.2598) Acc: 75.0% Elapsed: 2095.5s Epoch [1/1] Step [3100/3937] Loss: 0.0001 (CE: 0.0001) Acc: 100.0% Elapsed: 2165.5s Epoch [1/1] Step [3200/3937] Loss: 0.0008 (CE: 0.0008) Acc: 100.0% Elapsed: 2235.1s Epoch [1/1] Step [3300/3937] Loss: 1.1649 (CE: 0.9270) Acc: 75.0% Elapsed: 2304.9s Epoch [1/1] Step [3400/3937] Loss: 0.1889 (CE: 0.1420) Acc: 100.0% Elapsed: 2375.1s Epoch [1/1] Step [3500/3937] Loss: 0.5829 (CE: 0.4232) Acc: 75.0% Elapsed: 2444.9s Epoch [1/1] Step [3600/3937] Loss: 0.0006 (CE: 0.0006) Acc: 100.0% Elapsed: 2515.4s Epoch [1/1] Step [3700/3937] Loss: 0.0011 (CE: 0.0011) Acc: 100.0% Elapsed: 2585.0s Epoch [1/1] Step [3800/3937] Loss: 0.0207 (CE: 0.0198) Acc: 100.0% Elapsed: 2654.7s Epoch [1/1] Step [3900/3937] Loss: 0.0017 (CE: 0.0017) Acc: 100.0% Elapsed: 2724.3s Epoch [1/1] Step [3937/3937] Loss: 0.0313 (CE: 0.0288) Acc: 100.0% Elapsed: 2750.2s --- Epoch 1 Validation: Loss = 0.1211, Accuracy = 96.43% --- Saving new best checkpoint to checkpoints/von-modernbert-rlcd... Writing model shards: 0%| | 0/1 [00:00