[22:34:48] billing guard armed: AUTO_STOP=1, watchdog 2h (pid 103) [22:34:48] repo present; pulling Already up to date. [22:34:49] installing deps WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 26.1.2 [notice] To update, run: python -m pip install --upgrade pip torch 2.4.1+cu124 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [22:35:02] building corpus: target 200000000 tokens from sample/10BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/10BT -> data/processed/train.bin ========================================================================== target 200.0M tokens (our 32k tokenizer) 14 shards in subset, 14 remaining [ 1/14] 000_00000.parquet 194,000 docs 201.4M tok in 444s (dl 8s) | total 201.4M (100.7%) | 453.6K tok/s | ETA -0.0h DONE: 201.4M tokens -> data/processed/train.bin (0.40 GB) shards used: 1 train with: py -m src.train.train --preset 500m --data data/processed/train.bin --loader memmap --resume auto [22:42:27] pretraining 500m for 200 steps (ctx 1024, mb 24 x2) corpus 201,402,133 tokens (train 200,395,123 / val 1,007,010) — memmap (0.40 GB, streamed) model 500m: 501.1M params starting fresh (no --resume) iter 0 | train 10.642 | val 9.907 | lr 2.00e-04 | 43,180 tok/s | 62.72 GB | 0.000B tok iter 1: 69,762 tok/s (0.70s/iter) iter 2: 69,721 tok/s (0.70s/iter) iter 3: 69,702 tok/s (0.71s/iter) iter 4: 69,682 tok/s (0.71s/iter) iter 5: 69,681 tok/s (0.71s/iter) iter 20 | train 7.722 | val 7.769 | lr 5.90e-04 | 58,491 tok/s | 66.73 GB | 0.001B tok iter 40 | train 7.651 | val 7.636 | lr 5.54e-04 | 61,657 tok/s | 66.73 GB | 0.002B tok iter 60 | train 7.212 | val 7.226 | lr 4.96e-04 | 62,815 tok/s | 66.73 GB | 0.003B tok iter 80 | train 6.893 | val 6.987 | lr 4.21e-04 | 63,415 tok/s | 66.73 GB | 0.004B tok iter 100 | train 6.764 | val 6.815 | lr 3.36e-04 | 62,773 tok/s | 66.73 GB | 0.005B tok iter 120 | train 6.656 | val 6.669 | lr 2.51e-04 | 63,172 tok/s | 66.73 GB | 0.006B tok iter 140 | train 6.528 | val 6.553 | lr 1.74e-04 | 63,460 tok/s | 66.73 GB | 0.007B tok iter 160 | train 6.468 | val 6.521 | lr 1.13e-04 | 63,678 tok/s | 66.73 GB | 0.008B tok iter 180 | train 6.483 | val 6.457 | lr 7.36e-05 | 63,286 tok/s | 66.73 GB | 0.009B tok iter 200 | train 6.416 | val 6.422 | lr 6.00e-05 | 63,470 tok/s | 66.73 GB | 0.010B tok training complete [22:45:30] SFT from checkpoints/500m_final.pt rendering 18,151 conversations -> ctx 1024 blocks... rendered 5,000/18,151 rendered 10,000/18,151 rendered 15,000/18,151 packed 5,079 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 5,079 blocks x 1024 = 5,200,896 tokens, 4,567,314 trainable (87.8%) train 4,978 blocks / val 101 blocks 155 steps/epoch x 3.0 epochs = 465 steps (warmup 13) model 500m: 501.1M params step 0/465 | train 10.626 | val 10.544 | lr 1.54e-06 | ep 0.00 | 43.54 GB step 25/465 | train 7.857 | val 7.738 | lr 2.00e-05 | ep 0.16 | 47.55 GB step 50/465 | train 6.971 | val 6.907 | lr 1.97e-05 | ep 0.32 | 47.55 GB step 75/465 | train 6.322 | val 6.407 | lr 1.92e-05 | ep 0.48 | 47.55 GB step 100/465 | train 6.125 | val 6.158 | lr 1.84e-05 | ep 0.65 | 47.55 GB step 125/465 | train 5.935 | val 6.009 | lr 1.74e-05 | ep 0.81 | 47.55 GB step 150/465 | train 5.832 | val 5.818 | lr 1.62e-05 | ep 0.97 | 47.55 GB step 175/465 | train 5.560 | val 5.687 | lr 1.49e-05 | ep 1.13 | 47.55 GB step 200/465 | train 5.518 | val 5.626 | lr 1.34e-05 | ep 1.29 | 47.55 GB step 225/465 | train 5.660 | val 5.564 | lr 1.19e-05 | ep 1.45 | 47.55 GB step 250/465 | train 5.300 | val 5.485 | lr 1.03e-05 | ep 1.61 | 47.55 GB step 275/465 | train 5.056 | val 5.548 | lr 8.77e-06 | ep 1.77 | 47.55 GB step 300/465 | train 5.104 | val 5.413 | lr 7.30e-06 | ep 1.94 | 47.55 GB step 325/465 | train 5.087 | val 5.396 | lr 5.94e-06 | ep 2.10 | 47.55 GB step 350/465 | train 5.152 | val 5.356 | lr 4.73e-06 | ep 2.26 | 47.55 GB step 375/465 | train 5.277 | val 5.368 | lr 3.70e-06 | ep 2.42 | 47.55 GB step 400/465 | train 5.050 | val 5.385 | lr 2.90e-06 | ep 2.58 | 47.55 GB step 425/465 | train 4.972 | val 5.404 | lr 2.35e-06 | ep 2.74 | 47.55 GB step 450/465 | train 5.001 | val 5.284 | lr 2.05e-06 | ep 2.90 | 47.55 GB SFT complete -> checkpoints/500m_sft_final.pt sample: '\nSame two two: B. So the same one of the whole trick an\' the first:\n- "Rap" (2) is the exact number (3), the door (1).\n\n**Same machine — the first and the rest more like the whole trick on the gears for the same.\n- "I\'d tell the same gear in the door or just an\' you\'re still the other — it\'s a touch.' [22:50:25] exporting to HuggingFace format (with logit-parity check) Traceback (most recent call last): File "/workspace/sprocket/scripts/export_hf.py", line 199, in main() File "/workspace/sprocket/scripts/export_hf.py", line 87, in main from transformers import LlamaConfig, LlamaForCausalLM File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 2342, in __getattr__ module = self._get_module(self._class_to_module[name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 2576, in _get_module raise e File "/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py", line 2574, in _get_module return importlib.import_module("." + module_name, self.__name__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/transformers/models/llama/configuration_llama.py", line 26, in from ...utils.type_validators import interval File "/usr/local/lib/python3.11/dist-packages/transformers/utils/type_validators.py", line 19, in from ..activations import ACT2FN File "/usr/local/lib/python3.11/dist-packages/transformers/activations.py", line 22, in from .integrations.hub_kernels import use_kernel_forward_from_hub File "/usr/local/lib/python3.11/dist-packages/transformers/integrations/hub_kernels.py", line 24, in from ..conversion_mapping import get_checkpoint_conversion_mapping, register_checkpoint_conversion_mapping File "/usr/local/lib/python3.11/dist-packages/transformers/conversion_mapping.py", line 21, in from .core_model_loading import ( File "/usr/local/lib/python3.11/dist-packages/transformers/core_model_loading.py", line 33, in from .distributed.sharding_utils import DtensorShardOperation, _dtensor_from_local_like File "/usr/local/lib/python3.11/dist-packages/transformers/distributed/sharding_utils.py", line 28, in from torch.distributed.tensor import DTensor ImportError: cannot import name 'DTensor' from 'torch.distributed.tensor' (/usr/local/lib/python3.11/dist-packages/torch/distributed/tensor/__init__.py) [22:50:28] !! PIPELINE FAILED (exit 1) — see the log above [22:50:28] !! state is on the volume; fix and relaunch to resume [22:50:28] TERMINATING POD — exit code 1 Runpod config file not found, please run `runpodctl config` to create it pod "5ebzicymj4tdm7" removed [22:57:55] billing guard armed: AUTO_STOP=1, watchdog 1h (pid 79) [22:57:55] repo present; pulling Already up to date. [22:57:56] installing deps WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. [notice] A new release of pip is available: 24.2 -> 26.1.2 [notice] To update, run: python -m pip install --upgrade pip torch 2.4.1+cu124 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [22:58:07] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.4.1+cu124 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 14,928 SFT conversations present ok SFT corpus shipped in the repo ok HuggingFace push (write access proven) preflight OK — safe to spend GPU time [22:58:12] building corpus: target 200000000 tokens from sample/10BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/10BT -> data/processed/train.bin ========================================================================== target 200.0M tokens (our 32k tokenizer) RESUMING: 1 shards done, 201.4M tokens (100.7% of target) 14 shards in subset, 13 remaining DONE: 201.4M tokens -> data/processed/train.bin (0.40 GB) shards used: 1 train with: py -m src.train.train --preset 500m --data data/processed/train.bin --loader memmap --resume auto [22:58:13] pretraining 500m for 200 steps (ctx 1024, mb 24 x2) corpus 201,402,133 tokens (train 200,395,123 / val 1,007,010) — memmap (0.40 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 200 >= --max-iters 200. Nothing to do. (Raise --max-iters to train further.) [22:58:37] SFT from checkpoints/500m_final.pt loaded packed cache data/processed/sft_packed.npz: 5,079 blocks corpus: 5,079 blocks x 1024 = 5,200,896 tokens, 4,567,314 trainable (87.8%) train 4,978 blocks / val 101 blocks 155 steps/epoch x 3.0 epochs = 465 steps (warmup 13) model 500m: 501.1M params RESUMED checkpoints/500m_sft_latest.pt at step 466/465 SFT complete -> checkpoints/500m_sft_final.pt sample: '\nSame two two: B. So the same one of the whole trick an\' the first:\n- "Rap" (2) is the exact number (3), the door (1).\n\n**Same machine — the first and the rest more like the whole trick on the gears for the same.\n- "I\'d tell the same gear in the door or just an\' you\'re still the other — it\'s a touch.' [22:59:27] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=1024 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [23:00:09] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 2.43MB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 2.43MB / 1.00GB  Processing Files (0 / 1) : 0%| | 2.43MB / 1.00GB, 236kB/s New Data Upload : 2%|▏ | 2.43MB / 134MB, 236kB/s  ...hf-500m/model.safetensors: 7%|▋ | 66.8MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 66.8MB / 1.00GB, 6.43MB/s New Data Upload : 33%|███▎ | 66.8MB / 201MB, 6.43MB/s  ...hf-500m/model.safetensors: 7%|▋ | 69.8MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 69.8MB / 1.00GB, 6.57MB/s New Data Upload : 35%|███▍ | 69.8MB / 201MB, 6.57MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 66%|██████▋ | 134MB / 201MB, 12.5MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 11.7MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 11.7MB/s  ...hf-500m/model.safetensors: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 17.6MB/s New Data Upload : 100%|█████████▉| 201MB / 201MB, 17.6MB/s  ...hf-500m/model.safetensors: 20%|██ | 201MB / 1.00GB  ...hf-500m/model.safetensors: 20%|██ | 205MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 205MB / 1.00GB, 17.2MB/s New Data Upload : 61%|██████ | 205MB / 335MB, 17.2MB/s  ...hf-500m/model.safetensors: 27%|██▋ | 268MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 268MB / 1.00GB, 22.5MB/s New Data Upload : 67%|██████▋ | 268MB / 402MB, 22.5MB/s  ...hf-500m/model.safetensors: 38%|███▊ | 383MB / 1.00GB  Processing Files (0 / 1) : 38%|███▊ | 383MB / 1.00GB, 32.4MB/s New Data Upload : 82%|████████▏ | 383MB / 469MB, 32.4MB/s  ...hf-500m/model.safetensors: 40%|████ | 402MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 402MB / 1.00GB, 33.4MB/s New Data Upload : 86%|████████▌ | 402MB / 469MB, 33.4MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 38.6MB/s New Data Upload : 87%|████████▋ | 469MB / 537MB, 38.6MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 37.8MB/s New Data Upload : 87%|████████▋ | 469MB / 537MB, 37.8MB/s  ...hf-500m/model.safetensors: 54%|█████▎ | 536MB / 1.00GB  Processing Files (0 / 1) : 54%|█████▎ | 536MB / 1.00GB, 42.9MB/s New Data Upload : 89%|████████▉ | 536MB / 604MB, 42.9MB/s  ...hf-500m/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 47.8MB/s New Data Upload : 90%|████████▉ | 603MB / 671MB, 47.8MB/s  ...hf-500m/model.safetensors: 60%|██████ | 605MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 605MB / 1.00GB, 46.0MB/s New Data Upload : 82%|████████▏ | 605MB / 738MB, 46.0MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 56.4MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 56.4MB/s  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 61.0MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 61.0MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 872MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 872MB / 1.00GB, 65.5MB/s New Data Upload : 100%|█████████▉| 872MB / 872MB, 65.5MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 872MB / 1.00GB  ...hf-500m/model.safetensors: 94%|█████████▎| 939MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 939MB / 1.00GB, 68.5MB/s New Data Upload : 94%|█████████▎| 939MB / 1.00GB, 68.5MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.4MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 70.9MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 70.9MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 69.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 69.4MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 66.5MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 66.5MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 65.1MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 65.1MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 65.1MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 65.1MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [23:00:21] DONE. [23:00:21] base ckpt : /workspace/checkpoints/500m_final.pt [23:00:21] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [23:00:21] hf model : /workspace/hf-500m [23:00:21] pipeline finished cleanly [23:00:21] TERMINATING POD — exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "droospbffwdo62" removed [00:34:10] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 75) [00:34:11] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M c8478ce..8ff92dc main -> origin/main Updating c8478ce..8ff92dc Fast-forward NEXT_SESSION_PROMPT.md | 187 +++++++++++++++++++++++++++++++-------------- cloud/bootstrap.sh | 85 +++++++++++++++++---- scripts/tune_throughput.py | 159 ++++++++++++++++++++++++++++++++++++++ src/model/model.py | 40 ++++++++-- src/train/sft.py | 17 ++++- src/train/sft_data.py | 74 ++++++++++++++++-- 6 files changed, 469 insertions(+), 93 deletions(-) create mode 100644 scripts/tune_throughput.py [00:34:14] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [00:34:25] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 14,928 SFT conversations present ok SFT corpus shipped in the repo ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [00:34:31] building corpus: target 20000000000 tokens from sample/100BT (resumable) manifest is for subset 'sample/10BT', you asked for 'sample/100BT'. Use a different --out, or delete the manifest. [00:34:31] !! PIPELINE FAILED (exit 1) - see the log above [00:34:31] !! state is on the volume; fix and relaunch to resume [00:34:31] TERMINATING POD - exit code 1 Runpod config file not found, please run `runpodctl config` to create it pod "y6gzs8cs0rjku4" removed [00:50:23] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 69) [00:50:23] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M 8ff92dc..aaac744 main -> origin/main Updating 8ff92dc..aaac744 Fast-forward cloud/bootstrap.sh | 38 +- data/safety/SOURCE.json | 7 + data/safety/dna_prompts.jsonl | 939 +++++++++++++++++++++++++++++++++++++ scripts/fetch_safety_prompts.py | 99 ++++ workflows/sprocket-safety-round.js | 132 ++++++ workflows/sprocket-sft-round.js | 276 ++++++++++- 6 files changed, 1481 insertions(+), 10 deletions(-) create mode 100644 data/safety/SOURCE.json create mode 100644 data/safety/dna_prompts.jsonl create mode 100644 scripts/fetch_safety_prompts.py create mode 100644 workflows/sprocket-safety-round.js [00:50:24] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [00:50:36] corpus file: data/processed/train_sample_100BT.bin [00:50:36] FRESH: archived previous 500m checkpoints -> /workspace/checkpoints/_archived_20260728_005036 [00:50:36] FRESH: cleared the packed-SFT cache; marker set, will not fire again [00:50:36] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 14,928 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [00:50:40] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) 140 shards in subset, 140 remaining [ 1/140] 000_00000.parquet 726,000 docs 758.8M tok in 328s (dl 8s) | total 758.8M (3.8%) | 2.3M tok/s | ETA 2.3h [ 2/140] 000_00001.parquet 727,000 docs 758.1M tok in 314s (dl 9s) | total 1.5B (7.6%) | 2.4M tok/s | ETA 2.2h [ 3/140] 000_00002.parquet 729,000 docs 758.4M tok in 310s (dl 8s) | total 2.3B (11.4%) | 2.4M tok/s | ETA 2.1h [ 4/140] 000_00003.parquet 725,000 docs 758.4M tok in 319s (dl 9s) | total 3.0B (15.2%) | 2.4M tok/s | ETA 2.0h [ 5/140] 000_00004.parquet 722,000 docs 758.5M tok in 333s (dl 8s) | total 3.8B (19.0%) | 2.4M tok/s | ETA 1.9h [ 6/140] 000_00005.parquet 725,000 docs 758.5M tok in 326s (dl 10s) | total 4.6B (22.8%) | 2.4M tok/s | ETA 1.8h [ 7/140] 000_00006.parquet 725,000 docs 758.2M tok in 321s (dl 9s) | total 5.3B (26.5%) | 2.4M tok/s | ETA 1.7h [ 8/140] 000_00007.parquet 726,000 docs 758.5M tok in 334s (dl 12s) | total 6.1B (30.3%) | 2.3M tok/s | ETA 1.6h [ 9/140] 000_00008.parquet 728,000 docs 758.7M tok in 309s (dl 14s) | total 6.8B (34.1%) | 2.4M tok/s | ETA 1.6h [ 10/140] 000_00009.parquet 725,000 docs 757.9M tok in 315s (dl 10s) | total 7.6B (37.9%) | 2.4M tok/s | ETA 1.5h [ 11/140] 001_00000.parquet 731,000 docs 758.0M tok in 322s (dl 9s) | total 8.3B (41.7%) | 2.4M tok/s | ETA 1.4h [ 12/140] 001_00001.parquet 733,000 docs 757.0M tok in 321s (dl 10s) | total 9.1B (45.5%) | 2.4M tok/s | ETA 1.3h [ 13/140] 001_00002.parquet 731,000 docs 757.8M tok in 317s (dl 8s) | total 9.9B (49.3%) | 2.4M tok/s | ETA 1.2h [ 14/140] 001_00003.parquet 734,000 docs 757.2M tok in 317s (dl 10s) | total 10.6B (53.1%) | 2.4M tok/s | ETA 1.1h [ 15/140] 001_00004.parquet 735,000 docs 756.9M tok in 326s (dl 9s) | total 11.4B (56.9%) | 2.4M tok/s | ETA 1.0h [ 16/140] 001_00005.parquet 730,000 docs 757.3M tok in 326s (dl 10s) | total 12.1B (60.6%) | 2.4M tok/s | ETA 0.9h [ 17/140] 001_00006.parquet 728,000 docs 757.4M tok in 317s (dl 9s) | total 12.9B (64.4%) | 2.4M tok/s | ETA 0.8h [ 18/140] 001_00007.parquet 728,000 docs 757.7M tok in 328s (dl 9s) | total 13.6B (68.2%) | 2.4M tok/s | ETA 0.7h [ 19/140] 001_00008.parquet 729,000 docs 758.1M tok in 320s (dl 10s) | total 14.4B (72.0%) | 2.4M tok/s | ETA 0.7h [ 20/140] 001_00009.parquet 731,000 docs 757.8M tok in 275s (dl 9s) | total 15.2B (75.8%) | 2.4M tok/s | ETA 0.6h [ 21/140] 002_00000.parquet 730,000 docs 757.5M tok in 272s (dl 9s) | total 15.9B (79.6%) | 2.4M tok/s | ETA 0.5h [ 22/140] 002_00001.parquet 727,000 docs 756.6M tok in 296s (dl 9s) | total 16.7B (83.4%) | 2.4M tok/s | ETA 0.4h [ 23/140] 002_00002.parquet 726,000 docs 757.1M tok in 313s (dl 9s) | total 17.4B (87.2%) | 2.4M tok/s | ETA 0.3h [ 24/140] 002_00003.parquet 727,000 docs 757.5M tok in 313s (dl 8s) | total 18.2B (90.9%) | 2.4M tok/s | ETA 0.2h [ 25/140] 002_00004.parquet 730,000 docs 756.6M tok in 314s (dl 9s) | total 18.9B (94.7%) | 2.4M tok/s | ETA 0.1h [ 26/140] 002_00005.parquet 730,000 docs 757.4M tok in 315s (dl 8s) | total 19.7B (98.5%) | 2.4M tok/s | ETA 0.0h [ 27/140] 002_00006.parquet 288,000 docs 299.6M tok in 141s (dl 15s) | total 20.0B (100.0%) | 2.4M tok/s | ETA -0.0h DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [03:09:44] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params starting fresh (no --resume) iter 0 | train 10.630 | val 10.612 | lr 3.93e-07 | 3,770 tok/s | 50.59 GB | 0.000B tok iter 1: 116,740 tok/s (1.12s/iter) iter 2: 116,271 tok/s (1.13s/iter) iter 3: 116,289 tok/s (1.13s/iter) iter 4: 115,958 tok/s (1.13s/iter) iter 5: 115,913 tok/s (1.13s/iter) iter 20 | train 8.789 | val 8.694 | lr 8.26e-06 | 35,717 tok/s | 54.64 GB | 0.003B tok iter 40 | train 8.368 | val 8.189 | lr 1.61e-05 | 53,441 tok/s | 54.64 GB | 0.005B tok iter 60 | train 7.690 | val 7.651 | lr 2.40e-05 | 64,418 tok/s | 54.64 GB | 0.008B tok iter 80 | train 7.340 | val 7.259 | lr 3.18e-05 | 71,894 tok/s | 54.64 GB | 0.011B tok iter 100 | train 6.938 | val 6.944 | lr 3.97e-05 | 76,611 tok/s | 54.64 GB | 0.013B tok iter 120 | train 6.695 | val 6.721 | lr 4.76e-05 | 80,759 tok/s | 54.64 GB | 0.016B tok iter 140 | train 6.494 | val 6.521 | lr 5.54e-05 | 84,013 tok/s | 54.64 GB | 0.018B tok iter 160 | train 6.276 | val 6.387 | lr 6.33e-05 | 86,633 tok/s | 54.64 GB | 0.021B tok iter 180 | train 6.206 | val 6.238 | lr 7.12e-05 | 88,251 tok/s | 54.64 GB | 0.024B tok iter 200 | train 6.033 | val 6.069 | lr 7.90e-05 | 90,097 tok/s | 54.64 GB | 0.026B tok iter 220 | train 5.861 | val 5.994 | lr 8.69e-05 | 91,668 tok/s | 54.64 GB | 0.029B tok iter 240 | train 5.848 | val 5.874 | lr 9.48e-05 | 93,020 tok/s | 54.64 GB | 0.032B tok iter 260 | train 5.735 | val 5.759 | lr 1.03e-04 | 93,617 tok/s | 54.64 GB | 0.034B tok iter 280 | train 5.705 | val 5.651 | lr 1.10e-04 | 94,682 tok/s | 54.64 GB | 0.037B tok iter 300 | train 5.605 | val 5.618 | lr 1.18e-04 | 95,628 tok/s | 54.64 GB | 0.039B tok iter 320 | train 5.487 | val 5.551 | lr 1.26e-04 | 96,464 tok/s | 54.64 GB | 0.042B tok iter 340 | train 5.537 | val 5.446 | lr 1.34e-04 | 96,865 tok/s | 54.64 GB | 0.045B tok iter 360 | train 5.389 | val 5.352 | lr 1.42e-04 | 97,563 tok/s | 54.64 GB | 0.047B tok iter 380 | train 5.239 | val 5.318 | lr 1.50e-04 | 98,125 tok/s | 54.64 GB | 0.050B tok iter 400 | train 5.289 | val 5.234 | lr 1.58e-04 | 98,703 tok/s | 54.64 GB | 0.053B tok iter 420 | train 5.126 | val 5.142 | lr 1.66e-04 | 98,937 tok/s | 54.64 GB | 0.055B tok iter 440 | train 4.946 | val 5.110 | lr 1.73e-04 | 99,436 tok/s | 54.64 GB | 0.058B tok checkpoint @ iter 456 -> checkpoints/500m_latest.pt iter 460 | train 5.036 | val 5.016 | lr 1.81e-04 | 97,770 tok/s | 54.64 GB | 0.060B tok iter 480 | train 5.115 | val 5.010 | lr 1.89e-04 | 98,269 tok/s | 54.64 GB | 0.063B tok iter 500 | train 4.915 | val 4.980 | lr 1.97e-04 | 98,478 tok/s | 54.64 GB | 0.066B tok checkpoint @ iter 500 -> checkpoints/500m_latest.pt iter 520 | train 4.869 | val 4.915 | lr 2.05e-04 | 95,096 tok/s | 54.64 GB | 0.068B tok iter 540 | train 4.864 | val 4.844 | lr 2.13e-04 | 95,610 tok/s | 54.64 GB | 0.071B tok iter 560 | train 4.690 | val 4.785 | lr 2.21e-04 | 96,088 tok/s | 54.64 GB | 0.074B tok iter 580 | train 4.731 | val 4.810 | lr 2.28e-04 | 96,315 tok/s | 54.64 GB | 0.076B tok iter 600 | train 4.782 | val 4.761 | lr 2.36e-04 | 96,744 tok/s | 54.64 GB | 0.079B tok iter 620 | train 4.562 | val 4.736 | lr 2.44e-04 | 97,150 tok/s | 54.64 GB | 0.081B tok iter 640 | train 4.497 | val 4.671 | lr 2.52e-04 | 97,535 tok/s | 54.64 GB | 0.084B tok iter 660 | train 4.648 | val 4.608 | lr 2.60e-04 | 97,717 tok/s | 54.64 GB | 0.087B tok iter 680 | train 4.518 | val 4.594 | lr 2.68e-04 | 98,066 tok/s | 54.64 GB | 0.089B tok iter 700 | train 4.476 | val 4.554 | lr 2.76e-04 | 98,398 tok/s | 54.64 GB | 0.092B tok iter 720 | train 4.472 | val 4.452 | lr 2.83e-04 | 98,709 tok/s | 54.64 GB | 0.095B tok iter 740 | train 4.395 | val 4.441 | lr 2.91e-04 | 98,837 tok/s | 54.64 GB | 0.097B tok iter 760 | train 4.431 | val 4.410 | lr 2.99e-04 | 99,122 tok/s | 54.64 GB | 0.100B tok iter 780 | train 4.285 | val 4.377 | lr 3.07e-04 | 99,397 tok/s | 54.64 GB | 0.102B tok iter 800 | train 4.469 | val 4.402 | lr 3.15e-04 | 99,660 tok/s | 54.64 GB | 0.105B tok iter 820 | train 4.102 | val 4.308 | lr 3.23e-04 | 99,755 tok/s | 54.64 GB | 0.108B tok iter 840 | train 4.349 | val 4.299 | lr 3.31e-04 | 99,997 tok/s | 54.64 GB | 0.110B tok iter 860 | train 4.298 | val 4.279 | lr 3.39e-04 | 100,225 tok/s | 54.64 GB | 0.113B tok iter 880 | train 4.214 | val 4.206 | lr 3.46e-04 | 100,450 tok/s | 54.64 GB | 0.115B tok iter 900 | train 4.203 | val 4.229 | lr 3.54e-04 | 100,520 tok/s | 54.64 GB | 0.118B tok iter 920 | train 4.204 | val 4.214 | lr 3.62e-04 | 100,620 tok/s | 54.64 GB | 0.121B tok iter 940 | train 4.080 | val 4.233 | lr 3.70e-04 | 100,823 tok/s | 54.64 GB | 0.123B tok iter 960 | train 4.087 | val 4.153 | lr 3.78e-04 | 101,019 tok/s | 54.64 GB | 0.126B tok iter 980 | train 4.067 | val 4.121 | lr 3.86e-04 | 101,073 tok/s | 54.64 GB | 0.129B tok checkpoint @ iter 988 -> checkpoints/500m_latest.pt iter 1000 | train 4.382 | val 4.131 | lr 3.94e-04 | 100,224 tok/s | 54.64 GB | 0.131B tok checkpoint @ iter 1000 -> checkpoints/500m_latest.pt iter 1020 | train 4.081 | val 4.069 | lr 4.01e-04 | 98,464 tok/s | 54.64 GB | 0.134B tok iter 1040 | train 4.133 | val 4.124 | lr 4.09e-04 | 98,684 tok/s | 54.64 GB | 0.136B tok iter 1060 | train 4.078 | val 4.117 | lr 4.17e-04 | 98,772 tok/s | 54.64 GB | 0.139B tok iter 1080 | train 4.116 | val 4.073 | lr 4.25e-04 | 98,974 tok/s | 54.64 GB | 0.142B tok iter 1100 | train 4.030 | val 4.052 | lr 4.33e-04 | 99,154 tok/s | 54.64 GB | 0.144B tok iter 1120 | train 4.046 | val 4.066 | lr 4.41e-04 | 99,346 tok/s | 54.64 GB | 0.147B tok iter 1140 | train 3.963 | val 3.976 | lr 4.49e-04 | 99,419 tok/s | 54.64 GB | 0.150B tok iter 1160 | train 4.183 | val 4.049 | lr 4.56e-04 | 99,600 tok/s | 54.64 GB | 0.152B tok iter 1180 | train 3.959 | val 3.981 | lr 4.64e-04 | 99,775 tok/s | 54.64 GB | 0.155B tok iter 1200 | train 3.952 | val 3.954 | lr 4.72e-04 | 99,943 tok/s | 54.64 GB | 0.157B tok iter 1220 | train 3.954 | val 3.973 | lr 4.80e-04 | 100,002 tok/s | 54.64 GB | 0.160B tok iter 1240 | train 3.955 | val 3.978 | lr 4.88e-04 | 100,164 tok/s | 54.64 GB | 0.163B tok iter 1260 | train 3.894 | val 3.928 | lr 4.96e-04 | 100,322 tok/s | 54.64 GB | 0.165B tok iter 1280 | train 3.933 | val 3.982 | lr 5.04e-04 | 100,456 tok/s | 54.64 GB | 0.168B tok iter 1300 | train 4.085 | val 3.940 | lr 5.12e-04 | 100,506 tok/s | 54.64 GB | 0.171B tok iter 1320 | train 3.948 | val 3.938 | lr 5.19e-04 | 100,649 tok/s | 54.64 GB | 0.173B tok iter 1340 | train 3.886 | val 3.933 | lr 5.27e-04 | 100,790 tok/s | 54.64 GB | 0.176B tok iter 1360 | train 3.840 | val 3.897 | lr 5.35e-04 | 100,927 tok/s | 54.64 GB | 0.178B tok iter 1380 | train 3.918 | val 3.876 | lr 5.43e-04 | 100,967 tok/s | 54.64 GB | 0.181B tok iter 1400 | train 4.006 | val 3.842 | lr 5.51e-04 | 101,098 tok/s | 54.64 GB | 0.184B tok iter 1420 | train 3.748 | val 3.843 | lr 5.59e-04 | 101,225 tok/s | 54.64 GB | 0.186B tok iter 1440 | train 3.934 | val 3.866 | lr 5.67e-04 | 101,261 tok/s | 54.64 GB | 0.189B tok iter 1460 | train 3.787 | val 3.880 | lr 5.74e-04 | 101,294 tok/s | 54.64 GB | 0.191B tok iter 1480 | train 3.848 | val 3.835 | lr 5.82e-04 | 101,416 tok/s | 54.64 GB | 0.194B tok checkpoint @ iter 1488 -> checkpoints/500m_latest.pt iter 1500 | train 3.821 | val 3.888 | lr 5.90e-04 | 100,849 tok/s | 54.64 GB | 0.197B tok checkpoint @ iter 1500 -> checkpoints/500m_latest.pt iter 1520 | train 3.859 | val 3.858 | lr 5.98e-04 | 99,698 tok/s | 54.64 GB | 0.199B tok iter 1540 | train 3.851 | val 3.867 | lr 6.00e-04 | 99,750 tok/s | 54.64 GB | 0.202B tok iter 1560 | train 3.882 | val 3.817 | lr 6.00e-04 | 99,881 tok/s | 54.64 GB | 0.205B tok iter 1580 | train 3.695 | val 3.824 | lr 6.00e-04 | 100,008 tok/s | 54.64 GB | 0.207B tok iter 1600 | train 3.886 | val 3.786 | lr 6.00e-04 | 100,133 tok/s | 54.64 GB | 0.210B tok iter 1620 | train 3.772 | val 3.808 | lr 6.00e-04 | 100,173 tok/s | 54.64 GB | 0.212B tok iter 1640 | train 3.897 | val 3.805 | lr 6.00e-04 | 100,293 tok/s | 54.64 GB | 0.215B tok iter 1660 | train 3.719 | val 3.771 | lr 6.00e-04 | 100,411 tok/s | 54.64 GB | 0.218B tok iter 1680 | train 3.766 | val 3.783 | lr 6.00e-04 | 100,526 tok/s | 54.64 GB | 0.220B tok iter 1700 | train 3.711 | val 3.828 | lr 6.00e-04 | 100,545 tok/s | 54.64 GB | 0.223B tok iter 1720 | train 3.665 | val 3.753 | lr 6.00e-04 | 100,654 tok/s | 54.64 GB | 0.226B tok iter 1740 | train 3.777 | val 3.755 | lr 6.00e-04 | 100,762 tok/s | 54.64 GB | 0.228B tok iter 1760 | train 3.628 | val 3.758 | lr 6.00e-04 | 100,868 tok/s | 54.64 GB | 0.231B tok iter 1780 | train 3.699 | val 3.734 | lr 6.00e-04 | 100,900 tok/s | 54.64 GB | 0.233B tok iter 1800 | train 3.919 | val 3.676 | lr 6.00e-04 | 101,002 tok/s | 54.64 GB | 0.236B tok iter 1820 | train 3.581 | val 3.749 | lr 6.00e-04 | 101,102 tok/s | 54.64 GB | 0.239B tok iter 1840 | train 3.668 | val 3.731 | lr 6.00e-04 | 101,201 tok/s | 54.64 GB | 0.241B tok iter 1860 | train 3.766 | val 3.719 | lr 6.00e-04 | 101,226 tok/s | 54.64 GB | 0.244B tok iter 1880 | train 3.671 | val 3.685 | lr 6.00e-04 | 101,321 tok/s | 54.64 GB | 0.247B tok iter 1900 | train 3.766 | val 3.654 | lr 6.00e-04 | 101,415 tok/s | 54.64 GB | 0.249B tok iter 1920 | train 3.724 | val 3.714 | lr 6.00e-04 | 101,507 tok/s | 54.64 GB | 0.252B tok iter 1940 | train 3.731 | val 3.655 | lr 6.00e-04 | 101,514 tok/s | 54.64 GB | 0.254B tok iter 1960 | train 3.734 | val 3.704 | lr 6.00e-04 | 101,603 tok/s | 54.64 GB | 0.257B tok iter 1980 | train 3.661 | val 3.676 | lr 6.00e-04 | 101,690 tok/s | 54.64 GB | 0.260B tok checkpoint @ iter 1990 -> checkpoints/500m_latest.pt iter 2000 | train 3.635 | val 3.667 | lr 6.00e-04 | 101,243 tok/s | 54.64 GB | 0.262B tok checkpoint @ iter 2000 -> checkpoints/500m_latest.pt iter 2020 | train 3.625 | val 3.650 | lr 6.00e-04 | 100,233 tok/s | 54.64 GB | 0.265B tok iter 2040 | train 3.651 | val 3.664 | lr 6.00e-04 | 100,330 tok/s | 54.64 GB | 0.268B tok iter 2060 | train 3.696 | val 3.585 | lr 6.00e-04 | 100,425 tok/s | 54.64 GB | 0.270B tok iter 2080 | train 3.748 | val 3.676 | lr 6.00e-04 | 100,517 tok/s | 54.64 GB | 0.273B tok iter 2100 | train 3.699 | val 3.644 | lr 6.00e-04 | 100,551 tok/s | 54.64 GB | 0.275B tok iter 2120 | train 3.668 | val 3.639 | lr 6.00e-04 | 100,642 tok/s | 54.64 GB | 0.278B tok iter 2140 | train 3.687 | val 3.615 | lr 6.00e-04 | 100,730 tok/s | 54.64 GB | 0.281B tok iter 2160 | train 3.661 | val 3.675 | lr 6.00e-04 | 100,817 tok/s | 54.64 GB | 0.283B tok iter 2180 | train 3.577 | val 3.619 | lr 6.00e-04 | 100,849 tok/s | 54.64 GB | 0.286B tok iter 2200 | train 3.612 | val 3.605 | lr 6.00e-04 | 100,934 tok/s | 54.64 GB | 0.288B tok iter 2220 | train 3.556 | val 3.564 | lr 6.00e-04 | 101,017 tok/s | 54.64 GB | 0.291B tok iter 2240 | train 3.516 | val 3.618 | lr 6.00e-04 | 101,099 tok/s | 54.64 GB | 0.294B tok iter 2260 | train 3.537 | val 3.611 | lr 6.00e-04 | 101,127 tok/s | 54.64 GB | 0.296B tok iter 2280 | train 3.505 | val 3.620 | lr 6.00e-04 | 101,207 tok/s | 54.64 GB | 0.299B tok iter 2300 | train 3.567 | val 3.619 | lr 6.00e-04 | 101,285 tok/s | 54.64 GB | 0.302B tok iter 2320 | train 3.489 | val 3.563 | lr 6.00e-04 | 101,362 tok/s | 54.64 GB | 0.304B tok iter 2340 | train 3.640 | val 3.595 | lr 6.00e-04 | 101,386 tok/s | 54.64 GB | 0.307B tok iter 2360 | train 3.527 | val 3.567 | lr 6.00e-04 | 101,461 tok/s | 54.64 GB | 0.309B tok iter 2380 | train 3.466 | val 3.529 | lr 6.00e-04 | 101,535 tok/s | 54.64 GB | 0.312B tok iter 2400 | train 3.527 | val 3.547 | lr 6.00e-04 | 101,608 tok/s | 54.64 GB | 0.315B tok iter 2420 | train 3.530 | val 3.599 | lr 6.00e-04 | 101,630 tok/s | 54.64 GB | 0.317B tok iter 2440 | train 3.649 | val 3.549 | lr 6.00e-04 | 101,700 tok/s | 54.64 GB | 0.320B tok iter 2460 | train 3.499 | val 3.583 | lr 6.00e-04 | 101,770 tok/s | 54.64 GB | 0.323B tok iter 2480 | train 3.626 | val 3.516 | lr 6.00e-04 | 101,838 tok/s | 54.64 GB | 0.325B tok checkpoint @ iter 2490 -> checkpoints/500m_latest.pt iter 2500 | train 3.511 | val 3.575 | lr 6.00e-04 | 101,438 tok/s | 54.64 GB | 0.328B tok checkpoint @ iter 2500 -> checkpoints/500m_latest.pt iter 2520 | train 3.553 | val 3.575 | lr 6.00e-04 | 100,665 tok/s | 54.64 GB | 0.330B tok iter 2540 | train 3.466 | val 3.552 | lr 6.00e-04 | 100,741 tok/s | 54.64 GB | 0.333B tok iter 2560 | train 3.597 | val 3.547 | lr 6.00e-04 | 100,814 tok/s | 54.64 GB | 0.336B tok iter 2580 | train 3.542 | val 3.526 | lr 6.00e-04 | 100,840 tok/s | 54.64 GB | 0.338B tok iter 2600 | train 3.511 | val 3.539 | lr 6.00e-04 | 100,912 tok/s | 54.64 GB | 0.341B tok iter 2620 | train 3.378 | val 3.494 | lr 6.00e-04 | 100,983 tok/s | 54.64 GB | 0.344B tok iter 2640 | train 3.411 | val 3.458 | lr 6.00e-04 | 101,052 tok/s | 54.64 GB | 0.346B tok iter 2660 | train 3.474 | val 3.521 | lr 6.00e-04 | 101,077 tok/s | 54.64 GB | 0.349B tok iter 2680 | train 3.553 | val 3.500 | lr 6.00e-04 | 101,145 tok/s | 54.64 GB | 0.351B tok iter 2700 | train 3.595 | val 3.486 | lr 6.00e-04 | 101,211 tok/s | 54.64 GB | 0.354B tok iter 2720 | train 3.614 | val 3.461 | lr 6.00e-04 | 101,277 tok/s | 54.64 GB | 0.357B tok iter 2740 | train 3.389 | val 3.546 | lr 6.00e-04 | 101,300 tok/s | 54.64 GB | 0.359B tok iter 2760 | train 3.316 | val 3.545 | lr 6.00e-04 | 101,364 tok/s | 54.64 GB | 0.362B tok iter 2780 | train 3.290 | val 3.502 | lr 6.00e-04 | 101,420 tok/s | 54.64 GB | 0.365B tok iter 2800 | train 3.528 | val 3.488 | lr 6.00e-04 | 101,482 tok/s | 54.64 GB | 0.367B tok iter 2820 | train 3.516 | val 3.489 | lr 6.00e-04 | 101,502 tok/s | 54.64 GB | 0.370B tok iter 2840 | train 3.354 | val 3.530 | lr 6.00e-04 | 101,564 tok/s | 54.64 GB | 0.372B tok iter 2860 | train 3.524 | val 3.492 | lr 6.00e-04 | 101,625 tok/s | 54.64 GB | 0.375B tok iter 2880 | train 3.589 | val 3.503 | lr 6.00e-04 | 101,685 tok/s | 54.64 GB | 0.378B tok iter 2900 | train 3.449 | val 3.539 | lr 6.00e-04 | 101,703 tok/s | 54.64 GB | 0.380B tok iter 2920 | train 3.299 | val 3.435 | lr 6.00e-04 | 101,763 tok/s | 54.64 GB | 0.383B tok iter 2940 | train 3.569 | val 3.449 | lr 6.00e-04 | 101,821 tok/s | 54.64 GB | 0.385B tok iter 2960 | train 3.453 | val 3.471 | lr 6.00e-04 | 101,878 tok/s | 54.64 GB | 0.388B tok iter 2980 | train 3.511 | val 3.463 | lr 6.00e-04 | 101,894 tok/s | 54.64 GB | 0.391B tok checkpoint @ iter 2990 -> checkpoints/500m_latest.pt iter 3000 | train 3.517 | val 3.460 | lr 6.00e-04 | 101,593 tok/s | 54.64 GB | 0.393B tok checkpoint @ iter 3000 -> checkpoints/500m_latest.pt iter 3020 | train 3.563 | val 3.407 | lr 6.00e-04 | 100,977 tok/s | 54.64 GB | 0.396B tok iter 3040 | train 3.514 | val 3.446 | lr 6.00e-04 | 101,038 tok/s | 54.64 GB | 0.399B tok iter 3060 | train 3.492 | val 3.471 | lr 6.00e-04 | 101,060 tok/s | 54.64 GB | 0.401B tok iter 3080 | train 3.439 | val 3.430 | lr 6.00e-04 | 101,119 tok/s | 54.64 GB | 0.404B tok iter 3100 | train 3.237 | val 3.481 | lr 6.00e-04 | 101,178 tok/s | 54.64 GB | 0.406B tok iter 3120 | train 3.381 | val 3.411 | lr 6.00e-04 | 101,236 tok/s | 54.64 GB | 0.409B tok iter 3140 | train 3.534 | val 3.477 | lr 6.00e-04 | 101,255 tok/s | 54.64 GB | 0.412B tok iter 3160 | train 3.399 | val 3.436 | lr 6.00e-04 | 101,313 tok/s | 54.64 GB | 0.414B tok iter 3180 | train 3.482 | val 3.407 | lr 6.00e-04 | 101,369 tok/s | 54.64 GB | 0.417B tok iter 3200 | train 3.364 | val 3.456 | lr 6.00e-04 | 101,424 tok/s | 54.64 GB | 0.420B tok iter 3220 | train 3.386 | val 3.477 | lr 6.00e-04 | 101,442 tok/s | 54.64 GB | 0.422B tok iter 3240 | train 3.456 | val 3.455 | lr 6.00e-04 | 101,497 tok/s | 54.64 GB | 0.425B tok iter 3260 | train 3.389 | val 3.466 | lr 6.00e-04 | 101,551 tok/s | 54.64 GB | 0.427B tok iter 3280 | train 3.397 | val 3.465 | lr 6.00e-04 | 101,604 tok/s | 54.64 GB | 0.430B tok iter 3300 | train 3.437 | val 3.397 | lr 6.00e-04 | 101,620 tok/s | 54.64 GB | 0.433B tok iter 3320 | train 3.424 | val 3.452 | lr 6.00e-04 | 101,672 tok/s | 54.64 GB | 0.435B tok iter 3340 | train 3.436 | val 3.397 | lr 6.00e-04 | 101,724 tok/s | 54.64 GB | 0.438B tok iter 3360 | train 3.429 | val 3.445 | lr 6.00e-04 | 101,775 tok/s | 54.64 GB | 0.441B tok iter 3380 | train 3.391 | val 3.462 | lr 6.00e-04 | 101,789 tok/s | 54.64 GB | 0.443B tok iter 3400 | train 3.494 | val 3.414 | lr 6.00e-04 | 101,839 tok/s | 54.64 GB | 0.446B tok iter 3420 | train 3.395 | val 3.427 | lr 6.00e-04 | 101,889 tok/s | 54.64 GB | 0.448B tok iter 3440 | train 3.275 | val 3.405 | lr 6.00e-04 | 101,937 tok/s | 54.64 GB | 0.451B tok iter 3460 | train 3.381 | val 3.357 | lr 6.00e-04 | 101,951 tok/s | 54.64 GB | 0.454B tok iter 3480 | train 3.413 | val 3.426 | lr 6.00e-04 | 101,999 tok/s | 54.64 GB | 0.456B tok checkpoint @ iter 3491 -> checkpoints/500m_latest.pt iter 3500 | train 3.403 | val 3.369 | lr 6.00e-04 | 101,740 tok/s | 54.64 GB | 0.459B tok checkpoint @ iter 3500 -> checkpoints/500m_latest.pt iter 3520 | train 3.342 | val 3.439 | lr 6.00e-04 | 101,196 tok/s | 54.64 GB | 0.462B tok iter 3540 | train 3.395 | val 3.345 | lr 6.00e-04 | 101,214 tok/s | 54.64 GB | 0.464B tok iter 3560 | train 3.350 | val 3.389 | lr 6.00e-04 | 101,265 tok/s | 54.64 GB | 0.467B tok iter 3580 | train 3.343 | val 3.396 | lr 6.00e-04 | 101,315 tok/s | 54.64 GB | 0.469B tok iter 3600 | train 3.437 | val 3.416 | lr 6.00e-04 | 101,365 tok/s | 54.64 GB | 0.472B tok iter 3620 | train 3.410 | val 3.423 | lr 6.00e-04 | 101,377 tok/s | 54.64 GB | 0.475B tok iter 3640 | train 3.334 | val 3.419 | lr 6.00e-04 | 101,426 tok/s | 54.64 GB | 0.477B tok iter 3660 | train 3.341 | val 3.399 | lr 6.00e-04 | 101,474 tok/s | 54.64 GB | 0.480B tok iter 3680 | train 3.373 | val 3.341 | lr 6.00e-04 | 101,521 tok/s | 54.64 GB | 0.482B tok iter 3700 | train 3.319 | val 3.371 | lr 6.00e-04 | 101,536 tok/s | 54.64 GB | 0.485B tok iter 3720 | train 3.494 | val 3.410 | lr 6.00e-04 | 101,583 tok/s | 54.64 GB | 0.488B tok iter 3740 | train 3.403 | val 3.389 | lr 6.00e-04 | 101,629 tok/s | 54.64 GB | 0.490B tok iter 3760 | train 3.338 | val 3.391 | lr 6.00e-04 | 101,676 tok/s | 54.64 GB | 0.493B tok iter 3780 | train 3.393 | val 3.332 | lr 6.00e-04 | 101,689 tok/s | 54.64 GB | 0.496B tok iter 3800 | train 3.353 | val 3.355 | lr 6.00e-04 | 101,735 tok/s | 54.64 GB | 0.498B tok iter 3820 | train 3.485 | val 3.343 | lr 6.00e-04 | 101,780 tok/s | 54.64 GB | 0.501B tok iter 3840 | train 3.529 | val 3.404 | lr 6.00e-04 | 101,824 tok/s | 54.64 GB | 0.503B tok iter 3860 | train 3.345 | val 3.327 | lr 6.00e-04 | 101,837 tok/s | 54.64 GB | 0.506B tok iter 3880 | train 3.334 | val 3.407 | lr 6.00e-04 | 101,881 tok/s | 54.64 GB | 0.509B tok iter 3900 | train 3.309 | val 3.393 | lr 6.00e-04 | 101,924 tok/s | 54.64 GB | 0.511B tok iter 3920 | train 3.325 | val 3.380 | lr 6.00e-04 | 101,967 tok/s | 54.64 GB | 0.514B tok iter 3940 | train 3.400 | val 3.394 | lr 6.00e-04 | 101,979 tok/s | 54.64 GB | 0.517B tok iter 3960 | train 3.205 | val 3.357 | lr 6.00e-04 | 102,021 tok/s | 54.64 GB | 0.519B tok iter 3980 | train 3.365 | val 3.346 | lr 6.00e-04 | 102,063 tok/s | 54.64 GB | 0.522B tok checkpoint @ iter 3990 -> checkpoints/500m_latest.pt iter 4000 | train 3.418 | val 3.354 | lr 6.00e-04 | 101,840 tok/s | 54.64 GB | 0.524B tok checkpoint @ iter 4000 -> checkpoints/500m_latest.pt iter 4020 | train 3.344 | val 3.359 | lr 6.00e-04 | 101,363 tok/s | 54.64 GB | 0.527B tok iter 4040 | train 3.347 | val 3.292 | lr 6.00e-04 | 101,408 tok/s | 54.64 GB | 0.530B tok iter 4060 | train 3.380 | val 3.297 | lr 6.00e-04 | 101,452 tok/s | 54.64 GB | 0.532B tok iter 4080 | train 3.247 | val 3.346 | lr 6.00e-04 | 101,495 tok/s | 54.64 GB | 0.535B tok iter 4100 | train 3.221 | val 3.349 | lr 6.00e-04 | 101,509 tok/s | 54.64 GB | 0.538B tok iter 4120 | train 3.393 | val 3.339 | lr 6.00e-04 | 101,552 tok/s | 54.64 GB | 0.540B tok iter 4140 | train 3.352 | val 3.370 | lr 6.00e-04 | 101,594 tok/s | 54.64 GB | 0.543B tok iter 4160 | train 3.250 | val 3.367 | lr 6.00e-04 | 101,636 tok/s | 54.64 GB | 0.545B tok iter 4180 | train 3.399 | val 3.332 | lr 6.00e-04 | 101,649 tok/s | 54.64 GB | 0.548B tok iter 4200 | train 3.465 | val 3.329 | lr 6.00e-04 | 101,690 tok/s | 54.64 GB | 0.551B tok iter 4220 | train 3.351 | val 3.312 | lr 6.00e-04 | 101,731 tok/s | 54.64 GB | 0.553B tok iter 4240 | train 3.301 | val 3.372 | lr 6.00e-04 | 101,772 tok/s | 54.64 GB | 0.556B tok iter 4260 | train 3.369 | val 3.335 | lr 6.00e-04 | 101,784 tok/s | 54.64 GB | 0.558B tok iter 4280 | train 3.399 | val 3.331 | lr 6.00e-04 | 101,824 tok/s | 54.64 GB | 0.561B tok iter 4300 | train 3.228 | val 3.321 | lr 6.00e-04 | 101,863 tok/s | 54.64 GB | 0.564B tok iter 4320 | train 3.229 | val 3.296 | lr 6.00e-04 | 101,903 tok/s | 54.64 GB | 0.566B tok iter 4340 | train 3.309 | val 3.316 | lr 6.00e-04 | 101,914 tok/s | 54.64 GB | 0.569B tok iter 4360 | train 3.306 | val 3.327 | lr 6.00e-04 | 101,953 tok/s | 54.64 GB | 0.572B tok iter 4380 | train 3.316 | val 3.317 | lr 6.00e-04 | 101,991 tok/s | 54.64 GB | 0.574B tok iter 4400 | train 3.318 | val 3.312 | lr 6.00e-04 | 102,026 tok/s | 54.64 GB | 0.577B tok iter 4420 | train 3.244 | val 3.358 | lr 6.00e-04 | 102,036 tok/s | 54.64 GB | 0.579B tok iter 4440 | train 3.298 | val 3.303 | lr 6.00e-04 | 102,073 tok/s | 54.64 GB | 0.582B tok iter 4460 | train 3.283 | val 3.308 | lr 5.99e-04 | 102,106 tok/s | 54.64 GB | 0.585B tok iter 4480 | train 3.308 | val 3.313 | lr 5.99e-04 | 102,143 tok/s | 54.64 GB | 0.587B tok checkpoint @ iter 4491 -> checkpoints/500m_latest.pt iter 4500 | train 3.303 | val 3.336 | lr 5.99e-04 | 101,927 tok/s | 54.64 GB | 0.590B tok checkpoint @ iter 4500 -> checkpoints/500m_latest.pt iter 4520 | train 3.586 | val 3.318 | lr 5.99e-04 | 101,514 tok/s | 54.64 GB | 0.593B tok iter 4540 | train 3.394 | val 3.283 | lr 5.99e-04 | 101,554 tok/s | 54.64 GB | 0.595B tok iter 4560 | train 3.315 | val 3.319 | lr 5.99e-04 | 101,592 tok/s | 54.64 GB | 0.598B tok iter 4580 | train 3.422 | val 3.342 | lr 5.99e-04 | 101,604 tok/s | 54.64 GB | 0.600B tok iter 4600 | train 3.305 | val 3.321 | lr 5.99e-04 | 101,642 tok/s | 54.64 GB | 0.603B tok iter 4620 | train 3.404 | val 3.298 | lr 5.99e-04 | 101,680 tok/s | 54.64 GB | 0.606B tok iter 4640 | train 3.327 | val 3.301 | lr 5.99e-04 | 101,717 tok/s | 54.64 GB | 0.608B tok iter 4660 | train 3.417 | val 3.312 | lr 5.99e-04 | 101,728 tok/s | 54.64 GB | 0.611B tok iter 4680 | train 3.327 | val 3.335 | lr 5.99e-04 | 101,765 tok/s | 54.64 GB | 0.614B tok iter 4700 | train 3.177 | val 3.273 | lr 5.99e-04 | 101,801 tok/s | 54.64 GB | 0.616B tok iter 4720 | train 3.323 | val 3.292 | lr 5.99e-04 | 101,837 tok/s | 54.64 GB | 0.619B tok iter 4740 | train 3.231 | val 3.310 | lr 5.99e-04 | 101,847 tok/s | 54.64 GB | 0.621B tok iter 4760 | train 3.282 | val 3.319 | lr 5.99e-04 | 101,883 tok/s | 54.64 GB | 0.624B tok iter 4780 | train 3.314 | val 3.296 | lr 5.99e-04 | 101,918 tok/s | 54.64 GB | 0.627B tok iter 4800 | train 3.310 | val 3.330 | lr 5.99e-04 | 101,953 tok/s | 54.64 GB | 0.629B tok iter 4820 | train 3.321 | val 3.298 | lr 5.99e-04 | 101,964 tok/s | 54.64 GB | 0.632B tok iter 4840 | train 3.312 | val 3.311 | lr 5.99e-04 | 101,999 tok/s | 54.64 GB | 0.635B tok iter 4860 | train 3.308 | val 3.269 | lr 5.99e-04 | 102,033 tok/s | 54.64 GB | 0.637B tok iter 4880 | train 3.228 | val 3.220 | lr 5.99e-04 | 102,067 tok/s | 54.64 GB | 0.640B tok iter 4900 | train 3.290 | val 3.319 | lr 5.99e-04 | 102,076 tok/s | 54.64 GB | 0.642B tok iter 4920 | train 3.332 | val 3.302 | lr 5.99e-04 | 102,110 tok/s | 54.64 GB | 0.645B tok iter 4940 | train 3.292 | val 3.306 | lr 5.99e-04 | 102,143 tok/s | 54.64 GB | 0.648B tok iter 4960 | train 3.247 | val 3.313 | lr 5.99e-04 | 102,176 tok/s | 54.64 GB | 0.650B tok iter 4980 | train 3.308 | val 3.255 | lr 5.99e-04 | 102,185 tok/s | 54.64 GB | 0.653B tok checkpoint @ iter 4991 -> checkpoints/500m_latest.pt iter 5000 | train 3.224 | val 3.309 | lr 5.99e-04 | 102,005 tok/s | 54.64 GB | 0.655B tok checkpoint @ iter 5000 -> checkpoints/500m_latest.pt iter 5020 | train 3.242 | val 3.294 | lr 5.99e-04 | 101,623 tok/s | 54.64 GB | 0.658B tok iter 5040 | train 3.444 | val 3.255 | lr 5.99e-04 | 101,659 tok/s | 54.64 GB | 0.661B tok iter 5060 | train 3.372 | val 3.286 | lr 5.99e-04 | 101,670 tok/s | 54.64 GB | 0.663B tok iter 5080 | train 3.093 | val 3.261 | lr 5.99e-04 | 101,704 tok/s | 54.64 GB | 0.666B tok iter 5100 | train 3.309 | val 3.236 | lr 5.99e-04 | 101,739 tok/s | 54.64 GB | 0.669B tok iter 5120 | train 3.280 | val 3.249 | lr 5.99e-04 | 101,773 tok/s | 54.64 GB | 0.671B tok iter 5140 | train 3.320 | val 3.260 | lr 5.99e-04 | 101,782 tok/s | 54.64 GB | 0.674B tok iter 5160 | train 3.264 | val 3.259 | lr 5.99e-04 | 101,816 tok/s | 54.64 GB | 0.676B tok iter 5180 | train 3.390 | val 3.278 | lr 5.99e-04 | 101,849 tok/s | 54.64 GB | 0.679B tok iter 5200 | train 3.328 | val 3.252 | lr 5.99e-04 | 101,882 tok/s | 54.64 GB | 0.682B tok iter 5220 | train 3.144 | val 3.252 | lr 5.99e-04 | 101,891 tok/s | 54.64 GB | 0.684B tok iter 5240 | train 3.308 | val 3.260 | lr 5.99e-04 | 101,923 tok/s | 54.64 GB | 0.687B tok iter 5260 | train 3.218 | val 3.231 | lr 5.99e-04 | 101,955 tok/s | 54.64 GB | 0.690B tok iter 5280 | train 3.398 | val 3.303 | lr 5.99e-04 | 101,987 tok/s | 54.64 GB | 0.692B tok iter 5300 | train 3.284 | val 3.253 | lr 5.99e-04 | 101,992 tok/s | 54.64 GB | 0.695B tok iter 5320 | train 3.183 | val 3.295 | lr 5.99e-04 | 102,024 tok/s | 54.64 GB | 0.697B tok iter 5340 | train 3.146 | val 3.273 | lr 5.99e-04 | 102,055 tok/s | 54.64 GB | 0.700B tok iter 5360 | train 3.261 | val 3.268 | lr 5.99e-04 | 102,086 tok/s | 54.64 GB | 0.703B tok iter 5380 | train 3.287 | val 3.269 | lr 5.99e-04 | 102,095 tok/s | 54.64 GB | 0.705B tok iter 5400 | train 3.312 | val 3.240 | lr 5.99e-04 | 102,125 tok/s | 54.64 GB | 0.708B tok iter 5420 | train 3.175 | val 3.270 | lr 5.99e-04 | 102,156 tok/s | 54.64 GB | 0.711B tok iter 5440 | train 3.242 | val 3.251 | lr 5.99e-04 | 102,186 tok/s | 54.64 GB | 0.713B tok iter 5460 | train 3.223 | val 3.234 | lr 5.99e-04 | 102,194 tok/s | 54.64 GB | 0.716B tok iter 5480 | train 3.236 | val 3.238 | lr 5.99e-04 | 102,224 tok/s | 54.64 GB | 0.718B tok checkpoint @ iter 5491 -> checkpoints/500m_latest.pt iter 5500 | train 3.174 | val 3.255 | lr 5.99e-04 | 102,061 tok/s | 54.64 GB | 0.721B tok checkpoint @ iter 5500 -> checkpoints/500m_latest.pt iter 5520 | train 3.236 | val 3.246 | lr 5.99e-04 | 101,723 tok/s | 54.64 GB | 0.724B tok iter 5540 | train 3.206 | val 3.254 | lr 5.99e-04 | 101,733 tok/s | 54.64 GB | 0.726B tok iter 5560 | train 3.212 | val 3.246 | lr 5.99e-04 | 101,764 tok/s | 54.64 GB | 0.729B tok iter 5580 | train 3.206 | val 3.236 | lr 5.99e-04 | 101,795 tok/s | 54.64 GB | 0.732B tok iter 5600 | train 3.176 | val 3.283 | lr 5.99e-04 | 101,826 tok/s | 54.64 GB | 0.734B tok iter 5620 | train 3.231 | val 3.253 | lr 5.99e-04 | 101,835 tok/s | 54.64 GB | 0.737B tok iter 5640 | train 3.231 | val 3.275 | lr 5.99e-04 | 101,859 tok/s | 54.64 GB | 0.739B tok iter 5660 | train 3.328 | val 3.229 | lr 5.99e-04 | 101,890 tok/s | 54.64 GB | 0.742B tok iter 5680 | train 3.266 | val 3.194 | lr 5.99e-04 | 101,920 tok/s | 54.64 GB | 0.745B tok iter 5700 | train 3.263 | val 3.280 | lr 5.99e-04 | 101,928 tok/s | 54.64 GB | 0.747B tok iter 5720 | train 3.222 | val 3.267 | lr 5.99e-04 | 101,957 tok/s | 54.64 GB | 0.750B tok iter 5740 | train 3.215 | val 3.189 | lr 5.99e-04 | 101,987 tok/s | 54.64 GB | 0.752B tok iter 5760 | train 3.210 | val 3.201 | lr 5.99e-04 | 102,016 tok/s | 54.64 GB | 0.755B tok iter 5780 | train 3.173 | val 3.222 | lr 5.99e-04 | 102,025 tok/s | 54.64 GB | 0.758B tok iter 5800 | train 3.191 | val 3.232 | lr 5.99e-04 | 102,054 tok/s | 54.64 GB | 0.760B tok iter 5820 | train 3.362 | val 3.189 | lr 5.99e-04 | 102,082 tok/s | 54.64 GB | 0.763B tok iter 5840 | train 3.193 | val 3.243 | lr 5.99e-04 | 102,110 tok/s | 54.64 GB | 0.766B tok iter 5860 | train 3.004 | val 3.238 | lr 5.99e-04 | 102,118 tok/s | 54.64 GB | 0.768B tok iter 5880 | train 3.344 | val 3.246 | lr 5.99e-04 | 102,146 tok/s | 54.64 GB | 0.771B tok iter 5900 | train 3.385 | val 3.241 | lr 5.99e-04 | 102,174 tok/s | 54.64 GB | 0.773B tok iter 5920 | train 3.114 | val 3.217 | lr 5.99e-04 | 102,202 tok/s | 54.64 GB | 0.776B tok iter 5940 | train 3.270 | val 3.266 | lr 5.99e-04 | 102,210 tok/s | 54.64 GB | 0.779B tok iter 5960 | train 3.266 | val 3.170 | lr 5.99e-04 | 102,237 tok/s | 54.64 GB | 0.781B tok iter 5980 | train 3.267 | val 3.217 | lr 5.99e-04 | 102,265 tok/s | 54.64 GB | 0.784B tok checkpoint @ iter 5991 -> checkpoints/500m_latest.pt iter 6000 | train 3.207 | val 3.241 | lr 5.99e-04 | 102,119 tok/s | 54.64 GB | 0.787B tok checkpoint @ iter 6000 -> checkpoints/500m_latest.pt iter 6020 | train 3.243 | val 3.181 | lr 5.99e-04 | 101,787 tok/s | 54.64 GB | 0.789B tok iter 6040 | train 3.168 | val 3.160 | lr 5.99e-04 | 101,816 tok/s | 54.64 GB | 0.792B tok iter 6060 | train 3.190 | val 3.209 | lr 5.99e-04 | 101,845 tok/s | 54.64 GB | 0.794B tok iter 6080 | train 3.222 | val 3.224 | lr 5.99e-04 | 101,873 tok/s | 54.64 GB | 0.797B tok iter 6100 | train 3.184 | val 3.272 | lr 5.99e-04 | 101,881 tok/s | 54.64 GB | 0.800B tok iter 6120 | train 3.303 | val 3.238 | lr 5.99e-04 | 101,909 tok/s | 54.64 GB | 0.802B tok iter 6140 | train 3.265 | val 3.220 | lr 5.99e-04 | 101,933 tok/s | 54.64 GB | 0.805B tok iter 6160 | train 3.266 | val 3.164 | lr 5.99e-04 | 101,961 tok/s | 54.64 GB | 0.808B tok iter 6180 | train 3.207 | val 3.222 | lr 5.99e-04 | 101,968 tok/s | 54.64 GB | 0.810B tok iter 6200 | train 3.209 | val 3.205 | lr 5.99e-04 | 101,995 tok/s | 54.64 GB | 0.813B tok iter 6220 | train 3.190 | val 3.232 | lr 5.99e-04 | 102,022 tok/s | 54.64 GB | 0.815B tok iter 6240 | train 3.168 | val 3.219 | lr 5.99e-04 | 102,049 tok/s | 54.64 GB | 0.818B tok iter 6260 | train 3.280 | val 3.218 | lr 5.99e-04 | 102,057 tok/s | 54.64 GB | 0.821B tok iter 6280 | train 3.192 | val 3.194 | lr 5.99e-04 | 102,084 tok/s | 54.64 GB | 0.823B tok iter 6300 | train 3.194 | val 3.254 | lr 5.99e-04 | 102,110 tok/s | 54.64 GB | 0.826B tok iter 6320 | train 3.217 | val 3.214 | lr 5.99e-04 | 102,136 tok/s | 54.64 GB | 0.829B tok iter 6340 | train 3.270 | val 3.180 | lr 5.99e-04 | 102,143 tok/s | 54.64 GB | 0.831B tok iter 6360 | train 3.207 | val 3.230 | lr 5.99e-04 | 102,169 tok/s | 54.64 GB | 0.834B tok iter 6380 | train 3.257 | val 3.197 | lr 5.99e-04 | 102,195 tok/s | 54.64 GB | 0.836B tok iter 6400 | train 3.135 | val 3.243 | lr 5.99e-04 | 102,221 tok/s | 54.64 GB | 0.839B tok iter 6420 | train 3.306 | val 3.178 | lr 5.99e-04 | 102,227 tok/s | 54.64 GB | 0.842B tok iter 6440 | train 3.210 | val 3.257 | lr 5.99e-04 | 102,253 tok/s | 54.64 GB | 0.844B tok iter 6460 | train 3.373 | val 3.184 | lr 5.99e-04 | 102,278 tok/s | 54.64 GB | 0.847B tok iter 6480 | train 3.220 | val 3.207 | lr 5.99e-04 | 102,290 tok/s | 54.64 GB | 0.849B tok checkpoint @ iter 6490 -> checkpoints/500m_latest.pt iter 6500 | train 3.344 | val 3.195 | lr 5.99e-04 | 102,137 tok/s | 54.64 GB | 0.852B tok checkpoint @ iter 6500 -> checkpoints/500m_latest.pt iter 6520 | train 3.192 | val 3.254 | lr 5.99e-04 | 101,842 tok/s | 54.64 GB | 0.855B tok iter 6540 | train 3.186 | val 3.243 | lr 5.99e-04 | 101,868 tok/s | 54.64 GB | 0.857B tok iter 6560 | train 3.261 | val 3.185 | lr 5.99e-04 | 101,894 tok/s | 54.64 GB | 0.860B tok iter 6580 | train 3.338 | val 3.209 | lr 5.99e-04 | 101,901 tok/s | 54.64 GB | 0.863B tok iter 6600 | train 3.113 | val 3.152 | lr 5.98e-04 | 101,927 tok/s | 54.64 GB | 0.865B tok iter 6620 | train 3.226 | val 3.170 | lr 5.98e-04 | 101,953 tok/s | 54.64 GB | 0.868B tok iter 6640 | train 3.046 | val 3.221 | lr 5.98e-04 | 101,978 tok/s | 54.64 GB | 0.870B tok iter 6660 | train 3.033 | val 3.209 | lr 5.98e-04 | 101,985 tok/s | 54.64 GB | 0.873B tok iter 6680 | train 3.201 | val 3.192 | lr 5.98e-04 | 102,011 tok/s | 54.64 GB | 0.876B tok iter 6700 | train 3.176 | val 3.222 | lr 5.98e-04 | 102,036 tok/s | 54.64 GB | 0.878B tok iter 6720 | train 3.254 | val 3.181 | lr 5.98e-04 | 102,061 tok/s | 54.64 GB | 0.881B tok iter 6740 | train 3.303 | val 3.199 | lr 5.98e-04 | 102,068 tok/s | 54.64 GB | 0.884B tok iter 6760 | train 3.262 | val 3.163 | lr 5.98e-04 | 102,092 tok/s | 54.64 GB | 0.886B tok iter 6780 | train 3.174 | val 3.231 | lr 5.98e-04 | 102,117 tok/s | 54.64 GB | 0.889B tok iter 6800 | train 3.208 | val 3.158 | lr 5.98e-04 | 102,141 tok/s | 54.64 GB | 0.891B tok iter 6820 | train 3.257 | val 3.169 | lr 5.98e-04 | 102,147 tok/s | 54.64 GB | 0.894B tok iter 6840 | train 3.187 | val 3.208 | lr 5.98e-04 | 102,171 tok/s | 54.64 GB | 0.897B tok iter 6860 | train 3.277 | val 3.161 | lr 5.98e-04 | 102,196 tok/s | 54.64 GB | 0.899B tok iter 6880 | train 3.178 | val 3.198 | lr 5.98e-04 | 102,220 tok/s | 54.64 GB | 0.902B tok iter 6900 | train 3.246 | val 3.145 | lr 5.98e-04 | 102,226 tok/s | 54.64 GB | 0.905B tok iter 6920 | train 3.090 | val 3.219 | lr 5.98e-04 | 102,250 tok/s | 54.64 GB | 0.907B tok iter 6940 | train 3.169 | val 3.154 | lr 5.98e-04 | 102,274 tok/s | 54.64 GB | 0.910B tok iter 6960 | train 3.067 | val 3.156 | lr 5.98e-04 | 102,297 tok/s | 54.64 GB | 0.912B tok iter 6980 | train 3.195 | val 3.206 | lr 5.98e-04 | 102,303 tok/s | 54.64 GB | 0.915B tok checkpoint @ iter 6991 -> checkpoints/500m_latest.pt iter 7000 | train 3.132 | val 3.173 | lr 5.98e-04 | 102,177 tok/s | 54.64 GB | 0.918B tok checkpoint @ iter 7000 -> checkpoints/500m_latest.pt iter 7020 | train 3.119 | val 3.200 | lr 5.98e-04 | 101,901 tok/s | 54.64 GB | 0.920B tok iter 7040 | train 3.139 | val 3.212 | lr 5.98e-04 | 101,925 tok/s | 54.64 GB | 0.923B tok iter 7060 | train 3.135 | val 3.191 | lr 5.98e-04 | 101,933 tok/s | 54.64 GB | 0.925B tok iter 7080 | train 3.280 | val 3.138 | lr 5.98e-04 | 101,957 tok/s | 54.64 GB | 0.928B tok iter 7100 | train 3.201 | val 3.184 | lr 5.98e-04 | 101,981 tok/s | 54.64 GB | 0.931B tok iter 7120 | train 3.075 | val 3.182 | lr 5.98e-04 | 102,002 tok/s | 54.64 GB | 0.933B tok iter 7140 | train 3.118 | val 3.162 | lr 5.98e-04 | 102,009 tok/s | 54.64 GB | 0.936B tok iter 7160 | train 3.163 | val 3.222 | lr 5.98e-04 | 102,032 tok/s | 54.64 GB | 0.939B tok iter 7180 | train 3.111 | val 3.217 | lr 5.98e-04 | 102,056 tok/s | 54.64 GB | 0.941B tok iter 7200 | train 3.319 | val 3.185 | lr 5.98e-04 | 102,079 tok/s | 54.64 GB | 0.944B tok iter 7220 | train 3.208 | val 3.209 | lr 5.98e-04 | 102,085 tok/s | 54.64 GB | 0.946B tok iter 7240 | train 3.077 | val 3.194 | lr 5.98e-04 | 102,108 tok/s | 54.64 GB | 0.949B tok iter 7260 | train 3.298 | val 3.174 | lr 5.98e-04 | 102,131 tok/s | 54.64 GB | 0.952B tok iter 7280 | train 3.171 | val 3.173 | lr 5.98e-04 | 102,154 tok/s | 54.64 GB | 0.954B tok iter 7300 | train 3.200 | val 3.213 | lr 5.98e-04 | 102,160 tok/s | 54.64 GB | 0.957B tok iter 7320 | train 3.210 | val 3.180 | lr 5.98e-04 | 102,182 tok/s | 54.64 GB | 0.960B tok iter 7340 | train 3.122 | val 3.146 | lr 5.98e-04 | 102,205 tok/s | 54.64 GB | 0.962B tok iter 7360 | train 3.175 | val 3.156 | lr 5.98e-04 | 102,227 tok/s | 54.64 GB | 0.965B tok iter 7380 | train 3.298 | val 3.151 | lr 5.98e-04 | 102,232 tok/s | 54.64 GB | 0.967B tok iter 7400 | train 3.229 | val 3.173 | lr 5.98e-04 | 102,254 tok/s | 54.64 GB | 0.970B tok iter 7420 | train 3.231 | val 3.157 | lr 5.98e-04 | 102,276 tok/s | 54.64 GB | 0.973B tok iter 7440 | train 3.203 | val 3.185 | lr 5.98e-04 | 102,298 tok/s | 54.64 GB | 0.975B tok iter 7460 | train 3.221 | val 3.199 | lr 5.98e-04 | 102,304 tok/s | 54.64 GB | 0.978B tok iter 7480 | train 3.077 | val 3.176 | lr 5.98e-04 | 102,326 tok/s | 54.64 GB | 0.981B tok checkpoint @ iter 7491 -> checkpoints/500m_latest.pt iter 7500 | train 3.080 | val 3.173 | lr 5.98e-04 | 102,207 tok/s | 54.64 GB | 0.983B tok checkpoint @ iter 7500 -> checkpoints/500m_latest.pt iter 7520 | train 3.214 | val 3.170 | lr 5.98e-04 | 101,968 tok/s | 54.64 GB | 0.986B tok iter 7540 | train 3.223 | val 3.196 | lr 5.98e-04 | 101,975 tok/s | 54.64 GB | 0.988B tok iter 7560 | train 3.078 | val 3.143 | lr 5.98e-04 | 101,998 tok/s | 54.64 GB | 0.991B tok iter 7580 | train 3.258 | val 3.143 | lr 5.98e-04 | 102,020 tok/s | 54.64 GB | 0.994B tok iter 7600 | train 3.156 | val 3.155 | lr 5.98e-04 | 102,042 tok/s | 54.64 GB | 0.996B tok iter 7620 | train 3.007 | val 3.155 | lr 5.98e-04 | 102,048 tok/s | 54.64 GB | 0.999B tok iter 7640 | train 3.234 | val 3.213 | lr 5.98e-04 | 102,071 tok/s | 54.64 GB | 1.002B tok iter 7660 | train 3.050 | val 3.195 | lr 5.98e-04 | 102,092 tok/s | 54.64 GB | 1.004B tok iter 7680 | train 3.091 | val 3.169 | lr 5.98e-04 | 102,114 tok/s | 54.64 GB | 1.007B tok iter 7700 | train 3.134 | val 3.186 | lr 5.98e-04 | 102,120 tok/s | 54.64 GB | 1.009B tok iter 7720 | train 3.147 | val 3.152 | lr 5.98e-04 | 102,142 tok/s | 54.64 GB | 1.012B tok iter 7740 | train 3.032 | val 3.134 | lr 5.98e-04 | 102,163 tok/s | 54.64 GB | 1.015B tok iter 7760 | train 3.261 | val 3.121 | lr 5.98e-04 | 102,185 tok/s | 54.64 GB | 1.017B tok iter 7780 | train 3.070 | val 3.172 | lr 5.98e-04 | 102,190 tok/s | 54.64 GB | 1.020B tok iter 7800 | train 3.189 | val 3.164 | lr 5.98e-04 | 102,212 tok/s | 54.64 GB | 1.022B tok iter 7820 | train 3.073 | val 3.181 | lr 5.98e-04 | 102,233 tok/s | 54.64 GB | 1.025B tok iter 7840 | train 3.181 | val 3.174 | lr 5.98e-04 | 102,254 tok/s | 54.64 GB | 1.028B tok iter 7860 | train 3.035 | val 3.178 | lr 5.98e-04 | 102,257 tok/s | 54.64 GB | 1.030B tok iter 7880 | train 3.194 | val 3.147 | lr 5.98e-04 | 102,278 tok/s | 54.64 GB | 1.033B tok iter 7900 | train 2.883 | val 3.141 | lr 5.98e-04 | 102,299 tok/s | 54.64 GB | 1.036B tok iter 7920 | train 3.264 | val 3.158 | lr 5.98e-04 | 102,320 tok/s | 54.64 GB | 1.038B tok iter 7940 | train 3.096 | val 3.167 | lr 5.98e-04 | 102,324 tok/s | 54.64 GB | 1.041B tok iter 7960 | train 3.077 | val 3.128 | lr 5.98e-04 | 102,345 tok/s | 54.64 GB | 1.043B tok iter 7980 | train 3.156 | val 3.156 | lr 5.98e-04 | 102,365 tok/s | 54.64 GB | 1.046B tok checkpoint @ iter 7992 -> checkpoints/500m_latest.pt iter 8000 | train 3.045 | val 3.159 | lr 5.98e-04 | 102,257 tok/s | 54.64 GB | 1.049B tok checkpoint @ iter 8000 -> checkpoints/500m_latest.pt iter 8020 | train 3.214 | val 3.183 | lr 5.98e-04 | 102,005 tok/s | 54.64 GB | 1.051B tok iter 8040 | train 3.111 | val 3.165 | lr 5.98e-04 | 102,027 tok/s | 54.64 GB | 1.054B tok iter 8060 | train 3.022 | val 3.140 | lr 5.98e-04 | 102,048 tok/s | 54.64 GB | 1.057B tok iter 8080 | train 3.153 | val 3.141 | lr 5.97e-04 | 102,069 tok/s | 54.64 GB | 1.059B tok iter 8100 | train 3.238 | val 3.170 | lr 5.97e-04 | 102,074 tok/s | 54.64 GB | 1.062B tok iter 8120 | train 3.202 | val 3.145 | lr 5.97e-04 | 102,095 tok/s | 54.64 GB | 1.064B tok iter 8140 | train 3.126 | val 3.213 | lr 5.97e-04 | 102,115 tok/s | 54.64 GB | 1.067B tok iter 8160 | train 3.083 | val 3.146 | lr 5.97e-04 | 102,135 tok/s | 54.64 GB | 1.070B tok iter 8180 | train 3.152 | val 3.163 | lr 5.97e-04 | 102,140 tok/s | 54.64 GB | 1.072B tok iter 8200 | train 3.196 | val 3.131 | lr 5.97e-04 | 102,161 tok/s | 54.64 GB | 1.075B tok iter 8220 | train 3.154 | val 3.143 | lr 5.97e-04 | 102,181 tok/s | 54.64 GB | 1.078B tok iter 8240 | train 3.063 | val 3.145 | lr 5.97e-04 | 102,201 tok/s | 54.64 GB | 1.080B tok iter 8260 | train 3.082 | val 3.144 | lr 5.97e-04 | 102,206 tok/s | 54.64 GB | 1.083B tok iter 8280 | train 3.117 | val 3.114 | lr 5.97e-04 | 102,223 tok/s | 54.64 GB | 1.085B tok iter 8300 | train 3.096 | val 3.150 | lr 5.97e-04 | 102,243 tok/s | 54.64 GB | 1.088B tok iter 8320 | train 3.160 | val 3.110 | lr 5.97e-04 | 102,263 tok/s | 54.64 GB | 1.091B tok iter 8340 | train 3.065 | val 3.113 | lr 5.97e-04 | 102,268 tok/s | 54.64 GB | 1.093B tok iter 8360 | train 3.163 | val 3.125 | lr 5.97e-04 | 102,287 tok/s | 54.64 GB | 1.096B tok iter 8380 | train 3.097 | val 3.111 | lr 5.97e-04 | 102,307 tok/s | 54.64 GB | 1.099B tok iter 8400 | train 3.265 | val 3.079 | lr 5.97e-04 | 102,326 tok/s | 54.64 GB | 1.101B tok iter 8420 | train 3.141 | val 3.170 | lr 5.97e-04 | 102,331 tok/s | 54.64 GB | 1.104B tok iter 8440 | train 3.111 | val 3.169 | lr 5.97e-04 | 102,350 tok/s | 54.64 GB | 1.106B tok iter 8460 | train 3.064 | val 3.152 | lr 5.97e-04 | 102,369 tok/s | 54.64 GB | 1.109B tok iter 8480 | train 3.079 | val 3.151 | lr 5.97e-04 | 102,388 tok/s | 54.64 GB | 1.112B tok checkpoint @ iter 8491 -> checkpoints/500m_latest.pt iter 8500 | train 3.249 | val 3.152 | lr 5.97e-04 | 102,273 tok/s | 54.64 GB | 1.114B tok checkpoint @ iter 8500 -> checkpoints/500m_latest.pt iter 8520 | train 3.157 | val 3.143 | lr 5.97e-04 | 102,046 tok/s | 54.64 GB | 1.117B tok iter 8540 | train 3.117 | val 3.153 | lr 5.97e-04 | 102,066 tok/s | 54.64 GB | 1.119B tok iter 8560 | train 3.006 | val 3.145 | lr 5.97e-04 | 102,086 tok/s | 54.64 GB | 1.122B tok iter 8580 | train 3.029 | val 3.117 | lr 5.97e-04 | 102,092 tok/s | 54.64 GB | 1.125B tok iter 8600 | train 3.079 | val 3.132 | lr 5.97e-04 | 102,111 tok/s | 54.64 GB | 1.127B tok iter 8620 | train 3.218 | val 3.133 | lr 5.97e-04 | 102,130 tok/s | 54.64 GB | 1.130B tok iter 8640 | train 3.094 | val 3.131 | lr 5.97e-04 | 102,149 tok/s | 54.64 GB | 1.133B tok iter 8660 | train 3.039 | val 3.138 | lr 5.97e-04 | 102,154 tok/s | 54.64 GB | 1.135B tok iter 8680 | train 3.149 | val 3.122 | lr 5.97e-04 | 102,171 tok/s | 54.64 GB | 1.138B tok iter 8700 | train 3.144 | val 3.148 | lr 5.97e-04 | 102,190 tok/s | 54.64 GB | 1.140B tok iter 8720 | train 3.202 | val 3.129 | lr 5.97e-04 | 102,209 tok/s | 54.64 GB | 1.143B tok iter 8740 | train 3.164 | val 3.177 | lr 5.97e-04 | 102,214 tok/s | 54.64 GB | 1.146B tok iter 8760 | train 3.113 | val 3.124 | lr 5.97e-04 | 102,232 tok/s | 54.64 GB | 1.148B tok iter 8780 | train 3.046 | val 3.086 | lr 5.97e-04 | 102,251 tok/s | 54.64 GB | 1.151B tok iter 8800 | train 2.999 | val 3.125 | lr 5.97e-04 | 102,269 tok/s | 54.64 GB | 1.154B tok iter 8820 | train 3.240 | val 3.081 | lr 5.97e-04 | 102,274 tok/s | 54.64 GB | 1.156B tok iter 8840 | train 3.020 | val 3.147 | lr 5.97e-04 | 102,293 tok/s | 54.64 GB | 1.159B tok iter 8860 | train 3.310 | val 3.124 | lr 5.97e-04 | 102,311 tok/s | 54.64 GB | 1.161B tok iter 8880 | train 3.029 | val 3.104 | lr 5.97e-04 | 102,329 tok/s | 54.64 GB | 1.164B tok iter 8900 | train 3.197 | val 3.077 | lr 5.97e-04 | 102,334 tok/s | 54.64 GB | 1.167B tok iter 8920 | train 3.022 | val 3.162 | lr 5.97e-04 | 102,352 tok/s | 54.64 GB | 1.169B tok iter 8940 | train 3.148 | val 3.109 | lr 5.97e-04 | 102,371 tok/s | 54.64 GB | 1.172B tok iter 8960 | train 3.165 | val 3.128 | lr 5.97e-04 | 102,389 tok/s | 54.64 GB | 1.175B tok iter 8980 | train 3.192 | val 3.110 | lr 5.97e-04 | 102,393 tok/s | 54.64 GB | 1.177B tok checkpoint @ iter 8990 -> checkpoints/500m_latest.pt iter 9000 | train 3.252 | val 3.154 | lr 5.97e-04 | 102,296 tok/s | 54.64 GB | 1.180B tok checkpoint @ iter 9000 -> checkpoints/500m_latest.pt iter 9020 | train 3.170 | val 3.090 | lr 5.97e-04 | 102,096 tok/s | 54.64 GB | 1.182B tok iter 9040 | train 3.201 | val 3.126 | lr 5.97e-04 | 102,115 tok/s | 54.64 GB | 1.185B tok iter 9060 | train 3.025 | val 3.126 | lr 5.97e-04 | 102,120 tok/s | 54.64 GB | 1.188B tok iter 9080 | train 3.182 | val 3.099 | lr 5.97e-04 | 102,138 tok/s | 54.64 GB | 1.190B tok iter 9100 | train 3.063 | val 3.132 | lr 5.97e-04 | 102,156 tok/s | 54.64 GB | 1.193B tok iter 9120 | train 3.049 | val 3.144 | lr 5.97e-04 | 102,174 tok/s | 54.64 GB | 1.196B tok iter 9140 | train 3.118 | val 3.110 | lr 5.97e-04 | 102,179 tok/s | 54.64 GB | 1.198B tok iter 9160 | train 3.336 | val 3.104 | lr 5.97e-04 | 102,197 tok/s | 54.64 GB | 1.201B tok iter 9180 | train 3.089 | val 3.067 | lr 5.97e-04 | 102,215 tok/s | 54.64 GB | 1.203B tok iter 9200 | train 3.046 | val 3.100 | lr 5.97e-04 | 102,233 tok/s | 54.64 GB | 1.206B tok iter 9220 | train 3.112 | val 3.180 | lr 5.97e-04 | 102,238 tok/s | 54.64 GB | 1.209B tok iter 9240 | train 3.066 | val 3.102 | lr 5.97e-04 | 102,253 tok/s | 54.64 GB | 1.211B tok iter 9260 | train 2.938 | val 3.122 | lr 5.97e-04 | 102,271 tok/s | 54.64 GB | 1.214B tok iter 9280 | train 2.992 | val 3.085 | lr 5.96e-04 | 102,289 tok/s | 54.64 GB | 1.216B tok iter 9300 | train 2.939 | val 3.145 | lr 5.96e-04 | 102,293 tok/s | 54.64 GB | 1.219B tok iter 9320 | train 3.208 | val 3.111 | lr 5.96e-04 | 102,311 tok/s | 54.64 GB | 1.222B tok iter 9340 | train 3.145 | val 3.107 | lr 5.96e-04 | 102,328 tok/s | 54.64 GB | 1.224B tok iter 9360 | train 3.143 | val 3.108 | lr 5.96e-04 | 102,346 tok/s | 54.64 GB | 1.227B tok iter 9380 | train 3.211 | val 3.146 | lr 5.96e-04 | 102,350 tok/s | 54.64 GB | 1.230B tok iter 9400 | train 3.086 | val 3.107 | lr 5.96e-04 | 102,368 tok/s | 54.64 GB | 1.232B tok iter 9420 | train 2.971 | val 3.068 | lr 5.96e-04 | 102,385 tok/s | 54.64 GB | 1.235B tok iter 9440 | train 3.162 | val 3.133 | lr 5.96e-04 | 102,402 tok/s | 54.64 GB | 1.237B tok iter 9460 | train 3.124 | val 3.152 | lr 5.96e-04 | 102,405 tok/s | 54.64 GB | 1.240B tok iter 9480 | train 3.027 | val 3.129 | lr 5.96e-04 | 102,422 tok/s | 54.64 GB | 1.243B tok checkpoint @ iter 9491 -> checkpoints/500m_latest.pt iter 9500 | train 3.227 | val 3.090 | lr 5.96e-04 | 102,327 tok/s | 54.64 GB | 1.245B tok checkpoint @ iter 9500 -> checkpoints/500m_latest.pt iter 9520 | train 3.015 | val 3.085 | lr 5.96e-04 | 102,123 tok/s | 54.64 GB | 1.248B tok iter 9540 | train 3.067 | val 3.105 | lr 5.96e-04 | 102,127 tok/s | 54.64 GB | 1.251B tok iter 9560 | train 3.050 | val 3.181 | lr 5.96e-04 | 102,145 tok/s | 54.64 GB | 1.253B tok iter 9580 | train 3.049 | val 3.080 | lr 5.96e-04 | 102,162 tok/s | 54.64 GB | 1.256B tok iter 9600 | train 3.156 | val 3.093 | lr 5.96e-04 | 102,180 tok/s | 54.64 GB | 1.258B tok iter 9620 | train 3.070 | val 3.094 | lr 5.96e-04 | 102,184 tok/s | 54.64 GB | 1.261B tok iter 9640 | train 3.125 | val 3.064 | lr 5.96e-04 | 102,201 tok/s | 54.64 GB | 1.264B tok iter 9660 | train 3.131 | val 3.118 | lr 5.96e-04 | 102,219 tok/s | 54.64 GB | 1.266B tok iter 9680 | train 3.092 | val 3.156 | lr 5.96e-04 | 102,236 tok/s | 54.64 GB | 1.269B tok iter 9700 | train 3.227 | val 3.074 | lr 5.96e-04 | 102,240 tok/s | 54.64 GB | 1.272B tok iter 9720 | train 3.080 | val 3.116 | lr 5.96e-04 | 102,257 tok/s | 54.64 GB | 1.274B tok iter 9740 | train 3.146 | val 3.105 | lr 5.96e-04 | 102,274 tok/s | 54.64 GB | 1.277B tok iter 9760 | train 3.179 | val 3.159 | lr 5.96e-04 | 102,291 tok/s | 54.64 GB | 1.279B tok iter 9780 | train 3.161 | val 3.089 | lr 5.96e-04 | 102,295 tok/s | 54.64 GB | 1.282B tok iter 9800 | train 3.241 | val 3.143 | lr 5.96e-04 | 102,312 tok/s | 54.64 GB | 1.285B tok iter 9820 | train 3.059 | val 3.112 | lr 5.96e-04 | 102,329 tok/s | 54.64 GB | 1.287B tok iter 9840 | train 3.004 | val 3.096 | lr 5.96e-04 | 102,343 tok/s | 54.64 GB | 1.290B tok iter 9860 | train 3.063 | val 3.073 | lr 5.96e-04 | 102,347 tok/s | 54.64 GB | 1.293B tok iter 9880 | train 3.041 | val 3.109 | lr 5.96e-04 | 102,364 tok/s | 54.64 GB | 1.295B tok iter 9900 | train 3.240 | val 3.132 | lr 5.96e-04 | 102,380 tok/s | 54.64 GB | 1.298B tok iter 9920 | train 3.013 | val 3.095 | lr 5.96e-04 | 102,397 tok/s | 54.64 GB | 1.300B tok iter 9940 | train 3.268 | val 3.111 | lr 5.96e-04 | 102,401 tok/s | 54.64 GB | 1.303B tok iter 9960 | train 3.129 | val 3.140 | lr 5.96e-04 | 102,417 tok/s | 54.64 GB | 1.306B tok iter 9980 | train 3.181 | val 3.117 | lr 5.96e-04 | 102,433 tok/s | 54.64 GB | 1.308B tok checkpoint @ iter 9991 -> checkpoints/500m_latest.pt iter 10000 | train 3.161 | val 3.070 | lr 5.96e-04 | 102,341 tok/s | 54.64 GB | 1.311B tok checkpoint @ iter 10000 -> checkpoints/500m_latest.pt iter 10020 | train 3.079 | val 3.077 | lr 5.96e-04 | 102,137 tok/s | 54.64 GB | 1.313B tok iter 10040 | train 3.145 | val 3.095 | lr 5.96e-04 | 102,154 tok/s | 54.64 GB | 1.316B tok iter 10060 | train 3.117 | val 3.076 | lr 5.96e-04 | 102,171 tok/s | 54.64 GB | 1.319B tok iter 10080 | train 3.143 | val 3.105 | lr 5.96e-04 | 102,187 tok/s | 54.64 GB | 1.321B tok iter 10100 | train 3.032 | val 3.108 | lr 5.96e-04 | 102,192 tok/s | 54.64 GB | 1.324B tok iter 10120 | train 3.179 | val 3.105 | lr 5.96e-04 | 102,208 tok/s | 54.64 GB | 1.327B tok iter 10140 | train 3.132 | val 3.063 | lr 5.96e-04 | 102,225 tok/s | 54.64 GB | 1.329B tok iter 10160 | train 3.161 | val 3.121 | lr 5.96e-04 | 102,241 tok/s | 54.64 GB | 1.332B tok iter 10180 | train 3.003 | val 3.130 | lr 5.96e-04 | 102,245 tok/s | 54.64 GB | 1.334B tok iter 10200 | train 3.047 | val 3.113 | lr 5.96e-04 | 102,261 tok/s | 54.64 GB | 1.337B tok iter 10220 | train 3.161 | val 3.098 | lr 5.96e-04 | 102,277 tok/s | 54.64 GB | 1.340B tok iter 10240 | train 2.998 | val 3.100 | lr 5.96e-04 | 102,293 tok/s | 54.64 GB | 1.342B tok iter 10260 | train 2.979 | val 3.151 | lr 5.96e-04 | 102,297 tok/s | 54.64 GB | 1.345B tok iter 10280 | train 3.046 | val 3.087 | lr 5.96e-04 | 102,313 tok/s | 54.64 GB | 1.348B tok iter 10300 | train 3.182 | val 3.109 | lr 5.96e-04 | 102,329 tok/s | 54.64 GB | 1.350B tok iter 10320 | train 2.992 | val 3.067 | lr 5.95e-04 | 102,345 tok/s | 54.64 GB | 1.353B tok iter 10340 | train 3.107 | val 3.068 | lr 5.95e-04 | 102,349 tok/s | 54.64 GB | 1.355B tok iter 10360 | train 3.064 | val 3.101 | lr 5.95e-04 | 102,364 tok/s | 54.64 GB | 1.358B tok iter 10380 | train 3.143 | val 3.124 | lr 5.95e-04 | 102,379 tok/s | 54.64 GB | 1.361B tok iter 10400 | train 3.090 | val 3.102 | lr 5.95e-04 | 102,394 tok/s | 54.64 GB | 1.363B tok iter 10420 | train 3.006 | val 3.102 | lr 5.95e-04 | 102,398 tok/s | 54.64 GB | 1.366B tok iter 10440 | train 2.997 | val 3.080 | lr 5.95e-04 | 102,413 tok/s | 54.64 GB | 1.369B tok iter 10460 | train 3.087 | val 3.075 | lr 5.95e-04 | 102,429 tok/s | 54.64 GB | 1.371B tok iter 10480 | train 3.077 | val 3.092 | lr 5.95e-04 | 102,444 tok/s | 54.64 GB | 1.374B tok checkpoint @ iter 10491 -> checkpoints/500m_latest.pt iter 10500 | train 3.057 | val 3.089 | lr 5.95e-04 | 102,346 tok/s | 54.64 GB | 1.376B tok checkpoint @ iter 10500 -> checkpoints/500m_latest.pt iter 10520 | train 3.117 | val 3.048 | lr 5.95e-04 | 102,160 tok/s | 54.64 GB | 1.379B tok iter 10540 | train 3.168 | val 3.094 | lr 5.95e-04 | 102,176 tok/s | 54.64 GB | 1.382B tok iter 10560 | train 3.180 | val 3.111 | lr 5.95e-04 | 102,192 tok/s | 54.64 GB | 1.384B tok iter 10580 | train 3.121 | val 3.095 | lr 5.95e-04 | 102,196 tok/s | 54.64 GB | 1.387B tok iter 10600 | train 3.108 | val 3.089 | lr 5.95e-04 | 102,212 tok/s | 54.64 GB | 1.389B tok iter 10620 | train 3.022 | val 3.107 | lr 5.95e-04 | 102,227 tok/s | 54.64 GB | 1.392B tok iter 10640 | train 3.071 | val 3.088 | lr 5.95e-04 | 102,243 tok/s | 54.64 GB | 1.395B tok iter 10660 | train 3.196 | val 3.121 | lr 5.95e-04 | 102,247 tok/s | 54.64 GB | 1.397B tok iter 10680 | train 3.085 | val 3.075 | lr 5.95e-04 | 102,262 tok/s | 54.64 GB | 1.400B tok iter 10700 | train 2.998 | val 3.100 | lr 5.95e-04 | 102,278 tok/s | 54.64 GB | 1.403B tok iter 10720 | train 3.086 | val 3.114 | lr 5.95e-04 | 102,293 tok/s | 54.64 GB | 1.405B tok iter 10740 | train 3.092 | val 3.064 | lr 5.95e-04 | 102,298 tok/s | 54.64 GB | 1.408B tok iter 10760 | train 3.118 | val 3.087 | lr 5.95e-04 | 102,313 tok/s | 54.64 GB | 1.410B tok iter 10780 | train 3.147 | val 3.080 | lr 5.95e-04 | 102,328 tok/s | 54.64 GB | 1.413B tok iter 10800 | train 3.033 | val 3.089 | lr 5.95e-04 | 102,343 tok/s | 54.64 GB | 1.416B tok iter 10820 | train 2.931 | val 3.121 | lr 5.95e-04 | 102,347 tok/s | 54.64 GB | 1.418B tok iter 10840 | train 3.039 | val 3.060 | lr 5.95e-04 | 102,362 tok/s | 54.64 GB | 1.421B tok iter 10860 | train 3.081 | val 3.096 | lr 5.95e-04 | 102,377 tok/s | 54.64 GB | 1.424B tok iter 10880 | train 3.131 | val 3.133 | lr 5.95e-04 | 102,392 tok/s | 54.64 GB | 1.426B tok iter 10900 | train 3.075 | val 3.115 | lr 5.95e-04 | 102,396 tok/s | 54.64 GB | 1.429B tok iter 10920 | train 3.024 | val 3.078 | lr 5.95e-04 | 102,410 tok/s | 54.64 GB | 1.431B tok iter 10940 | train 3.180 | val 3.100 | lr 5.95e-04 | 102,425 tok/s | 54.64 GB | 1.434B tok iter 10960 | train 3.065 | val 3.086 | lr 5.95e-04 | 102,439 tok/s | 54.64 GB | 1.437B tok iter 10980 | train 2.968 | val 3.115 | lr 5.95e-04 | 102,443 tok/s | 54.64 GB | 1.439B tok checkpoint @ iter 10991 -> checkpoints/500m_latest.pt iter 11000 | train 3.277 | val 3.107 | lr 5.95e-04 | 102,363 tok/s | 54.64 GB | 1.442B tok checkpoint @ iter 11000 -> checkpoints/500m_latest.pt iter 11020 | train 3.097 | val 3.099 | lr 5.95e-04 | 102,194 tok/s | 54.64 GB | 1.445B tok iter 11040 | train 3.035 | val 3.072 | lr 5.95e-04 | 102,209 tok/s | 54.64 GB | 1.447B tok iter 11060 | train 3.135 | val 3.095 | lr 5.95e-04 | 102,213 tok/s | 54.64 GB | 1.450B tok iter 11080 | train 3.206 | val 3.138 | lr 5.95e-04 | 102,228 tok/s | 54.64 GB | 1.452B tok iter 11100 | train 2.979 | val 3.110 | lr 5.95e-04 | 102,243 tok/s | 54.64 GB | 1.455B tok iter 11120 | train 3.097 | val 3.090 | lr 5.95e-04 | 102,258 tok/s | 54.64 GB | 1.458B tok iter 11140 | train 3.156 | val 3.094 | lr 5.95e-04 | 102,262 tok/s | 54.64 GB | 1.460B tok iter 11160 | train 3.076 | val 3.025 | lr 5.95e-04 | 102,276 tok/s | 54.64 GB | 1.463B tok iter 11180 | train 3.172 | val 3.102 | lr 5.95e-04 | 102,291 tok/s | 54.64 GB | 1.466B tok iter 11200 | train 3.195 | val 3.046 | lr 5.95e-04 | 102,306 tok/s | 54.64 GB | 1.468B tok iter 11220 | train 3.122 | val 3.106 | lr 5.95e-04 | 102,310 tok/s | 54.64 GB | 1.471B tok iter 11240 | train 2.991 | val 3.011 | lr 5.95e-04 | 102,323 tok/s | 54.64 GB | 1.473B tok iter 11260 | train 3.113 | val 3.074 | lr 5.94e-04 | 102,337 tok/s | 54.64 GB | 1.476B tok iter 11280 | train 3.144 | val 3.042 | lr 5.94e-04 | 102,352 tok/s | 54.64 GB | 1.479B tok iter 11300 | train 3.090 | val 3.097 | lr 5.94e-04 | 102,355 tok/s | 54.64 GB | 1.481B tok iter 11320 | train 3.142 | val 3.031 | lr 5.94e-04 | 102,370 tok/s | 54.64 GB | 1.484B tok iter 11340 | train 3.198 | val 3.063 | lr 5.94e-04 | 102,384 tok/s | 54.64 GB | 1.486B tok iter 11360 | train 3.019 | val 3.113 | lr 5.94e-04 | 102,398 tok/s | 54.64 GB | 1.489B tok iter 11380 | train 3.044 | val 3.081 | lr 5.94e-04 | 102,402 tok/s | 54.64 GB | 1.492B tok iter 11400 | train 3.214 | val 3.126 | lr 5.94e-04 | 102,416 tok/s | 54.64 GB | 1.494B tok iter 11420 | train 3.031 | val 3.067 | lr 5.94e-04 | 102,430 tok/s | 54.64 GB | 1.497B tok iter 11440 | train 3.064 | val 3.085 | lr 5.94e-04 | 102,444 tok/s | 54.64 GB | 1.500B tok iter 11460 | train 3.187 | val 3.061 | lr 5.94e-04 | 102,448 tok/s | 54.64 GB | 1.502B tok iter 11480 | train 3.112 | val 3.044 | lr 5.94e-04 | 102,462 tok/s | 54.64 GB | 1.505B tok checkpoint @ iter 11491 -> checkpoints/500m_latest.pt iter 11500 | train 3.115 | val 3.079 | lr 5.94e-04 | 102,380 tok/s | 54.64 GB | 1.507B tok checkpoint @ iter 11500 -> checkpoints/500m_latest.pt iter 11520 | train 2.980 | val 3.146 | lr 5.94e-04 | 102,207 tok/s | 54.64 GB | 1.510B tok iter 11540 | train 3.123 | val 3.072 | lr 5.94e-04 | 102,211 tok/s | 54.64 GB | 1.513B tok iter 11560 | train 3.105 | val 3.040 | lr 5.94e-04 | 102,226 tok/s | 54.64 GB | 1.515B tok iter 11580 | train 3.068 | val 3.039 | lr 5.94e-04 | 102,240 tok/s | 54.64 GB | 1.518B tok iter 11600 | train 2.897 | val 3.040 | lr 5.94e-04 | 102,254 tok/s | 54.64 GB | 1.521B tok iter 11620 | train 3.057 | val 3.045 | lr 5.94e-04 | 102,258 tok/s | 54.64 GB | 1.523B tok iter 11640 | train 3.178 | val 3.067 | lr 5.94e-04 | 102,272 tok/s | 54.64 GB | 1.526B tok iter 11660 | train 3.003 | val 3.025 | lr 5.94e-04 | 102,286 tok/s | 54.64 GB | 1.528B tok iter 11680 | train 3.041 | val 3.087 | lr 5.94e-04 | 102,300 tok/s | 54.64 GB | 1.531B tok iter 11700 | train 3.222 | val 3.059 | lr 5.94e-04 | 102,304 tok/s | 54.64 GB | 1.534B tok iter 11720 | train 3.087 | val 3.053 | lr 5.94e-04 | 102,318 tok/s | 54.64 GB | 1.536B tok iter 11740 | train 3.126 | val 3.076 | lr 5.94e-04 | 102,332 tok/s | 54.64 GB | 1.539B tok iter 11760 | train 3.049 | val 3.056 | lr 5.94e-04 | 102,346 tok/s | 54.64 GB | 1.542B tok iter 11780 | train 2.933 | val 3.049 | lr 5.94e-04 | 102,349 tok/s | 54.64 GB | 1.544B tok iter 11800 | train 2.983 | val 3.085 | lr 5.94e-04 | 102,363 tok/s | 54.64 GB | 1.547B tok iter 11820 | train 3.027 | val 3.091 | lr 5.94e-04 | 102,377 tok/s | 54.64 GB | 1.549B tok iter 11840 | train 3.070 | val 3.089 | lr 5.94e-04 | 102,390 tok/s | 54.64 GB | 1.552B tok iter 11860 | train 3.020 | val 3.063 | lr 5.94e-04 | 102,394 tok/s | 54.64 GB | 1.555B tok iter 11880 | train 3.089 | val 3.051 | lr 5.94e-04 | 102,407 tok/s | 54.64 GB | 1.557B tok iter 11900 | train 3.120 | val 3.057 | lr 5.94e-04 | 102,419 tok/s | 54.64 GB | 1.560B tok iter 11920 | train 3.043 | val 3.096 | lr 5.94e-04 | 102,432 tok/s | 54.64 GB | 1.563B tok iter 11940 | train 3.107 | val 3.048 | lr 5.94e-04 | 102,435 tok/s | 54.64 GB | 1.565B tok iter 11960 | train 2.992 | val 3.086 | lr 5.94e-04 | 102,449 tok/s | 54.64 GB | 1.568B tok iter 11980 | train 3.059 | val 3.065 | lr 5.94e-04 | 102,463 tok/s | 54.64 GB | 1.570B tok checkpoint @ iter 11991 -> checkpoints/500m_latest.pt iter 12000 | train 3.082 | val 3.077 | lr 5.94e-04 | 102,389 tok/s | 54.64 GB | 1.573B tok checkpoint @ iter 12000 -> checkpoints/500m_latest.pt iter 12020 | train 2.996 | val 3.037 | lr 5.94e-04 | 102,208 tok/s | 54.64 GB | 1.576B tok iter 12040 | train 3.053 | val 3.075 | lr 5.94e-04 | 102,223 tok/s | 54.64 GB | 1.578B tok iter 12060 | train 3.158 | val 3.044 | lr 5.94e-04 | 102,235 tok/s | 54.64 GB | 1.581B tok iter 12080 | train 3.037 | val 3.087 | lr 5.94e-04 | 102,249 tok/s | 54.64 GB | 1.583B tok iter 12100 | train 3.100 | val 3.068 | lr 5.93e-04 | 102,252 tok/s | 54.64 GB | 1.586B tok iter 12120 | train 3.183 | val 3.082 | lr 5.93e-04 | 102,266 tok/s | 54.64 GB | 1.589B tok iter 12140 | train 2.965 | val 3.021 | lr 5.93e-04 | 102,280 tok/s | 54.64 GB | 1.591B tok iter 12160 | train 3.109 | val 3.059 | lr 5.93e-04 | 102,293 tok/s | 54.64 GB | 1.594B tok iter 12180 | train 3.122 | val 3.039 | lr 5.93e-04 | 102,296 tok/s | 54.64 GB | 1.597B tok iter 12200 | train 3.080 | val 3.050 | lr 5.93e-04 | 102,310 tok/s | 54.64 GB | 1.599B tok iter 12220 | train 2.938 | val 3.066 | lr 5.93e-04 | 102,323 tok/s | 54.64 GB | 1.602B tok iter 12240 | train 3.210 | val 3.038 | lr 5.93e-04 | 102,336 tok/s | 54.64 GB | 1.604B tok iter 12260 | train 3.059 | val 3.051 | lr 5.93e-04 | 102,340 tok/s | 54.64 GB | 1.607B tok iter 12280 | train 3.035 | val 3.036 | lr 5.93e-04 | 102,353 tok/s | 54.64 GB | 1.610B tok iter 12300 | train 2.917 | val 3.056 | lr 5.93e-04 | 102,366 tok/s | 54.64 GB | 1.612B tok iter 12320 | train 3.086 | val 3.050 | lr 5.93e-04 | 102,380 tok/s | 54.64 GB | 1.615B tok iter 12340 | train 2.980 | val 3.079 | lr 5.93e-04 | 102,383 tok/s | 54.64 GB | 1.618B tok iter 12360 | train 2.970 | val 3.042 | lr 5.93e-04 | 102,396 tok/s | 54.64 GB | 1.620B tok iter 12380 | train 3.031 | val 3.067 | lr 5.93e-04 | 102,409 tok/s | 54.64 GB | 1.623B tok iter 12400 | train 3.024 | val 3.055 | lr 5.93e-04 | 102,422 tok/s | 54.64 GB | 1.625B tok iter 12420 | train 3.135 | val 3.045 | lr 5.93e-04 | 102,426 tok/s | 54.64 GB | 1.628B tok iter 12440 | train 3.050 | val 3.064 | lr 5.93e-04 | 102,439 tok/s | 54.64 GB | 1.631B tok iter 12460 | train 3.029 | val 3.053 | lr 5.93e-04 | 102,451 tok/s | 54.64 GB | 1.633B tok iter 12480 | train 3.090 | val 3.036 | lr 5.93e-04 | 102,464 tok/s | 54.64 GB | 1.636B tok checkpoint @ iter 12490 -> checkpoints/500m_latest.pt iter 12500 | train 2.951 | val 3.032 | lr 5.93e-04 | 102,383 tok/s | 54.64 GB | 1.639B tok checkpoint @ iter 12500 -> checkpoints/500m_latest.pt iter 12520 | train 2.993 | val 3.059 | lr 5.93e-04 | 102,230 tok/s | 54.64 GB | 1.641B tok iter 12540 | train 3.034 | val 3.054 | lr 5.93e-04 | 102,244 tok/s | 54.64 GB | 1.644B tok iter 12560 | train 3.074 | val 3.041 | lr 5.93e-04 | 102,257 tok/s | 54.64 GB | 1.646B tok iter 12580 | train 3.118 | val 3.122 | lr 5.93e-04 | 102,260 tok/s | 54.64 GB | 1.649B tok iter 12600 | train 2.996 | val 3.032 | lr 5.93e-04 | 102,273 tok/s | 54.64 GB | 1.652B tok iter 12620 | train 3.220 | val 3.105 | lr 5.93e-04 | 102,276 tok/s | 54.64 GB | 1.654B tok iter 12640 | train 3.024 | val 2.997 | lr 5.93e-04 | 102,289 tok/s | 54.64 GB | 1.657B tok iter 12660 | train 3.057 | val 3.094 | lr 5.93e-04 | 102,292 tok/s | 54.64 GB | 1.660B tok iter 12680 | train 2.967 | val 3.093 | lr 5.93e-04 | 102,305 tok/s | 54.64 GB | 1.662B tok iter 12700 | train 2.959 | val 3.070 | lr 5.93e-04 | 102,318 tok/s | 54.64 GB | 1.665B tok iter 12720 | train 2.989 | val 3.041 | lr 5.93e-04 | 102,331 tok/s | 54.64 GB | 1.667B tok iter 12740 | train 3.147 | val 3.106 | lr 5.93e-04 | 102,334 tok/s | 54.64 GB | 1.670B tok iter 12760 | train 3.048 | val 3.068 | lr 5.93e-04 | 102,347 tok/s | 54.64 GB | 1.673B tok iter 12780 | train 3.117 | val 3.061 | lr 5.93e-04 | 102,360 tok/s | 54.64 GB | 1.675B tok iter 12800 | train 3.107 | val 3.000 | lr 5.93e-04 | 102,373 tok/s | 54.64 GB | 1.678B tok iter 12820 | train 3.031 | val 3.052 | lr 5.93e-04 | 102,376 tok/s | 54.64 GB | 1.680B tok iter 12840 | train 3.074 | val 3.059 | lr 5.93e-04 | 102,388 tok/s | 54.64 GB | 1.683B tok iter 12860 | train 3.161 | val 3.066 | lr 5.93e-04 | 102,401 tok/s | 54.64 GB | 1.686B tok iter 12880 | train 3.131 | val 3.070 | lr 5.93e-04 | 102,413 tok/s | 54.64 GB | 1.688B tok iter 12900 | train 3.086 | val 3.067 | lr 5.92e-04 | 102,417 tok/s | 54.64 GB | 1.691B tok iter 12920 | train 3.094 | val 3.050 | lr 5.92e-04 | 102,428 tok/s | 54.64 GB | 1.694B tok iter 12940 | train 3.046 | val 3.017 | lr 5.92e-04 | 102,440 tok/s | 54.64 GB | 1.696B tok iter 12960 | train 3.013 | val 3.096 | lr 5.92e-04 | 102,453 tok/s | 54.64 GB | 1.699B tok iter 12980 | train 3.019 | val 3.022 | lr 5.92e-04 | 102,456 tok/s | 54.64 GB | 1.701B tok checkpoint @ iter 12990 -> checkpoints/500m_latest.pt iter 13000 | train 3.047 | val 3.076 | lr 5.92e-04 | 102,384 tok/s | 54.64 GB | 1.704B tok checkpoint @ iter 13000 -> checkpoints/500m_latest.pt iter 13020 | train 3.047 | val 3.051 | lr 5.92e-04 | 102,238 tok/s | 54.64 GB | 1.707B tok iter 13040 | train 3.135 | val 3.049 | lr 5.92e-04 | 102,251 tok/s | 54.64 GB | 1.709B tok iter 13060 | train 2.921 | val 3.022 | lr 5.92e-04 | 102,254 tok/s | 54.64 GB | 1.712B tok iter 13080 | train 2.996 | val 3.057 | lr 5.92e-04 | 102,267 tok/s | 54.64 GB | 1.715B tok iter 13100 | train 3.110 | val 3.027 | lr 5.92e-04 | 102,279 tok/s | 54.64 GB | 1.717B tok iter 13120 | train 3.007 | val 3.049 | lr 5.92e-04 | 102,292 tok/s | 54.64 GB | 1.720B tok iter 13140 | train 3.050 | val 3.040 | lr 5.92e-04 | 102,295 tok/s | 54.64 GB | 1.722B tok iter 13160 | train 3.161 | val 3.024 | lr 5.92e-04 | 102,307 tok/s | 54.64 GB | 1.725B tok iter 13180 | train 3.059 | val 3.024 | lr 5.92e-04 | 102,320 tok/s | 54.64 GB | 1.728B tok iter 13200 | train 2.948 | val 3.043 | lr 5.92e-04 | 102,332 tok/s | 54.64 GB | 1.730B tok iter 13220 | train 3.125 | val 3.069 | lr 5.92e-04 | 102,335 tok/s | 54.64 GB | 1.733B tok iter 13240 | train 3.098 | val 3.081 | lr 5.92e-04 | 102,348 tok/s | 54.64 GB | 1.736B tok iter 13260 | train 3.056 | val 3.065 | lr 5.92e-04 | 102,360 tok/s | 54.64 GB | 1.738B tok iter 13280 | train 2.859 | val 3.070 | lr 5.92e-04 | 102,372 tok/s | 54.64 GB | 1.741B tok iter 13300 | train 3.072 | val 3.032 | lr 5.92e-04 | 102,375 tok/s | 54.64 GB | 1.743B tok iter 13320 | train 3.009 | val 3.040 | lr 5.92e-04 | 102,388 tok/s | 54.64 GB | 1.746B tok iter 13340 | train 2.905 | val 2.991 | lr 5.92e-04 | 102,400 tok/s | 54.64 GB | 1.749B tok iter 13360 | train 3.021 | val 3.047 | lr 5.92e-04 | 102,412 tok/s | 54.64 GB | 1.751B tok iter 13380 | train 3.082 | val 3.053 | lr 5.92e-04 | 102,415 tok/s | 54.64 GB | 1.754B tok iter 13400 | train 3.038 | val 3.092 | lr 5.92e-04 | 102,427 tok/s | 54.64 GB | 1.756B tok iter 13420 | train 2.913 | val 3.102 | lr 5.92e-04 | 102,439 tok/s | 54.64 GB | 1.759B tok iter 13440 | train 2.928 | val 3.020 | lr 5.92e-04 | 102,451 tok/s | 54.64 GB | 1.762B tok iter 13460 | train 2.932 | val 3.058 | lr 5.92e-04 | 102,454 tok/s | 54.64 GB | 1.764B tok iter 13480 | train 3.118 | val 3.034 | lr 5.92e-04 | 102,466 tok/s | 54.64 GB | 1.767B tok checkpoint @ iter 13492 -> checkpoints/500m_latest.pt iter 13500 | train 3.069 | val 3.011 | lr 5.92e-04 | 102,399 tok/s | 54.64 GB | 1.770B tok checkpoint @ iter 13500 -> checkpoints/500m_latest.pt iter 13520 | train 3.030 | val 3.037 | lr 5.92e-04 | 102,263 tok/s | 54.64 GB | 1.772B tok iter 13540 | train 3.026 | val 3.083 | lr 5.92e-04 | 102,266 tok/s | 54.64 GB | 1.775B tok iter 13560 | train 3.134 | val 3.058 | lr 5.92e-04 | 102,279 tok/s | 54.64 GB | 1.777B tok iter 13580 | train 3.084 | val 3.032 | lr 5.92e-04 | 102,291 tok/s | 54.64 GB | 1.780B tok iter 13600 | train 3.128 | val 3.094 | lr 5.92e-04 | 102,303 tok/s | 54.64 GB | 1.783B tok iter 13620 | train 3.148 | val 3.037 | lr 5.92e-04 | 102,306 tok/s | 54.64 GB | 1.785B tok iter 13640 | train 2.960 | val 3.071 | lr 5.91e-04 | 102,318 tok/s | 54.64 GB | 1.788B tok iter 13660 | train 3.169 | val 3.047 | lr 5.91e-04 | 102,329 tok/s | 54.64 GB | 1.791B tok iter 13680 | train 3.011 | val 3.028 | lr 5.91e-04 | 102,341 tok/s | 54.64 GB | 1.793B tok iter 13700 | train 3.109 | val 3.022 | lr 5.91e-04 | 102,344 tok/s | 54.64 GB | 1.796B tok iter 13720 | train 3.032 | val 3.027 | lr 5.91e-04 | 102,356 tok/s | 54.64 GB | 1.798B tok iter 13740 | train 3.208 | val 3.011 | lr 5.91e-04 | 102,368 tok/s | 54.64 GB | 1.801B tok iter 13760 | train 2.999 | val 3.061 | lr 5.91e-04 | 102,380 tok/s | 54.64 GB | 1.804B tok iter 13780 | train 2.907 | val 3.034 | lr 5.91e-04 | 102,382 tok/s | 54.64 GB | 1.806B tok iter 13800 | train 3.143 | val 3.002 | lr 5.91e-04 | 102,393 tok/s | 54.64 GB | 1.809B tok iter 13820 | train 3.183 | val 3.043 | lr 5.91e-04 | 102,405 tok/s | 54.64 GB | 1.812B tok iter 13840 | train 2.813 | val 3.051 | lr 5.91e-04 | 102,417 tok/s | 54.64 GB | 1.814B tok iter 13860 | train 2.990 | val 3.054 | lr 5.91e-04 | 102,420 tok/s | 54.64 GB | 1.817B tok iter 13880 | train 2.980 | val 3.036 | lr 5.91e-04 | 102,431 tok/s | 54.64 GB | 1.819B tok iter 13900 | train 3.072 | val 3.049 | lr 5.91e-04 | 102,443 tok/s | 54.64 GB | 1.822B tok iter 13920 | train 3.088 | val 3.052 | lr 5.91e-04 | 102,455 tok/s | 54.64 GB | 1.825B tok iter 13940 | train 3.027 | val 3.038 | lr 5.91e-04 | 102,457 tok/s | 54.64 GB | 1.827B tok iter 13960 | train 3.076 | val 3.001 | lr 5.91e-04 | 102,469 tok/s | 54.64 GB | 1.830B tok iter 13980 | train 3.056 | val 3.029 | lr 5.91e-04 | 102,481 tok/s | 54.64 GB | 1.833B tok checkpoint @ iter 13991 -> checkpoints/500m_latest.pt iter 14000 | train 2.896 | val 3.047 | lr 5.91e-04 | 102,415 tok/s | 54.64 GB | 1.835B tok checkpoint @ iter 14000 -> checkpoints/500m_latest.pt iter 14020 | train 3.056 | val 3.042 | lr 5.91e-04 | 102,267 tok/s | 54.64 GB | 1.838B tok iter 14040 | train 2.942 | val 3.060 | lr 5.91e-04 | 102,279 tok/s | 54.64 GB | 1.840B tok iter 14060 | train 2.961 | val 3.053 | lr 5.91e-04 | 102,291 tok/s | 54.64 GB | 1.843B tok iter 14080 | train 3.034 | val 3.009 | lr 5.91e-04 | 102,303 tok/s | 54.64 GB | 1.846B tok iter 14100 | train 3.151 | val 3.044 | lr 5.91e-04 | 102,306 tok/s | 54.64 GB | 1.848B tok iter 14120 | train 3.093 | val 3.077 | lr 5.91e-04 | 102,317 tok/s | 54.64 GB | 1.851B tok iter 14140 | train 3.170 | val 3.037 | lr 5.91e-04 | 102,329 tok/s | 54.64 GB | 1.853B tok iter 14160 | train 3.077 | val 3.024 | lr 5.91e-04 | 102,340 tok/s | 54.64 GB | 1.856B tok iter 14180 | train 2.913 | val 3.043 | lr 5.91e-04 | 102,343 tok/s | 54.64 GB | 1.859B tok iter 14200 | train 3.009 | val 3.035 | lr 5.91e-04 | 102,355 tok/s | 54.64 GB | 1.861B tok iter 14220 | train 3.038 | val 3.042 | lr 5.91e-04 | 102,366 tok/s | 54.64 GB | 1.864B tok iter 14240 | train 3.073 | val 3.013 | lr 5.91e-04 | 102,378 tok/s | 54.64 GB | 1.867B tok iter 14260 | train 3.127 | val 3.047 | lr 5.91e-04 | 102,381 tok/s | 54.64 GB | 1.869B tok iter 14280 | train 2.902 | val 3.048 | lr 5.91e-04 | 102,392 tok/s | 54.64 GB | 1.872B tok iter 14300 | train 2.931 | val 3.013 | lr 5.91e-04 | 102,404 tok/s | 54.64 GB | 1.874B tok iter 14320 | train 3.056 | val 3.028 | lr 5.90e-04 | 102,415 tok/s | 54.64 GB | 1.877B tok iter 14340 | train 2.934 | val 3.004 | lr 5.90e-04 | 102,418 tok/s | 54.64 GB | 1.880B tok iter 14360 | train 2.807 | val 3.012 | lr 5.90e-04 | 102,429 tok/s | 54.64 GB | 1.882B tok iter 14380 | train 3.006 | val 3.051 | lr 5.90e-04 | 102,440 tok/s | 54.64 GB | 1.885B tok iter 14400 | train 2.973 | val 3.057 | lr 5.90e-04 | 102,452 tok/s | 54.64 GB | 1.888B tok iter 14420 | train 3.058 | val 3.053 | lr 5.90e-04 | 102,454 tok/s | 54.64 GB | 1.890B tok iter 14440 | train 3.127 | val 3.029 | lr 5.90e-04 | 102,466 tok/s | 54.64 GB | 1.893B tok iter 14460 | train 3.114 | val 3.001 | lr 5.90e-04 | 102,477 tok/s | 54.64 GB | 1.895B tok iter 14480 | train 3.044 | val 3.046 | lr 5.90e-04 | 102,488 tok/s | 54.64 GB | 1.898B tok checkpoint @ iter 14491 -> checkpoints/500m_latest.pt iter 14500 | train 3.115 | val 3.046 | lr 5.90e-04 | 102,417 tok/s | 54.64 GB | 1.901B tok checkpoint @ iter 14500 -> checkpoints/500m_latest.pt iter 14520 | train 2.993 | val 3.059 | lr 5.90e-04 | 102,288 tok/s | 54.64 GB | 1.903B tok iter 14540 | train 3.002 | val 3.057 | lr 5.90e-04 | 102,299 tok/s | 54.64 GB | 1.906B tok iter 14560 | train 3.027 | val 3.020 | lr 5.90e-04 | 102,310 tok/s | 54.64 GB | 1.909B tok iter 14580 | train 3.018 | val 3.031 | lr 5.90e-04 | 102,313 tok/s | 54.64 GB | 1.911B tok iter 14600 | train 2.988 | val 3.040 | lr 5.90e-04 | 102,325 tok/s | 54.64 GB | 1.914B tok iter 14620 | train 3.074 | val 3.020 | lr 5.90e-04 | 102,335 tok/s | 54.64 GB | 1.916B tok iter 14640 | train 2.980 | val 2.995 | lr 5.90e-04 | 102,346 tok/s | 54.64 GB | 1.919B tok iter 14660 | train 3.123 | val 2.987 | lr 5.90e-04 | 102,349 tok/s | 54.64 GB | 1.922B tok iter 14680 | train 2.883 | val 3.027 | lr 5.90e-04 | 102,360 tok/s | 54.64 GB | 1.924B tok iter 14700 | train 3.087 | val 2.998 | lr 5.90e-04 | 102,371 tok/s | 54.64 GB | 1.927B tok iter 14720 | train 3.052 | val 3.009 | lr 5.90e-04 | 102,382 tok/s | 54.64 GB | 1.930B tok iter 14740 | train 3.100 | val 3.054 | lr 5.90e-04 | 102,384 tok/s | 54.64 GB | 1.932B tok iter 14760 | train 3.047 | val 3.033 | lr 5.90e-04 | 102,396 tok/s | 54.64 GB | 1.935B tok iter 14780 | train 3.018 | val 2.997 | lr 5.90e-04 | 102,407 tok/s | 54.64 GB | 1.937B tok iter 14800 | train 3.129 | val 3.040 | lr 5.90e-04 | 102,417 tok/s | 54.64 GB | 1.940B tok iter 14820 | train 3.115 | val 3.024 | lr 5.90e-04 | 102,420 tok/s | 54.64 GB | 1.943B tok iter 14840 | train 3.058 | val 3.029 | lr 5.90e-04 | 102,431 tok/s | 54.64 GB | 1.945B tok iter 14860 | train 3.046 | val 3.019 | lr 5.90e-04 | 102,442 tok/s | 54.64 GB | 1.948B tok iter 14880 | train 3.170 | val 3.024 | lr 5.90e-04 | 102,453 tok/s | 54.64 GB | 1.950B tok iter 14900 | train 3.053 | val 2.997 | lr 5.90e-04 | 102,455 tok/s | 54.64 GB | 1.953B tok iter 14920 | train 3.074 | val 2.978 | lr 5.90e-04 | 102,466 tok/s | 54.64 GB | 1.956B tok iter 14940 | train 3.046 | val 3.027 | lr 5.90e-04 | 102,477 tok/s | 54.64 GB | 1.958B tok iter 14960 | train 3.060 | val 3.058 | lr 5.90e-04 | 102,488 tok/s | 54.64 GB | 1.961B tok iter 14980 | train 3.068 | val 3.063 | lr 5.89e-04 | 102,490 tok/s | 54.64 GB | 1.964B tok checkpoint @ iter 14991 -> checkpoints/500m_latest.pt iter 15000 | train 3.015 | val 3.026 | lr 5.89e-04 | 102,431 tok/s | 54.64 GB | 1.966B tok checkpoint @ iter 15000 -> checkpoints/500m_latest.pt iter 15020 | train 2.916 | val 3.018 | lr 5.89e-04 | 102,307 tok/s | 54.64 GB | 1.969B tok iter 15040 | train 2.959 | val 3.045 | lr 5.89e-04 | 102,318 tok/s | 54.64 GB | 1.971B tok iter 15060 | train 3.059 | val 3.042 | lr 5.89e-04 | 102,321 tok/s | 54.64 GB | 1.974B tok iter 15080 | train 2.896 | val 3.015 | lr 5.89e-04 | 102,332 tok/s | 54.64 GB | 1.977B tok iter 15100 | train 3.057 | val 3.078 | lr 5.89e-04 | 102,343 tok/s | 54.64 GB | 1.979B tok iter 15120 | train 2.993 | val 2.994 | lr 5.89e-04 | 102,354 tok/s | 54.64 GB | 1.982B tok iter 15140 | train 3.052 | val 3.009 | lr 5.89e-04 | 102,356 tok/s | 54.64 GB | 1.985B tok iter 15160 | train 2.941 | val 3.059 | lr 5.89e-04 | 102,367 tok/s | 54.64 GB | 1.987B tok iter 15180 | train 3.068 | val 3.024 | lr 5.89e-04 | 102,378 tok/s | 54.64 GB | 1.990B tok iter 15200 | train 3.012 | val 3.040 | lr 5.89e-04 | 102,389 tok/s | 54.64 GB | 1.992B tok iter 15220 | train 3.038 | val 3.035 | lr 5.89e-04 | 102,391 tok/s | 54.64 GB | 1.995B tok iter 15240 | train 2.986 | val 3.032 | lr 5.89e-04 | 102,402 tok/s | 54.64 GB | 1.998B tok iter 15260 | train 2.899 | val 3.007 | lr 5.89e-04 | 102,413 tok/s | 54.64 GB | 2.000B tok iter 15280 | train 3.117 | val 3.045 | lr 5.89e-04 | 102,423 tok/s | 54.64 GB | 2.003B tok iter 15300 | train 2.973 | val 3.031 | lr 5.89e-04 | 102,426 tok/s | 54.64 GB | 2.006B tok iter 15320 | train 3.011 | val 3.003 | lr 5.89e-04 | 102,437 tok/s | 54.64 GB | 2.008B tok iter 15340 | train 3.078 | val 3.017 | lr 5.89e-04 | 102,447 tok/s | 54.64 GB | 2.011B tok iter 15360 | train 2.911 | val 3.013 | lr 5.89e-04 | 102,458 tok/s | 54.64 GB | 2.013B tok iter 15380 | train 3.082 | val 3.041 | lr 5.89e-04 | 102,460 tok/s | 54.64 GB | 2.016B tok iter 15400 | train 3.079 | val 3.027 | lr 5.89e-04 | 102,471 tok/s | 54.64 GB | 2.019B tok iter 15420 | train 2.945 | val 3.048 | lr 5.89e-04 | 102,481 tok/s | 54.64 GB | 2.021B tok iter 15440 | train 3.010 | val 3.032 | lr 5.89e-04 | 102,492 tok/s | 54.64 GB | 2.024B tok iter 15460 | train 3.007 | val 3.043 | lr 5.89e-04 | 102,494 tok/s | 54.64 GB | 2.027B tok iter 15480 | train 3.166 | val 3.047 | lr 5.89e-04 | 102,504 tok/s | 54.64 GB | 2.029B tok checkpoint @ iter 15491 -> checkpoints/500m_latest.pt iter 15500 | train 3.053 | val 2.979 | lr 5.89e-04 | 102,443 tok/s | 54.64 GB | 2.032B tok checkpoint @ iter 15500 -> checkpoints/500m_latest.pt iter 15520 | train 3.003 | val 3.014 | lr 5.89e-04 | 102,312 tok/s | 54.64 GB | 2.034B tok iter 15540 | train 3.030 | val 2.993 | lr 5.89e-04 | 102,315 tok/s | 54.64 GB | 2.037B tok iter 15560 | train 3.013 | val 3.045 | lr 5.89e-04 | 102,326 tok/s | 54.64 GB | 2.040B tok iter 15580 | train 2.983 | val 3.049 | lr 5.89e-04 | 102,336 tok/s | 54.64 GB | 2.042B tok iter 15600 | train 3.064 | val 2.976 | lr 5.89e-04 | 102,347 tok/s | 54.64 GB | 2.045B tok iter 15620 | train 3.114 | val 3.018 | lr 5.88e-04 | 102,350 tok/s | 54.64 GB | 2.047B tok iter 15640 | train 2.916 | val 3.026 | lr 5.88e-04 | 102,360 tok/s | 54.64 GB | 2.050B tok iter 15660 | train 3.107 | val 3.062 | lr 5.88e-04 | 102,371 tok/s | 54.64 GB | 2.053B tok iter 15680 | train 3.112 | val 2.996 | lr 5.88e-04 | 102,381 tok/s | 54.64 GB | 2.055B tok iter 15700 | train 3.132 | val 3.034 | lr 5.88e-04 | 102,384 tok/s | 54.64 GB | 2.058B tok iter 15720 | train 2.863 | val 3.092 | lr 5.88e-04 | 102,394 tok/s | 54.64 GB | 2.061B tok iter 15740 | train 3.098 | val 3.003 | lr 5.88e-04 | 102,404 tok/s | 54.64 GB | 2.063B tok iter 15760 | train 3.023 | val 2.997 | lr 5.88e-04 | 102,414 tok/s | 54.64 GB | 2.066B tok iter 15780 | train 3.028 | val 2.998 | lr 5.88e-04 | 102,417 tok/s | 54.64 GB | 2.068B tok iter 15800 | train 2.950 | val 3.001 | lr 5.88e-04 | 102,427 tok/s | 54.64 GB | 2.071B tok iter 15820 | train 2.985 | val 3.055 | lr 5.88e-04 | 102,438 tok/s | 54.64 GB | 2.074B tok iter 15840 | train 3.009 | val 2.972 | lr 5.88e-04 | 102,448 tok/s | 54.64 GB | 2.076B tok iter 15860 | train 2.948 | val 2.996 | lr 5.88e-04 | 102,450 tok/s | 54.64 GB | 2.079B tok iter 15880 | train 3.154 | val 3.027 | lr 5.88e-04 | 102,461 tok/s | 54.64 GB | 2.082B tok iter 15900 | train 3.039 | val 3.015 | lr 5.88e-04 | 102,471 tok/s | 54.64 GB | 2.084B tok iter 15920 | train 3.007 | val 2.977 | lr 5.88e-04 | 102,481 tok/s | 54.64 GB | 2.087B tok iter 15940 | train 2.896 | val 2.990 | lr 5.88e-04 | 102,483 tok/s | 54.64 GB | 2.089B tok iter 15960 | train 3.074 | val 3.035 | lr 5.88e-04 | 102,493 tok/s | 54.64 GB | 2.092B tok iter 15980 | train 2.820 | val 2.993 | lr 5.88e-04 | 102,504 tok/s | 54.64 GB | 2.095B tok checkpoint @ iter 15991 -> checkpoints/500m_latest.pt iter 16000 | train 3.089 | val 3.005 | lr 5.88e-04 | 102,448 tok/s | 54.64 GB | 2.097B tok checkpoint @ iter 16000 -> checkpoints/500m_latest.pt iter 16020 | train 2.927 | val 3.026 | lr 5.88e-04 | 102,321 tok/s | 54.64 GB | 2.100B tok iter 16040 | train 3.044 | val 2.995 | lr 5.88e-04 | 102,331 tok/s | 54.64 GB | 2.103B tok iter 16060 | train 3.093 | val 3.054 | lr 5.88e-04 | 102,342 tok/s | 54.64 GB | 2.105B tok iter 16080 | train 2.990 | val 3.018 | lr 5.88e-04 | 102,352 tok/s | 54.64 GB | 2.108B tok iter 16100 | train 2.942 | val 3.018 | lr 5.88e-04 | 102,354 tok/s | 54.64 GB | 2.110B tok iter 16120 | train 3.000 | val 2.982 | lr 5.88e-04 | 102,365 tok/s | 54.64 GB | 2.113B tok iter 16140 | train 3.072 | val 3.015 | lr 5.88e-04 | 102,375 tok/s | 54.64 GB | 2.116B tok iter 16160 | train 3.050 | val 2.978 | lr 5.88e-04 | 102,385 tok/s | 54.64 GB | 2.118B tok iter 16180 | train 3.087 | val 3.012 | lr 5.88e-04 | 102,388 tok/s | 54.64 GB | 2.121B tok iter 16200 | train 3.036 | val 2.999 | lr 5.88e-04 | 102,398 tok/s | 54.64 GB | 2.123B tok iter 16220 | train 3.083 | val 3.042 | lr 5.87e-04 | 102,408 tok/s | 54.64 GB | 2.126B tok iter 16240 | train 3.158 | val 2.964 | lr 5.87e-04 | 102,418 tok/s | 54.64 GB | 2.129B tok iter 16260 | train 2.993 | val 3.036 | lr 5.87e-04 | 102,420 tok/s | 54.64 GB | 2.131B tok iter 16280 | train 3.052 | val 3.032 | lr 5.87e-04 | 102,429 tok/s | 54.64 GB | 2.134B tok iter 16300 | train 2.961 | val 3.004 | lr 5.87e-04 | 102,439 tok/s | 54.64 GB | 2.137B tok iter 16320 | train 3.009 | val 3.011 | lr 5.87e-04 | 102,449 tok/s | 54.64 GB | 2.139B tok iter 16340 | train 2.945 | val 3.003 | lr 5.87e-04 | 102,452 tok/s | 54.64 GB | 2.142B tok iter 16360 | train 3.066 | val 2.997 | lr 5.87e-04 | 102,461 tok/s | 54.64 GB | 2.144B tok iter 16380 | train 3.242 | val 3.033 | lr 5.87e-04 | 102,471 tok/s | 54.64 GB | 2.147B tok iter 16400 | train 2.950 | val 3.018 | lr 5.87e-04 | 102,481 tok/s | 54.64 GB | 2.150B tok iter 16420 | train 2.986 | val 3.012 | lr 5.87e-04 | 102,483 tok/s | 54.64 GB | 2.152B tok iter 16440 | train 3.036 | val 3.004 | lr 5.87e-04 | 102,493 tok/s | 54.64 GB | 2.155B tok iter 16460 | train 3.003 | val 3.045 | lr 5.87e-04 | 102,503 tok/s | 54.64 GB | 2.158B tok iter 16480 | train 3.043 | val 2.976 | lr 5.87e-04 | 102,513 tok/s | 54.64 GB | 2.160B tok checkpoint @ iter 16491 -> checkpoints/500m_latest.pt iter 16500 | train 2.966 | val 2.981 | lr 5.87e-04 | 102,448 tok/s | 54.64 GB | 2.163B tok checkpoint @ iter 16500 -> checkpoints/500m_latest.pt iter 16520 | train 3.061 | val 3.031 | lr 5.87e-04 | 102,329 tok/s | 54.64 GB | 2.165B tok iter 16540 | train 2.985 | val 3.018 | lr 5.87e-04 | 102,339 tok/s | 54.64 GB | 2.168B tok iter 16560 | train 2.897 | val 3.023 | lr 5.87e-04 | 102,349 tok/s | 54.64 GB | 2.171B tok iter 16580 | train 3.019 | val 3.006 | lr 5.87e-04 | 102,352 tok/s | 54.64 GB | 2.173B tok iter 16600 | train 2.955 | val 3.010 | lr 5.87e-04 | 102,362 tok/s | 54.64 GB | 2.176B tok iter 16620 | train 3.181 | val 3.012 | lr 5.87e-04 | 102,371 tok/s | 54.64 GB | 2.179B tok iter 16640 | train 3.066 | val 3.002 | lr 5.87e-04 | 102,381 tok/s | 54.64 GB | 2.181B tok iter 16660 | train 3.056 | val 3.000 | lr 5.87e-04 | 102,384 tok/s | 54.64 GB | 2.184B tok iter 16680 | train 3.087 | val 2.986 | lr 5.87e-04 | 102,393 tok/s | 54.64 GB | 2.186B tok iter 16700 | train 3.054 | val 2.953 | lr 5.87e-04 | 102,403 tok/s | 54.64 GB | 2.189B tok iter 16720 | train 3.027 | val 3.007 | lr 5.87e-04 | 102,413 tok/s | 54.64 GB | 2.192B tok iter 16740 | train 3.016 | val 3.017 | lr 5.87e-04 | 102,416 tok/s | 54.64 GB | 2.194B tok iter 16760 | train 2.994 | val 3.051 | lr 5.87e-04 | 102,425 tok/s | 54.64 GB | 2.197B tok iter 16780 | train 2.948 | val 3.005 | lr 5.87e-04 | 102,435 tok/s | 54.64 GB | 2.200B tok iter 16800 | train 2.907 | val 3.024 | lr 5.86e-04 | 102,445 tok/s | 54.64 GB | 2.202B tok iter 16820 | train 3.013 | val 2.982 | lr 5.86e-04 | 102,447 tok/s | 54.64 GB | 2.205B tok iter 16840 | train 2.994 | val 3.036 | lr 5.86e-04 | 102,457 tok/s | 54.64 GB | 2.207B tok iter 16860 | train 3.021 | val 2.983 | lr 5.86e-04 | 102,466 tok/s | 54.64 GB | 2.210B tok iter 16880 | train 2.889 | val 3.053 | lr 5.86e-04 | 102,476 tok/s | 54.64 GB | 2.213B tok iter 16900 | train 2.867 | val 3.012 | lr 5.86e-04 | 102,478 tok/s | 54.64 GB | 2.215B tok iter 16920 | train 2.931 | val 2.969 | lr 5.86e-04 | 102,488 tok/s | 54.64 GB | 2.218B tok iter 16940 | train 3.026 | val 3.039 | lr 5.86e-04 | 102,497 tok/s | 54.64 GB | 2.220B tok iter 16960 | train 3.081 | val 2.984 | lr 5.86e-04 | 102,507 tok/s | 54.64 GB | 2.223B tok iter 16980 | train 3.014 | val 2.989 | lr 5.86e-04 | 102,509 tok/s | 54.64 GB | 2.226B tok checkpoint @ iter 16992 -> checkpoints/500m_latest.pt iter 17000 | train 2.949 | val 3.033 | lr 5.86e-04 | 102,455 tok/s | 54.64 GB | 2.228B tok checkpoint @ iter 17000 -> checkpoints/500m_latest.pt iter 17020 | train 2.989 | val 2.988 | lr 5.86e-04 | 102,337 tok/s | 54.64 GB | 2.231B tok iter 17040 | train 2.988 | val 3.007 | lr 5.86e-04 | 102,347 tok/s | 54.64 GB | 2.234B tok iter 17060 | train 3.011 | val 3.022 | lr 5.86e-04 | 102,350 tok/s | 54.64 GB | 2.236B tok iter 17080 | train 2.975 | val 2.980 | lr 5.86e-04 | 102,359 tok/s | 54.64 GB | 2.239B tok iter 17100 | train 3.047 | val 2.957 | lr 5.86e-04 | 102,369 tok/s | 54.64 GB | 2.241B tok iter 17120 | train 3.078 | val 2.965 | lr 5.86e-04 | 102,378 tok/s | 54.64 GB | 2.244B tok iter 17140 | train 3.104 | val 3.010 | lr 5.86e-04 | 102,380 tok/s | 54.64 GB | 2.247B tok iter 17160 | train 3.017 | val 3.021 | lr 5.86e-04 | 102,390 tok/s | 54.64 GB | 2.249B tok iter 17180 | train 2.976 | val 3.003 | lr 5.86e-04 | 102,399 tok/s | 54.64 GB | 2.252B tok iter 17200 | train 2.961 | val 2.961 | lr 5.86e-04 | 102,409 tok/s | 54.64 GB | 2.255B tok iter 17220 | train 3.000 | val 3.058 | lr 5.86e-04 | 102,411 tok/s | 54.64 GB | 2.257B tok iter 17240 | train 2.932 | val 2.951 | lr 5.86e-04 | 102,421 tok/s | 54.64 GB | 2.260B tok iter 17260 | train 3.103 | val 3.002 | lr 5.86e-04 | 102,430 tok/s | 54.64 GB | 2.262B tok iter 17280 | train 3.065 | val 3.012 | lr 5.86e-04 | 102,439 tok/s | 54.64 GB | 2.265B tok iter 17300 | train 2.996 | val 3.018 | lr 5.86e-04 | 102,442 tok/s | 54.64 GB | 2.268B tok iter 17320 | train 3.031 | val 2.972 | lr 5.86e-04 | 102,451 tok/s | 54.64 GB | 2.270B tok iter 17340 | train 3.073 | val 3.001 | lr 5.86e-04 | 102,461 tok/s | 54.64 GB | 2.273B tok iter 17360 | train 2.991 | val 3.018 | lr 5.85e-04 | 102,470 tok/s | 54.64 GB | 2.276B tok iter 17380 | train 2.843 | val 3.020 | lr 5.85e-04 | 102,472 tok/s | 54.64 GB | 2.278B tok iter 17400 | train 2.985 | val 3.005 | lr 5.85e-04 | 102,482 tok/s | 54.64 GB | 2.281B tok iter 17420 | train 3.023 | val 2.994 | lr 5.85e-04 | 102,491 tok/s | 54.64 GB | 2.283B tok iter 17440 | train 3.204 | val 2.991 | lr 5.85e-04 | 102,500 tok/s | 54.64 GB | 2.286B tok iter 17460 | train 3.085 | val 2.959 | lr 5.85e-04 | 102,502 tok/s | 54.64 GB | 2.289B tok iter 17480 | train 2.898 | val 3.004 | lr 5.85e-04 | 102,511 tok/s | 54.64 GB | 2.291B tok checkpoint @ iter 17491 -> checkpoints/500m_latest.pt iter 17500 | train 3.000 | val 3.045 | lr 5.85e-04 | 102,455 tok/s | 54.64 GB | 2.294B tok checkpoint @ iter 17500 -> checkpoints/500m_latest.pt iter 17520 | train 3.057 | val 3.030 | lr 5.85e-04 | 102,338 tok/s | 54.64 GB | 2.297B tok iter 17540 | train 3.031 | val 2.948 | lr 5.85e-04 | 102,341 tok/s | 54.64 GB | 2.299B tok iter 17560 | train 3.104 | val 2.990 | lr 5.85e-04 | 102,350 tok/s | 54.64 GB | 2.302B tok iter 17580 | train 3.060 | val 3.034 | lr 5.85e-04 | 102,360 tok/s | 54.64 GB | 2.304B tok iter 17600 | train 3.010 | val 2.954 | lr 5.85e-04 | 102,369 tok/s | 54.64 GB | 2.307B tok iter 17620 | train 3.067 | val 3.031 | lr 5.85e-04 | 102,371 tok/s | 54.64 GB | 2.310B tok iter 17640 | train 3.102 | val 3.061 | lr 5.85e-04 | 102,381 tok/s | 54.64 GB | 2.312B tok iter 17660 | train 3.024 | val 2.940 | lr 5.85e-04 | 102,390 tok/s | 54.64 GB | 2.315B tok iter 17680 | train 2.928 | val 3.069 | lr 5.85e-04 | 102,399 tok/s | 54.64 GB | 2.317B tok iter 17700 | train 3.039 | val 3.019 | lr 5.85e-04 | 102,401 tok/s | 54.64 GB | 2.320B tok iter 17720 | train 3.074 | val 2.998 | lr 5.85e-04 | 102,411 tok/s | 54.64 GB | 2.323B tok iter 17740 | train 2.968 | val 3.022 | lr 5.85e-04 | 102,420 tok/s | 54.64 GB | 2.325B tok iter 17760 | train 2.965 | val 3.041 | lr 5.85e-04 | 102,429 tok/s | 54.64 GB | 2.328B tok iter 17780 | train 3.088 | val 2.961 | lr 5.85e-04 | 102,431 tok/s | 54.64 GB | 2.331B tok iter 17800 | train 3.034 | val 2.921 | lr 5.85e-04 | 102,440 tok/s | 54.64 GB | 2.333B tok iter 17820 | train 3.058 | val 2.991 | lr 5.85e-04 | 102,450 tok/s | 54.64 GB | 2.336B tok iter 17840 | train 3.086 | val 2.997 | lr 5.85e-04 | 102,459 tok/s | 54.64 GB | 2.338B tok iter 17860 | train 3.057 | val 3.022 | lr 5.85e-04 | 102,460 tok/s | 54.64 GB | 2.341B tok iter 17880 | train 3.046 | val 2.946 | lr 5.85e-04 | 102,470 tok/s | 54.64 GB | 2.344B tok iter 17900 | train 3.025 | val 2.978 | lr 5.84e-04 | 102,479 tok/s | 54.64 GB | 2.346B tok iter 17920 | train 2.877 | val 3.010 | lr 5.84e-04 | 102,488 tok/s | 54.64 GB | 2.349B tok iter 17940 | train 2.979 | val 3.023 | lr 5.84e-04 | 102,490 tok/s | 54.64 GB | 2.352B tok iter 17960 | train 3.140 | val 3.020 | lr 5.84e-04 | 102,499 tok/s | 54.64 GB | 2.354B tok iter 17980 | train 3.031 | val 3.037 | lr 5.84e-04 | 102,507 tok/s | 54.64 GB | 2.357B tok checkpoint @ iter 17991 -> checkpoints/500m_latest.pt iter 18000 | train 2.944 | val 3.030 | lr 5.84e-04 | 102,454 tok/s | 54.64 GB | 2.359B tok checkpoint @ iter 18000 -> checkpoints/500m_latest.pt iter 18020 | train 3.037 | val 2.979 | lr 5.84e-04 | 102,337 tok/s | 54.64 GB | 2.362B tok iter 18040 | train 3.009 | val 3.016 | lr 5.84e-04 | 102,346 tok/s | 54.64 GB | 2.365B tok iter 18060 | train 2.881 | val 3.003 | lr 5.84e-04 | 102,355 tok/s | 54.64 GB | 2.367B tok iter 18080 | train 2.922 | val 2.991 | lr 5.84e-04 | 102,365 tok/s | 54.64 GB | 2.370B tok iter 18100 | train 3.122 | val 2.985 | lr 5.84e-04 | 102,367 tok/s | 54.64 GB | 2.373B tok iter 18120 | train 3.007 | val 3.010 | lr 5.84e-04 | 102,375 tok/s | 54.64 GB | 2.375B tok iter 18140 | train 3.084 | val 2.969 | lr 5.84e-04 | 102,385 tok/s | 54.64 GB | 2.378B tok iter 18160 | train 2.971 | val 2.968 | lr 5.84e-04 | 102,393 tok/s | 54.64 GB | 2.380B tok iter 18180 | train 3.151 | val 3.017 | lr 5.84e-04 | 102,396 tok/s | 54.64 GB | 2.383B tok iter 18200 | train 3.032 | val 2.991 | lr 5.84e-04 | 102,405 tok/s | 54.64 GB | 2.386B tok iter 18220 | train 3.047 | val 2.995 | lr 5.84e-04 | 102,414 tok/s | 54.64 GB | 2.388B tok iter 18240 | train 3.086 | val 3.042 | lr 5.84e-04 | 102,422 tok/s | 54.64 GB | 2.391B tok iter 18260 | train 3.012 | val 3.017 | lr 5.84e-04 | 102,425 tok/s | 54.64 GB | 2.394B tok iter 18280 | train 3.072 | val 3.063 | lr 5.84e-04 | 102,433 tok/s | 54.64 GB | 2.396B tok iter 18300 | train 2.886 | val 2.980 | lr 5.84e-04 | 102,442 tok/s | 54.64 GB | 2.399B tok iter 18320 | train 2.964 | val 3.031 | lr 5.84e-04 | 102,451 tok/s | 54.64 GB | 2.401B tok iter 18340 | train 3.021 | val 3.004 | lr 5.84e-04 | 102,454 tok/s | 54.64 GB | 2.404B tok iter 18360 | train 3.169 | val 3.006 | lr 5.84e-04 | 102,462 tok/s | 54.64 GB | 2.407B tok iter 18380 | train 2.995 | val 2.970 | lr 5.84e-04 | 102,471 tok/s | 54.64 GB | 2.409B tok iter 18400 | train 2.992 | val 2.960 | lr 5.84e-04 | 102,480 tok/s | 54.64 GB | 2.412B tok iter 18420 | train 2.915 | val 2.999 | lr 5.84e-04 | 102,482 tok/s | 54.64 GB | 2.414B tok iter 18440 | train 2.993 | val 3.011 | lr 5.83e-04 | 102,491 tok/s | 54.64 GB | 2.417B tok iter 18460 | train 2.947 | val 2.996 | lr 5.83e-04 | 102,500 tok/s | 54.64 GB | 2.420B tok iter 18480 | train 3.029 | val 3.000 | lr 5.83e-04 | 102,509 tok/s | 54.64 GB | 2.422B tok checkpoint @ iter 18491 -> checkpoints/500m_latest.pt iter 18500 | train 2.855 | val 2.964 | lr 5.83e-04 | 102,452 tok/s | 54.64 GB | 2.425B tok checkpoint @ iter 18500 -> checkpoints/500m_latest.pt iter 18520 | train 3.001 | val 3.006 | lr 5.83e-04 | 102,347 tok/s | 54.64 GB | 2.428B tok iter 18540 | train 3.035 | val 2.981 | lr 5.83e-04 | 102,356 tok/s | 54.64 GB | 2.430B tok iter 18560 | train 2.987 | val 2.988 | lr 5.83e-04 | 102,365 tok/s | 54.64 GB | 2.433B tok iter 18580 | train 3.066 | val 2.982 | lr 5.83e-04 | 102,366 tok/s | 54.64 GB | 2.435B tok iter 18600 | train 2.914 | val 2.979 | lr 5.83e-04 | 102,375 tok/s | 54.64 GB | 2.438B tok iter 18620 | train 2.891 | val 3.097 | lr 5.83e-04 | 102,384 tok/s | 54.64 GB | 2.441B tok iter 18640 | train 2.913 | val 2.987 | lr 5.83e-04 | 102,393 tok/s | 54.64 GB | 2.443B tok iter 18660 | train 2.924 | val 2.962 | lr 5.83e-04 | 102,394 tok/s | 54.64 GB | 2.446B tok iter 18680 | train 3.080 | val 2.988 | lr 5.83e-04 | 102,403 tok/s | 54.64 GB | 2.449B tok iter 18700 | train 2.961 | val 2.965 | lr 5.83e-04 | 102,412 tok/s | 54.64 GB | 2.451B tok iter 18720 | train 2.969 | val 2.949 | lr 5.83e-04 | 102,420 tok/s | 54.64 GB | 2.454B tok iter 18740 | train 2.980 | val 2.993 | lr 5.83e-04 | 102,422 tok/s | 54.64 GB | 2.456B tok iter 18760 | train 3.074 | val 2.989 | lr 5.83e-04 | 102,430 tok/s | 54.64 GB | 2.459B tok iter 18780 | train 2.972 | val 3.011 | lr 5.83e-04 | 102,439 tok/s | 54.64 GB | 2.462B tok iter 18800 | train 3.067 | val 3.031 | lr 5.83e-04 | 102,448 tok/s | 54.64 GB | 2.464B tok iter 18820 | train 3.104 | val 3.000 | lr 5.83e-04 | 102,448 tok/s | 54.64 GB | 2.467B tok iter 18840 | train 2.911 | val 2.971 | lr 5.83e-04 | 102,456 tok/s | 54.64 GB | 2.470B tok iter 18860 | train 3.062 | val 3.024 | lr 5.83e-04 | 102,465 tok/s | 54.64 GB | 2.472B tok iter 18880 | train 2.992 | val 2.976 | lr 5.83e-04 | 102,474 tok/s | 54.64 GB | 2.475B tok iter 18900 | train 2.947 | val 2.993 | lr 5.83e-04 | 102,475 tok/s | 54.64 GB | 2.477B tok iter 18920 | train 2.937 | val 2.988 | lr 5.83e-04 | 102,484 tok/s | 54.64 GB | 2.480B tok iter 18940 | train 2.957 | val 3.005 | lr 5.82e-04 | 102,492 tok/s | 54.64 GB | 2.483B tok iter 18960 | train 2.946 | val 3.018 | lr 5.82e-04 | 102,501 tok/s | 54.64 GB | 2.485B tok iter 18980 | train 2.908 | val 3.006 | lr 5.82e-04 | 102,502 tok/s | 54.64 GB | 2.488B tok checkpoint @ iter 18990 -> checkpoints/500m_latest.pt iter 19000 | train 3.063 | val 3.006 | lr 5.82e-04 | 102,455 tok/s | 54.64 GB | 2.490B tok checkpoint @ iter 19000 -> checkpoints/500m_latest.pt iter 19020 | train 3.209 | val 2.991 | lr 5.82e-04 | 102,352 tok/s | 54.64 GB | 2.493B tok iter 19040 | train 3.008 | val 2.984 | lr 5.82e-04 | 102,360 tok/s | 54.64 GB | 2.496B tok iter 19060 | train 3.083 | val 3.006 | lr 5.82e-04 | 102,362 tok/s | 54.64 GB | 2.498B tok iter 19080 | train 2.965 | val 3.050 | lr 5.82e-04 | 102,371 tok/s | 54.64 GB | 2.501B tok iter 19100 | train 2.991 | val 2.967 | lr 5.82e-04 | 102,379 tok/s | 54.64 GB | 2.504B tok iter 19120 | train 2.877 | val 2.983 | lr 5.82e-04 | 102,388 tok/s | 54.64 GB | 2.506B tok iter 19140 | train 2.908 | val 2.967 | lr 5.82e-04 | 102,389 tok/s | 54.64 GB | 2.509B tok iter 19160 | train 3.014 | val 2.918 | lr 5.82e-04 | 102,398 tok/s | 54.64 GB | 2.511B tok iter 19180 | train 2.989 | val 3.001 | lr 5.82e-04 | 102,406 tok/s | 54.64 GB | 2.514B tok iter 19200 | train 2.928 | val 2.988 | lr 5.82e-04 | 102,414 tok/s | 54.64 GB | 2.517B tok iter 19220 | train 3.078 | val 3.011 | lr 5.82e-04 | 102,416 tok/s | 54.64 GB | 2.519B tok iter 19240 | train 2.947 | val 3.019 | lr 5.82e-04 | 102,424 tok/s | 54.64 GB | 2.522B tok iter 19260 | train 2.983 | val 2.951 | lr 5.82e-04 | 102,432 tok/s | 54.64 GB | 2.525B tok iter 19280 | train 3.027 | val 2.988 | lr 5.82e-04 | 102,441 tok/s | 54.64 GB | 2.527B tok iter 19300 | train 3.062 | val 2.995 | lr 5.82e-04 | 102,442 tok/s | 54.64 GB | 2.530B tok iter 19320 | train 2.943 | val 2.995 | lr 5.82e-04 | 102,450 tok/s | 54.64 GB | 2.532B tok iter 19340 | train 3.022 | val 2.988 | lr 5.82e-04 | 102,459 tok/s | 54.64 GB | 2.535B tok iter 19360 | train 2.910 | val 2.986 | lr 5.82e-04 | 102,467 tok/s | 54.64 GB | 2.538B tok iter 19380 | train 3.062 | val 2.967 | lr 5.82e-04 | 102,468 tok/s | 54.64 GB | 2.540B tok iter 19400 | train 2.942 | val 3.030 | lr 5.82e-04 | 102,477 tok/s | 54.64 GB | 2.543B tok iter 19420 | train 3.013 | val 2.979 | lr 5.82e-04 | 102,485 tok/s | 54.64 GB | 2.546B tok iter 19440 | train 3.118 | val 2.967 | lr 5.81e-04 | 102,493 tok/s | 54.64 GB | 2.548B tok iter 19460 | train 3.132 | val 2.962 | lr 5.81e-04 | 102,495 tok/s | 54.64 GB | 2.551B tok iter 19480 | train 2.814 | val 3.027 | lr 5.81e-04 | 102,503 tok/s | 54.64 GB | 2.553B tok checkpoint @ iter 19490 -> checkpoints/500m_latest.pt iter 19500 | train 2.901 | val 3.023 | lr 5.81e-04 | 102,458 tok/s | 54.64 GB | 2.556B tok checkpoint @ iter 19500 -> checkpoints/500m_latest.pt iter 19520 | train 3.084 | val 2.989 | lr 5.81e-04 | 102,361 tok/s | 54.64 GB | 2.559B tok iter 19540 | train 2.964 | val 2.941 | lr 5.81e-04 | 102,362 tok/s | 54.64 GB | 2.561B tok iter 19560 | train 3.045 | val 2.981 | lr 5.81e-04 | 102,371 tok/s | 54.64 GB | 2.564B tok iter 19580 | train 2.975 | val 3.021 | lr 5.81e-04 | 102,379 tok/s | 54.64 GB | 2.567B tok iter 19600 | train 2.808 | val 3.023 | lr 5.81e-04 | 102,388 tok/s | 54.64 GB | 2.569B tok iter 19620 | train 2.931 | val 2.950 | lr 5.81e-04 | 102,389 tok/s | 54.64 GB | 2.572B tok iter 19640 | train 3.000 | val 3.004 | lr 5.81e-04 | 102,396 tok/s | 54.64 GB | 2.574B tok iter 19660 | train 3.123 | val 2.972 | lr 5.81e-04 | 102,405 tok/s | 54.64 GB | 2.577B tok iter 19680 | train 2.888 | val 3.001 | lr 5.81e-04 | 102,413 tok/s | 54.64 GB | 2.580B tok iter 19700 | train 3.091 | val 3.016 | lr 5.81e-04 | 102,414 tok/s | 54.64 GB | 2.582B tok iter 19720 | train 3.098 | val 2.964 | lr 5.81e-04 | 102,423 tok/s | 54.64 GB | 2.585B tok iter 19740 | train 3.043 | val 2.969 | lr 5.81e-04 | 102,431 tok/s | 54.64 GB | 2.587B tok iter 19760 | train 3.052 | val 2.991 | lr 5.81e-04 | 102,439 tok/s | 54.64 GB | 2.590B tok iter 19780 | train 3.000 | val 2.956 | lr 5.81e-04 | 102,440 tok/s | 54.64 GB | 2.593B tok iter 19800 | train 2.908 | val 2.981 | lr 5.81e-04 | 102,449 tok/s | 54.64 GB | 2.595B tok iter 19820 | train 2.960 | val 2.972 | lr 5.81e-04 | 102,457 tok/s | 54.64 GB | 2.598B tok iter 19840 | train 2.832 | val 2.974 | lr 5.81e-04 | 102,465 tok/s | 54.64 GB | 2.601B tok iter 19860 | train 3.080 | val 2.988 | lr 5.81e-04 | 102,466 tok/s | 54.64 GB | 2.603B tok iter 19880 | train 3.005 | val 2.975 | lr 5.81e-04 | 102,475 tok/s | 54.64 GB | 2.606B tok iter 19900 | train 2.916 | val 2.974 | lr 5.81e-04 | 102,483 tok/s | 54.64 GB | 2.608B tok iter 19920 | train 3.057 | val 2.980 | lr 5.80e-04 | 102,491 tok/s | 54.64 GB | 2.611B tok iter 19940 | train 3.063 | val 2.981 | lr 5.80e-04 | 102,492 tok/s | 54.64 GB | 2.614B tok iter 19960 | train 3.039 | val 3.014 | lr 5.80e-04 | 102,500 tok/s | 54.64 GB | 2.616B tok iter 19980 | train 3.021 | val 2.989 | lr 5.80e-04 | 102,508 tok/s | 54.64 GB | 2.619B tok checkpoint @ iter 19990 -> checkpoints/500m_latest.pt iter 20000 | train 2.867 | val 2.955 | lr 5.80e-04 | 102,465 tok/s | 54.64 GB | 2.622B tok checkpoint @ iter 20000 -> checkpoints/500m_latest.pt iter 20020 | train 2.865 | val 2.967 | lr 5.80e-04 | 102,364 tok/s | 54.64 GB | 2.624B tok iter 20040 | train 2.946 | val 2.973 | lr 5.80e-04 | 102,372 tok/s | 54.64 GB | 2.627B tok iter 20060 | train 3.060 | val 2.985 | lr 5.80e-04 | 102,380 tok/s | 54.64 GB | 2.629B tok iter 20080 | train 2.938 | val 3.004 | lr 5.80e-04 | 102,388 tok/s | 54.64 GB | 2.632B tok iter 20100 | train 3.050 | val 2.988 | lr 5.80e-04 | 102,390 tok/s | 54.64 GB | 2.635B tok iter 20120 | train 3.069 | val 2.976 | lr 5.80e-04 | 102,398 tok/s | 54.64 GB | 2.637B tok iter 20140 | train 2.973 | val 2.972 | lr 5.80e-04 | 102,406 tok/s | 54.64 GB | 2.640B tok iter 20160 | train 3.017 | val 3.010 | lr 5.80e-04 | 102,414 tok/s | 54.64 GB | 2.643B tok iter 20180 | train 2.949 | val 2.970 | lr 5.80e-04 | 102,415 tok/s | 54.64 GB | 2.645B tok iter 20200 | train 3.000 | val 2.931 | lr 5.80e-04 | 102,423 tok/s | 54.64 GB | 2.648B tok iter 20220 | train 2.968 | val 2.992 | lr 5.80e-04 | 102,431 tok/s | 54.64 GB | 2.650B tok iter 20240 | train 2.929 | val 3.000 | lr 5.80e-04 | 102,439 tok/s | 54.64 GB | 2.653B tok iter 20260 | train 2.952 | val 3.010 | lr 5.80e-04 | 102,441 tok/s | 54.64 GB | 2.656B tok iter 20280 | train 2.947 | val 2.977 | lr 5.80e-04 | 102,449 tok/s | 54.64 GB | 2.658B tok iter 20300 | train 3.064 | val 2.967 | lr 5.80e-04 | 102,457 tok/s | 54.64 GB | 2.661B tok iter 20320 | train 2.971 | val 2.976 | lr 5.80e-04 | 102,465 tok/s | 54.64 GB | 2.664B tok iter 20340 | train 2.978 | val 3.006 | lr 5.80e-04 | 102,466 tok/s | 54.64 GB | 2.666B tok iter 20360 | train 2.946 | val 2.940 | lr 5.80e-04 | 102,474 tok/s | 54.64 GB | 2.669B tok iter 20380 | train 3.199 | val 3.015 | lr 5.80e-04 | 102,482 tok/s | 54.64 GB | 2.671B tok iter 20400 | train 2.982 | val 2.905 | lr 5.79e-04 | 102,490 tok/s | 54.64 GB | 2.674B tok iter 20420 | train 2.974 | val 2.934 | lr 5.79e-04 | 102,491 tok/s | 54.64 GB | 2.677B tok iter 20440 | train 2.994 | val 2.956 | lr 5.79e-04 | 102,499 tok/s | 54.64 GB | 2.679B tok iter 20460 | train 2.957 | val 2.970 | lr 5.79e-04 | 102,507 tok/s | 54.64 GB | 2.682B tok iter 20480 | train 2.937 | val 2.973 | lr 5.79e-04 | 102,514 tok/s | 54.64 GB | 2.684B tok checkpoint @ iter 20491 -> checkpoints/500m_latest.pt iter 20500 | train 2.867 | val 2.970 | lr 5.79e-04 | 102,462 tok/s | 54.64 GB | 2.687B tok checkpoint @ iter 20500 -> checkpoints/500m_latest.pt iter 20520 | train 2.952 | val 2.988 | lr 5.79e-04 | 102,366 tok/s | 54.64 GB | 2.690B tok iter 20540 | train 3.012 | val 2.985 | lr 5.79e-04 | 102,374 tok/s | 54.64 GB | 2.692B tok iter 20560 | train 2.840 | val 2.958 | lr 5.79e-04 | 102,382 tok/s | 54.64 GB | 2.695B tok iter 20580 | train 2.954 | val 2.993 | lr 5.79e-04 | 102,384 tok/s | 54.64 GB | 2.698B tok iter 20600 | train 2.971 | val 3.024 | lr 5.79e-04 | 102,392 tok/s | 54.64 GB | 2.700B tok iter 20620 | train 2.981 | val 2.965 | lr 5.79e-04 | 102,400 tok/s | 54.64 GB | 2.703B tok iter 20640 | train 2.929 | val 2.992 | lr 5.79e-04 | 102,407 tok/s | 54.64 GB | 2.705B tok iter 20660 | train 2.914 | val 2.966 | lr 5.79e-04 | 102,409 tok/s | 54.64 GB | 2.708B tok iter 20680 | train 2.931 | val 2.998 | lr 5.79e-04 | 102,417 tok/s | 54.64 GB | 2.711B tok iter 20700 | train 2.962 | val 2.972 | lr 5.79e-04 | 102,425 tok/s | 54.64 GB | 2.713B tok iter 20720 | train 3.049 | val 2.984 | lr 5.79e-04 | 102,432 tok/s | 54.64 GB | 2.716B tok iter 20740 | train 2.878 | val 2.969 | lr 5.79e-04 | 102,434 tok/s | 54.64 GB | 2.719B tok iter 20760 | train 2.982 | val 2.976 | lr 5.79e-04 | 102,441 tok/s | 54.64 GB | 2.721B tok iter 20780 | train 2.915 | val 2.944 | lr 5.79e-04 | 102,449 tok/s | 54.64 GB | 2.724B tok iter 20800 | train 2.957 | val 2.963 | lr 5.79e-04 | 102,457 tok/s | 54.64 GB | 2.726B tok iter 20820 | train 2.985 | val 2.978 | lr 5.79e-04 | 102,458 tok/s | 54.64 GB | 2.729B tok iter 20840 | train 2.770 | val 2.958 | lr 5.79e-04 | 102,466 tok/s | 54.64 GB | 2.732B tok iter 20860 | train 2.969 | val 2.984 | lr 5.78e-04 | 102,473 tok/s | 54.64 GB | 2.734B tok iter 20880 | train 2.900 | val 2.926 | lr 5.78e-04 | 102,481 tok/s | 54.64 GB | 2.737B tok iter 20900 | train 2.815 | val 2.973 | lr 5.78e-04 | 102,482 tok/s | 54.64 GB | 2.740B tok iter 20920 | train 2.919 | val 2.976 | lr 5.78e-04 | 102,490 tok/s | 54.64 GB | 2.742B tok iter 20940 | train 2.905 | val 2.969 | lr 5.78e-04 | 102,498 tok/s | 54.64 GB | 2.745B tok iter 20960 | train 2.864 | val 3.003 | lr 5.78e-04 | 102,506 tok/s | 54.64 GB | 2.747B tok iter 20980 | train 3.006 | val 2.974 | lr 5.78e-04 | 102,507 tok/s | 54.64 GB | 2.750B tok checkpoint @ iter 20990 -> checkpoints/500m_latest.pt iter 21000 | train 2.961 | val 2.999 | lr 5.78e-04 | 102,466 tok/s | 54.64 GB | 2.753B tok checkpoint @ iter 21000 -> checkpoints/500m_latest.pt iter 21020 | train 3.041 | val 2.988 | lr 5.78e-04 | 102,373 tok/s | 54.64 GB | 2.755B tok iter 21040 | train 2.905 | val 2.994 | lr 5.78e-04 | 102,381 tok/s | 54.64 GB | 2.758B tok iter 21060 | train 3.077 | val 2.956 | lr 5.78e-04 | 102,383 tok/s | 54.64 GB | 2.761B tok iter 21080 | train 2.951 | val 2.943 | lr 5.78e-04 | 102,391 tok/s | 54.64 GB | 2.763B tok iter 21100 | train 3.019 | val 2.997 | lr 5.78e-04 | 102,398 tok/s | 54.64 GB | 2.766B tok iter 21120 | train 2.934 | val 2.999 | lr 5.78e-04 | 102,406 tok/s | 54.64 GB | 2.768B tok iter 21140 | train 3.048 | val 3.012 | lr 5.78e-04 | 102,407 tok/s | 54.64 GB | 2.771B tok iter 21160 | train 2.873 | val 3.017 | lr 5.78e-04 | 102,415 tok/s | 54.64 GB | 2.774B tok iter 21180 | train 2.786 | val 2.989 | lr 5.78e-04 | 102,423 tok/s | 54.64 GB | 2.776B tok iter 21200 | train 3.004 | val 2.969 | lr 5.78e-04 | 102,430 tok/s | 54.64 GB | 2.779B tok iter 21220 | train 2.914 | val 2.926 | lr 5.78e-04 | 102,432 tok/s | 54.64 GB | 2.781B tok iter 21240 | train 2.994 | val 2.957 | lr 5.78e-04 | 102,439 tok/s | 54.64 GB | 2.784B tok iter 21260 | train 2.961 | val 2.973 | lr 5.78e-04 | 102,447 tok/s | 54.64 GB | 2.787B tok iter 21280 | train 2.925 | val 2.968 | lr 5.78e-04 | 102,455 tok/s | 54.64 GB | 2.789B tok iter 21300 | train 3.049 | val 2.953 | lr 5.77e-04 | 102,456 tok/s | 54.64 GB | 2.792B tok iter 21320 | train 2.996 | val 2.959 | lr 5.77e-04 | 102,463 tok/s | 54.64 GB | 2.795B tok iter 21340 | train 2.830 | val 3.014 | lr 5.77e-04 | 102,470 tok/s | 54.64 GB | 2.797B tok iter 21360 | train 2.953 | val 2.967 | lr 5.77e-04 | 102,478 tok/s | 54.64 GB | 2.800B tok iter 21380 | train 3.011 | val 2.972 | lr 5.77e-04 | 102,479 tok/s | 54.64 GB | 2.802B tok iter 21400 | train 3.057 | val 2.983 | lr 5.77e-04 | 102,486 tok/s | 54.64 GB | 2.805B tok iter 21420 | train 2.983 | val 2.977 | lr 5.77e-04 | 102,494 tok/s | 54.64 GB | 2.808B tok iter 21440 | train 3.178 | val 2.968 | lr 5.77e-04 | 102,501 tok/s | 54.64 GB | 2.810B tok iter 21460 | train 2.979 | val 2.969 | lr 5.77e-04 | 102,503 tok/s | 54.64 GB | 2.813B tok iter 21480 | train 2.941 | val 2.952 | lr 5.77e-04 | 102,510 tok/s | 54.64 GB | 2.816B tok checkpoint @ iter 21490 -> checkpoints/500m_latest.pt iter 21500 | train 2.971 | val 2.973 | lr 5.77e-04 | 102,469 tok/s | 54.64 GB | 2.818B tok checkpoint @ iter 21500 -> checkpoints/500m_latest.pt iter 21520 | train 2.991 | val 2.989 | lr 5.77e-04 | 102,376 tok/s | 54.64 GB | 2.821B tok iter 21540 | train 2.899 | val 3.022 | lr 5.77e-04 | 102,378 tok/s | 54.64 GB | 2.823B tok iter 21560 | train 2.931 | val 2.941 | lr 5.77e-04 | 102,385 tok/s | 54.64 GB | 2.826B tok iter 21580 | train 3.045 | val 2.958 | lr 5.77e-04 | 102,393 tok/s | 54.64 GB | 2.829B tok iter 21600 | train 2.951 | val 2.969 | lr 5.77e-04 | 102,401 tok/s | 54.64 GB | 2.831B tok iter 21620 | train 2.946 | val 2.992 | lr 5.77e-04 | 102,402 tok/s | 54.64 GB | 2.834B tok iter 21640 | train 3.023 | val 2.989 | lr 5.77e-04 | 102,410 tok/s | 54.64 GB | 2.837B tok iter 21660 | train 2.979 | val 2.995 | lr 5.77e-04 | 102,417 tok/s | 54.64 GB | 2.839B tok iter 21680 | train 2.969 | val 2.953 | lr 5.77e-04 | 102,425 tok/s | 54.64 GB | 2.842B tok iter 21700 | train 2.960 | val 2.995 | lr 5.77e-04 | 102,426 tok/s | 54.64 GB | 2.844B tok iter 21720 | train 3.111 | val 2.962 | lr 5.77e-04 | 102,434 tok/s | 54.64 GB | 2.847B tok iter 21740 | train 2.888 | val 2.966 | lr 5.76e-04 | 102,441 tok/s | 54.64 GB | 2.850B tok iter 21760 | train 2.955 | val 2.977 | lr 5.76e-04 | 102,448 tok/s | 54.64 GB | 2.852B tok iter 21780 | train 2.969 | val 2.936 | lr 5.76e-04 | 102,450 tok/s | 54.64 GB | 2.855B tok iter 21800 | train 2.943 | val 2.982 | lr 5.76e-04 | 102,457 tok/s | 54.64 GB | 2.858B tok iter 21820 | train 3.122 | val 2.994 | lr 5.76e-04 | 102,465 tok/s | 54.64 GB | 2.860B tok iter 21840 | train 2.974 | val 2.897 | lr 5.76e-04 | 102,472 tok/s | 54.64 GB | 2.863B tok iter 21860 | train 3.016 | val 2.981 | lr 5.76e-04 | 102,473 tok/s | 54.64 GB | 2.865B tok iter 21880 | train 2.841 | val 2.921 | lr 5.76e-04 | 102,481 tok/s | 54.64 GB | 2.868B tok iter 21900 | train 2.969 | val 2.989 | lr 5.76e-04 | 102,488 tok/s | 54.64 GB | 2.871B tok iter 21920 | train 2.945 | val 2.942 | lr 5.76e-04 | 102,495 tok/s | 54.64 GB | 2.873B tok iter 21940 | train 2.889 | val 2.969 | lr 5.76e-04 | 102,496 tok/s | 54.64 GB | 2.876B tok iter 21960 | train 2.842 | val 2.978 | lr 5.76e-04 | 102,504 tok/s | 54.64 GB | 2.878B tok iter 21980 | train 2.904 | val 2.968 | lr 5.76e-04 | 102,511 tok/s | 54.64 GB | 2.881B tok checkpoint @ iter 21990 -> checkpoints/500m_latest.pt iter 22000 | train 3.035 | val 2.989 | lr 5.76e-04 | 102,470 tok/s | 54.64 GB | 2.884B tok checkpoint @ iter 22000 -> checkpoints/500m_latest.pt iter 22020 | train 2.934 | val 2.972 | lr 5.76e-04 | 102,378 tok/s | 54.64 GB | 2.886B tok iter 22040 | train 3.010 | val 2.996 | lr 5.76e-04 | 102,385 tok/s | 54.64 GB | 2.889B tok iter 22060 | train 3.003 | val 2.982 | lr 5.76e-04 | 102,393 tok/s | 54.64 GB | 2.892B tok iter 22080 | train 2.952 | val 2.958 | lr 5.76e-04 | 102,400 tok/s | 54.64 GB | 2.894B tok iter 22100 | train 2.935 | val 2.927 | lr 5.76e-04 | 102,401 tok/s | 54.64 GB | 2.897B tok iter 22120 | train 2.988 | val 2.997 | lr 5.76e-04 | 102,409 tok/s | 54.64 GB | 2.899B tok iter 22140 | train 2.848 | val 2.916 | lr 5.76e-04 | 102,416 tok/s | 54.64 GB | 2.902B tok iter 22160 | train 2.866 | val 2.931 | lr 5.76e-04 | 102,422 tok/s | 54.64 GB | 2.905B tok iter 22180 | train 2.981 | val 2.994 | lr 5.75e-04 | 102,424 tok/s | 54.64 GB | 2.907B tok iter 22200 | train 3.088 | val 2.969 | lr 5.75e-04 | 102,431 tok/s | 54.64 GB | 2.910B tok iter 22220 | train 2.875 | val 2.980 | lr 5.75e-04 | 102,438 tok/s | 54.64 GB | 2.913B tok iter 22240 | train 2.816 | val 2.932 | lr 5.75e-04 | 102,446 tok/s | 54.64 GB | 2.915B tok iter 22260 | train 2.890 | val 2.951 | lr 5.75e-04 | 102,447 tok/s | 54.64 GB | 2.918B tok iter 22280 | train 2.838 | val 2.936 | lr 5.75e-04 | 102,454 tok/s | 54.64 GB | 2.920B tok iter 22300 | train 3.068 | val 2.993 | lr 5.75e-04 | 102,462 tok/s | 54.64 GB | 2.923B tok iter 22320 | train 3.084 | val 2.983 | lr 5.75e-04 | 102,469 tok/s | 54.64 GB | 2.926B tok iter 22340 | train 2.936 | val 2.954 | lr 5.75e-04 | 102,470 tok/s | 54.64 GB | 2.928B tok iter 22360 | train 2.980 | val 2.959 | lr 5.75e-04 | 102,477 tok/s | 54.64 GB | 2.931B tok iter 22380 | train 2.916 | val 2.989 | lr 5.75e-04 | 102,484 tok/s | 54.64 GB | 2.934B tok iter 22400 | train 2.935 | val 2.992 | lr 5.75e-04 | 102,491 tok/s | 54.64 GB | 2.936B tok iter 22420 | train 3.151 | val 2.968 | lr 5.75e-04 | 102,492 tok/s | 54.64 GB | 2.939B tok iter 22440 | train 2.887 | val 2.948 | lr 5.75e-04 | 102,500 tok/s | 54.64 GB | 2.941B tok iter 22460 | train 3.146 | val 2.964 | lr 5.75e-04 | 102,507 tok/s | 54.64 GB | 2.944B tok iter 22480 | train 3.029 | val 2.963 | lr 5.75e-04 | 102,514 tok/s | 54.64 GB | 2.947B tok checkpoint @ iter 22490 -> checkpoints/500m_latest.pt iter 22500 | train 2.816 | val 2.942 | lr 5.75e-04 | 102,468 tok/s | 54.64 GB | 2.949B tok checkpoint @ iter 22500 -> checkpoints/500m_latest.pt iter 22520 | train 2.949 | val 2.930 | lr 5.75e-04 | 102,381 tok/s | 54.64 GB | 2.952B tok iter 22540 | train 3.028 | val 2.912 | lr 5.75e-04 | 102,388 tok/s | 54.64 GB | 2.954B tok iter 22560 | train 2.966 | val 2.956 | lr 5.75e-04 | 102,396 tok/s | 54.64 GB | 2.957B tok iter 22580 | train 2.908 | val 2.923 | lr 5.75e-04 | 102,397 tok/s | 54.64 GB | 2.960B tok iter 22600 | train 3.061 | val 2.941 | lr 5.74e-04 | 102,404 tok/s | 54.64 GB | 2.962B tok iter 22620 | train 2.842 | val 2.997 | lr 5.74e-04 | 102,411 tok/s | 54.64 GB | 2.965B tok iter 22640 | train 3.019 | val 2.986 | lr 5.74e-04 | 102,419 tok/s | 54.64 GB | 2.968B tok iter 22660 | train 2.975 | val 2.997 | lr 5.74e-04 | 102,420 tok/s | 54.64 GB | 2.970B tok iter 22680 | train 2.856 | val 2.994 | lr 5.74e-04 | 102,427 tok/s | 54.64 GB | 2.973B tok iter 22700 | train 2.935 | val 2.955 | lr 5.74e-04 | 102,434 tok/s | 54.64 GB | 2.975B tok iter 22720 | train 2.937 | val 2.972 | lr 5.74e-04 | 102,441 tok/s | 54.64 GB | 2.978B tok iter 22740 | train 3.061 | val 2.952 | lr 5.74e-04 | 102,443 tok/s | 54.64 GB | 2.981B tok iter 22760 | train 2.888 | val 2.963 | lr 5.74e-04 | 102,450 tok/s | 54.64 GB | 2.983B tok iter 22780 | train 2.874 | val 2.957 | lr 5.74e-04 | 102,457 tok/s | 54.64 GB | 2.986B tok iter 22800 | train 2.972 | val 2.943 | lr 5.74e-04 | 102,464 tok/s | 54.64 GB | 2.989B tok iter 22820 | train 3.034 | val 2.983 | lr 5.74e-04 | 102,465 tok/s | 54.64 GB | 2.991B tok iter 22840 | train 3.076 | val 3.007 | lr 5.74e-04 | 102,472 tok/s | 54.64 GB | 2.994B tok iter 22860 | train 3.009 | val 2.958 | lr 5.74e-04 | 102,479 tok/s | 54.64 GB | 2.996B tok iter 22880 | train 2.953 | val 2.964 | lr 5.74e-04 | 102,486 tok/s | 54.64 GB | 2.999B tok iter 22900 | train 3.039 | val 2.942 | lr 5.74e-04 | 102,487 tok/s | 54.64 GB | 3.002B tok iter 22920 | train 3.018 | val 2.912 | lr 5.74e-04 | 102,494 tok/s | 54.64 GB | 3.004B tok iter 22940 | train 2.897 | val 2.991 | lr 5.74e-04 | 102,501 tok/s | 54.64 GB | 3.007B tok iter 22960 | train 2.699 | val 2.986 | lr 5.74e-04 | 102,508 tok/s | 54.64 GB | 3.010B tok iter 22980 | train 2.954 | val 2.980 | lr 5.74e-04 | 102,509 tok/s | 54.64 GB | 3.012B tok checkpoint @ iter 22990 -> checkpoints/500m_latest.pt iter 23000 | train 2.894 | val 2.976 | lr 5.74e-04 | 102,471 tok/s | 54.64 GB | 3.015B tok checkpoint @ iter 23000 -> checkpoints/500m_latest.pt iter 23020 | train 2.860 | val 2.968 | lr 5.73e-04 | 102,388 tok/s | 54.64 GB | 3.017B tok iter 23040 | train 3.004 | val 2.964 | lr 5.73e-04 | 102,395 tok/s | 54.64 GB | 3.020B tok iter 23060 | train 2.913 | val 2.991 | lr 5.73e-04 | 102,396 tok/s | 54.64 GB | 3.023B tok iter 23080 | train 2.912 | val 3.034 | lr 5.73e-04 | 102,403 tok/s | 54.64 GB | 3.025B tok iter 23100 | train 3.029 | val 2.981 | lr 5.73e-04 | 102,410 tok/s | 54.64 GB | 3.028B tok iter 23120 | train 2.998 | val 2.927 | lr 5.73e-04 | 102,417 tok/s | 54.64 GB | 3.031B tok iter 23140 | train 2.957 | val 2.983 | lr 5.73e-04 | 102,418 tok/s | 54.64 GB | 3.033B tok iter 23160 | train 2.985 | val 2.972 | lr 5.73e-04 | 102,425 tok/s | 54.64 GB | 3.036B tok iter 23180 | train 2.917 | val 2.982 | lr 5.73e-04 | 102,432 tok/s | 54.64 GB | 3.038B tok iter 23200 | train 2.816 | val 2.955 | lr 5.73e-04 | 102,439 tok/s | 54.64 GB | 3.041B tok iter 23220 | train 2.971 | val 3.002 | lr 5.73e-04 | 102,440 tok/s | 54.64 GB | 3.044B tok iter 23240 | train 2.963 | val 2.949 | lr 5.73e-04 | 102,447 tok/s | 54.64 GB | 3.046B tok iter 23260 | train 3.005 | val 2.956 | lr 5.73e-04 | 102,454 tok/s | 54.64 GB | 3.049B tok iter 23280 | train 2.952 | val 2.999 | lr 5.73e-04 | 102,461 tok/s | 54.64 GB | 3.051B tok iter 23300 | train 3.071 | val 2.953 | lr 5.73e-04 | 102,462 tok/s | 54.64 GB | 3.054B tok iter 23320 | train 2.967 | val 2.981 | lr 5.73e-04 | 102,469 tok/s | 54.64 GB | 3.057B tok iter 23340 | train 3.050 | val 2.942 | lr 5.73e-04 | 102,476 tok/s | 54.64 GB | 3.059B tok iter 23360 | train 2.979 | val 2.976 | lr 5.73e-04 | 102,483 tok/s | 54.64 GB | 3.062B tok iter 23380 | train 3.017 | val 2.992 | lr 5.73e-04 | 102,484 tok/s | 54.64 GB | 3.065B tok iter 23400 | train 3.011 | val 2.953 | lr 5.73e-04 | 102,491 tok/s | 54.64 GB | 3.067B tok iter 23420 | train 2.794 | val 2.967 | lr 5.72e-04 | 102,498 tok/s | 54.64 GB | 3.070B tok iter 23440 | train 2.959 | val 2.916 | lr 5.72e-04 | 102,505 tok/s | 54.64 GB | 3.072B tok iter 23460 | train 2.969 | val 2.967 | lr 5.72e-04 | 102,506 tok/s | 54.64 GB | 3.075B tok iter 23480 | train 2.859 | val 2.962 | lr 5.72e-04 | 102,513 tok/s | 54.64 GB | 3.078B tok checkpoint @ iter 23490 -> checkpoints/500m_latest.pt iter 23500 | train 2.869 | val 2.891 | lr 5.72e-04 | 102,475 tok/s | 54.64 GB | 3.080B tok checkpoint @ iter 23500 -> checkpoints/500m_latest.pt iter 23520 | train 2.841 | val 2.945 | lr 5.72e-04 | 102,395 tok/s | 54.64 GB | 3.083B tok iter 23540 | train 2.933 | val 2.991 | lr 5.72e-04 | 102,396 tok/s | 54.64 GB | 3.086B tok iter 23560 | train 2.996 | val 2.946 | lr 5.72e-04 | 102,403 tok/s | 54.64 GB | 3.088B tok iter 23580 | train 2.923 | val 2.951 | lr 5.72e-04 | 102,410 tok/s | 54.64 GB | 3.091B tok iter 23600 | train 3.027 | val 3.001 | lr 5.72e-04 | 102,417 tok/s | 54.64 GB | 3.093B tok iter 23620 | train 2.893 | val 2.965 | lr 5.72e-04 | 102,418 tok/s | 54.64 GB | 3.096B tok iter 23640 | train 3.036 | val 2.987 | lr 5.72e-04 | 102,425 tok/s | 54.64 GB | 3.099B tok iter 23660 | train 2.855 | val 3.000 | lr 5.72e-04 | 102,432 tok/s | 54.64 GB | 3.101B tok iter 23680 | train 2.967 | val 3.005 | lr 5.72e-04 | 102,439 tok/s | 54.64 GB | 3.104B tok iter 23700 | train 3.008 | val 2.979 | lr 5.72e-04 | 102,440 tok/s | 54.64 GB | 3.107B tok iter 23720 | train 2.844 | val 2.937 | lr 5.72e-04 | 102,447 tok/s | 54.64 GB | 3.109B tok iter 23740 | train 3.041 | val 2.959 | lr 5.72e-04 | 102,453 tok/s | 54.64 GB | 3.112B tok iter 23760 | train 2.953 | val 3.014 | lr 5.72e-04 | 102,460 tok/s | 54.64 GB | 3.114B tok iter 23780 | train 2.964 | val 2.928 | lr 5.72e-04 | 102,461 tok/s | 54.64 GB | 3.117B tok iter 23800 | train 2.875 | val 3.003 | lr 5.72e-04 | 102,468 tok/s | 54.64 GB | 3.120B tok iter 23820 | train 2.723 | val 2.977 | lr 5.71e-04 | 102,475 tok/s | 54.64 GB | 3.122B tok iter 23840 | train 2.969 | val 2.964 | lr 5.71e-04 | 102,481 tok/s | 54.64 GB | 3.125B tok iter 23860 | train 3.049 | val 2.961 | lr 5.71e-04 | 102,482 tok/s | 54.64 GB | 3.128B tok iter 23880 | train 2.941 | val 2.944 | lr 5.71e-04 | 102,489 tok/s | 54.64 GB | 3.130B tok iter 23900 | train 2.967 | val 2.965 | lr 5.71e-04 | 102,495 tok/s | 54.64 GB | 3.133B tok iter 23920 | train 3.018 | val 2.903 | lr 5.71e-04 | 102,502 tok/s | 54.64 GB | 3.135B tok iter 23940 | train 3.067 | val 3.002 | lr 5.71e-04 | 102,503 tok/s | 54.64 GB | 3.138B tok iter 23960 | train 2.881 | val 2.944 | lr 5.71e-04 | 102,510 tok/s | 54.64 GB | 3.141B tok iter 23980 | train 2.964 | val 2.940 | lr 5.71e-04 | 102,517 tok/s | 54.64 GB | 3.143B tok checkpoint @ iter 23991 -> checkpoints/500m_latest.pt iter 24000 | train 2.886 | val 2.935 | lr 5.71e-04 | 102,478 tok/s | 54.64 GB | 3.146B tok checkpoint @ iter 24000 -> checkpoints/500m_latest.pt iter 24020 | train 2.913 | val 2.992 | lr 5.71e-04 | 102,391 tok/s | 54.64 GB | 3.148B tok iter 24040 | train 2.893 | val 2.943 | lr 5.71e-04 | 102,398 tok/s | 54.64 GB | 3.151B tok iter 24060 | train 3.055 | val 2.961 | lr 5.71e-04 | 102,404 tok/s | 54.64 GB | 3.154B tok iter 24080 | train 2.900 | val 2.972 | lr 5.71e-04 | 102,411 tok/s | 54.64 GB | 3.156B tok iter 24100 | train 2.958 | val 2.939 | lr 5.71e-04 | 102,412 tok/s | 54.64 GB | 3.159B tok iter 24120 | train 2.925 | val 2.930 | lr 5.71e-04 | 102,419 tok/s | 54.64 GB | 3.162B tok iter 24140 | train 2.935 | val 2.968 | lr 5.71e-04 | 102,426 tok/s | 54.64 GB | 3.164B tok iter 24160 | train 2.988 | val 2.970 | lr 5.71e-04 | 102,433 tok/s | 54.64 GB | 3.167B tok iter 24180 | train 2.973 | val 2.943 | lr 5.71e-04 | 102,434 tok/s | 54.64 GB | 3.169B tok iter 24200 | train 3.032 | val 2.967 | lr 5.71e-04 | 102,440 tok/s | 54.64 GB | 3.172B tok iter 24220 | train 2.944 | val 2.955 | lr 5.70e-04 | 102,447 tok/s | 54.64 GB | 3.175B tok iter 24240 | train 2.886 | val 2.993 | lr 5.70e-04 | 102,454 tok/s | 54.64 GB | 3.177B tok iter 24260 | train 2.845 | val 2.913 | lr 5.70e-04 | 102,455 tok/s | 54.64 GB | 3.180B tok iter 24280 | train 2.859 | val 2.942 | lr 5.70e-04 | 102,462 tok/s | 54.64 GB | 3.183B tok iter 24300 | train 3.035 | val 2.941 | lr 5.70e-04 | 102,468 tok/s | 54.64 GB | 3.185B tok iter 24320 | train 3.004 | val 2.919 | lr 5.70e-04 | 102,475 tok/s | 54.64 GB | 3.188B tok iter 24340 | train 2.846 | val 2.950 | lr 5.70e-04 | 102,476 tok/s | 54.64 GB | 3.190B tok iter 24360 | train 2.962 | val 2.976 | lr 5.70e-04 | 102,483 tok/s | 54.64 GB | 3.193B tok iter 24380 | train 3.000 | val 2.964 | lr 5.70e-04 | 102,489 tok/s | 54.64 GB | 3.196B tok iter 24400 | train 2.930 | val 2.934 | lr 5.70e-04 | 102,496 tok/s | 54.64 GB | 3.198B tok iter 24420 | train 2.911 | val 2.999 | lr 5.70e-04 | 102,497 tok/s | 54.64 GB | 3.201B tok iter 24440 | train 2.930 | val 2.933 | lr 5.70e-04 | 102,504 tok/s | 54.64 GB | 3.204B tok iter 24460 | train 2.799 | val 2.938 | lr 5.70e-04 | 102,510 tok/s | 54.64 GB | 3.206B tok iter 24480 | train 2.911 | val 2.934 | lr 5.70e-04 | 102,517 tok/s | 54.64 GB | 3.209B tok checkpoint @ iter 24491 -> checkpoints/500m_latest.pt iter 24500 | train 3.080 | val 2.975 | lr 5.70e-04 | 102,474 tok/s | 54.64 GB | 3.211B tok checkpoint @ iter 24500 -> checkpoints/500m_latest.pt iter 24520 | train 2.963 | val 2.897 | lr 5.70e-04 | 102,394 tok/s | 54.64 GB | 3.214B tok iter 24540 | train 2.978 | val 2.974 | lr 5.70e-04 | 102,401 tok/s | 54.64 GB | 3.217B tok iter 24560 | train 2.954 | val 2.925 | lr 5.70e-04 | 102,408 tok/s | 54.64 GB | 3.219B tok iter 24580 | train 2.851 | val 2.957 | lr 5.70e-04 | 102,409 tok/s | 54.64 GB | 3.222B tok iter 24600 | train 2.856 | val 2.965 | lr 5.70e-04 | 102,415 tok/s | 54.64 GB | 3.225B tok iter 24620 | train 2.787 | val 2.965 | lr 5.69e-04 | 102,422 tok/s | 54.64 GB | 3.227B tok iter 24640 | train 2.970 | val 2.957 | lr 5.69e-04 | 102,428 tok/s | 54.64 GB | 3.230B tok iter 24660 | train 2.943 | val 2.945 | lr 5.69e-04 | 102,430 tok/s | 54.64 GB | 3.232B tok iter 24680 | train 3.061 | val 2.938 | lr 5.69e-04 | 102,436 tok/s | 54.64 GB | 3.235B tok iter 24700 | train 2.742 | val 2.948 | lr 5.69e-04 | 102,442 tok/s | 54.64 GB | 3.238B tok iter 24720 | train 2.960 | val 2.902 | lr 5.69e-04 | 102,449 tok/s | 54.64 GB | 3.240B tok iter 24740 | train 2.946 | val 2.972 | lr 5.69e-04 | 102,450 tok/s | 54.64 GB | 3.243B tok iter 24760 | train 2.863 | val 2.929 | lr 5.69e-04 | 102,456 tok/s | 54.64 GB | 3.245B tok iter 24780 | train 2.940 | val 2.903 | lr 5.69e-04 | 102,463 tok/s | 54.64 GB | 3.248B tok iter 24800 | train 2.807 | val 2.957 | lr 5.69e-04 | 102,469 tok/s | 54.64 GB | 3.251B tok iter 24820 | train 2.912 | val 2.999 | lr 5.69e-04 | 102,471 tok/s | 54.64 GB | 3.253B tok iter 24840 | train 2.979 | val 2.950 | lr 5.69e-04 | 102,477 tok/s | 54.64 GB | 3.256B tok iter 24860 | train 2.884 | val 2.974 | lr 5.69e-04 | 102,483 tok/s | 54.64 GB | 3.259B tok iter 24880 | train 2.975 | val 2.933 | lr 5.69e-04 | 102,490 tok/s | 54.64 GB | 3.261B tok iter 24900 | train 2.966 | val 3.008 | lr 5.69e-04 | 102,491 tok/s | 54.64 GB | 3.264B tok iter 24920 | train 3.035 | val 2.947 | lr 5.69e-04 | 102,497 tok/s | 54.64 GB | 3.266B tok iter 24940 | train 3.016 | val 3.002 | lr 5.69e-04 | 102,504 tok/s | 54.64 GB | 3.269B tok iter 24960 | train 3.057 | val 2.998 | lr 5.69e-04 | 102,510 tok/s | 54.64 GB | 3.272B tok iter 24980 | train 2.881 | val 2.971 | lr 5.69e-04 | 102,511 tok/s | 54.64 GB | 3.274B tok checkpoint @ iter 24990 -> checkpoints/500m_latest.pt iter 25000 | train 2.964 | val 2.969 | lr 5.68e-04 | 102,475 tok/s | 54.64 GB | 3.277B tok checkpoint @ iter 25000 -> checkpoints/500m_latest.pt iter 25020 | train 2.975 | val 2.953 | lr 5.68e-04 | 102,398 tok/s | 54.64 GB | 3.280B tok iter 25040 | train 2.872 | val 2.946 | lr 5.68e-04 | 102,405 tok/s | 54.64 GB | 3.282B tok iter 25060 | train 2.896 | val 2.910 | lr 5.68e-04 | 102,406 tok/s | 54.64 GB | 3.285B tok iter 25080 | train 2.946 | val 2.956 | lr 5.68e-04 | 102,413 tok/s | 54.64 GB | 3.287B tok iter 25100 | train 2.984 | val 2.980 | lr 5.68e-04 | 102,419 tok/s | 54.64 GB | 3.290B tok iter 25120 | train 3.008 | val 2.945 | lr 5.68e-04 | 102,426 tok/s | 54.64 GB | 3.293B tok iter 25140 | train 3.035 | val 2.944 | lr 5.68e-04 | 102,427 tok/s | 54.64 GB | 3.295B tok iter 25160 | train 2.974 | val 2.928 | lr 5.68e-04 | 102,433 tok/s | 54.64 GB | 3.298B tok iter 25180 | train 2.986 | val 2.929 | lr 5.68e-04 | 102,440 tok/s | 54.64 GB | 3.301B tok iter 25200 | train 2.977 | val 2.993 | lr 5.68e-04 | 102,446 tok/s | 54.64 GB | 3.303B tok iter 25220 | train 2.945 | val 2.987 | lr 5.68e-04 | 102,447 tok/s | 54.64 GB | 3.306B tok iter 25240 | train 2.994 | val 2.974 | lr 5.68e-04 | 102,454 tok/s | 54.64 GB | 3.308B tok iter 25260 | train 2.984 | val 2.961 | lr 5.68e-04 | 102,460 tok/s | 54.64 GB | 3.311B tok iter 25280 | train 2.920 | val 3.008 | lr 5.68e-04 | 102,466 tok/s | 54.64 GB | 3.314B tok iter 25300 | train 2.957 | val 2.959 | lr 5.68e-04 | 102,467 tok/s | 54.64 GB | 3.316B tok iter 25320 | train 2.849 | val 2.915 | lr 5.68e-04 | 102,474 tok/s | 54.64 GB | 3.319B tok iter 25340 | train 3.041 | val 2.952 | lr 5.68e-04 | 102,480 tok/s | 54.64 GB | 3.321B tok iter 25360 | train 2.991 | val 2.944 | lr 5.68e-04 | 102,486 tok/s | 54.64 GB | 3.324B tok iter 25380 | train 2.861 | val 2.947 | lr 5.67e-04 | 102,487 tok/s | 54.64 GB | 3.327B tok iter 25400 | train 3.005 | val 2.959 | lr 5.67e-04 | 102,494 tok/s | 54.64 GB | 3.329B tok iter 25420 | train 2.997 | val 2.923 | lr 5.67e-04 | 102,500 tok/s | 54.64 GB | 3.332B tok iter 25440 | train 3.106 | val 2.966 | lr 5.67e-04 | 102,506 tok/s | 54.64 GB | 3.335B tok iter 25460 | train 2.934 | val 2.979 | lr 5.67e-04 | 102,507 tok/s | 54.64 GB | 3.337B tok iter 25480 | train 2.817 | val 2.916 | lr 5.67e-04 | 102,514 tok/s | 54.64 GB | 3.340B tok checkpoint @ iter 25491 -> checkpoints/500m_latest.pt iter 25500 | train 3.011 | val 2.997 | lr 5.67e-04 | 102,478 tok/s | 54.64 GB | 3.342B tok checkpoint @ iter 25500 -> checkpoints/500m_latest.pt iter 25520 | train 2.905 | val 2.939 | lr 5.67e-04 | 102,402 tok/s | 54.64 GB | 3.345B tok iter 25540 | train 2.917 | val 3.004 | lr 5.67e-04 | 102,403 tok/s | 54.64 GB | 3.348B tok iter 25560 | train 3.077 | val 2.986 | lr 5.67e-04 | 102,409 tok/s | 54.64 GB | 3.350B tok iter 25580 | train 2.980 | val 2.953 | lr 5.67e-04 | 102,416 tok/s | 54.64 GB | 3.353B tok iter 25600 | train 2.908 | val 2.883 | lr 5.67e-04 | 102,422 tok/s | 54.64 GB | 3.356B tok iter 25620 | train 2.950 | val 2.963 | lr 5.67e-04 | 102,423 tok/s | 54.64 GB | 3.358B tok iter 25640 | train 2.799 | val 2.892 | lr 5.67e-04 | 102,430 tok/s | 54.64 GB | 3.361B tok iter 25660 | train 2.993 | val 2.956 | lr 5.67e-04 | 102,436 tok/s | 54.64 GB | 3.363B tok iter 25680 | train 3.145 | val 2.965 | lr 5.67e-04 | 102,442 tok/s | 54.64 GB | 3.366B tok iter 25700 | train 2.853 | val 2.958 | lr 5.67e-04 | 102,443 tok/s | 54.64 GB | 3.369B tok iter 25720 | train 2.978 | val 2.979 | lr 5.67e-04 | 102,449 tok/s | 54.64 GB | 3.371B tok iter 25740 | train 2.914 | val 2.943 | lr 5.66e-04 | 102,456 tok/s | 54.64 GB | 3.374B tok iter 25760 | train 2.936 | val 2.951 | lr 5.66e-04 | 102,462 tok/s | 54.64 GB | 3.377B tok iter 25780 | train 3.009 | val 2.933 | lr 5.66e-04 | 102,463 tok/s | 54.64 GB | 3.379B tok iter 25800 | train 2.818 | val 2.941 | lr 5.66e-04 | 102,469 tok/s | 54.64 GB | 3.382B tok iter 25820 | train 2.953 | val 2.939 | lr 5.66e-04 | 102,475 tok/s | 54.64 GB | 3.384B tok iter 25840 | train 2.949 | val 2.950 | lr 5.66e-04 | 102,482 tok/s | 54.64 GB | 3.387B tok iter 25860 | train 2.900 | val 2.930 | lr 5.66e-04 | 102,483 tok/s | 54.64 GB | 3.390B tok iter 25880 | train 2.939 | val 2.937 | lr 5.66e-04 | 102,489 tok/s | 54.64 GB | 3.392B tok iter 25900 | train 2.909 | val 2.924 | lr 5.66e-04 | 102,495 tok/s | 54.64 GB | 3.395B tok iter 25920 | train 3.088 | val 2.940 | lr 5.66e-04 | 102,501 tok/s | 54.64 GB | 3.398B tok iter 25940 | train 2.949 | val 2.929 | lr 5.66e-04 | 102,502 tok/s | 54.64 GB | 3.400B tok iter 25960 | train 2.816 | val 2.968 | lr 5.66e-04 | 102,509 tok/s | 54.64 GB | 3.403B tok iter 25980 | train 2.814 | val 2.961 | lr 5.66e-04 | 102,515 tok/s | 54.64 GB | 3.405B tok checkpoint @ iter 25990 -> checkpoints/500m_latest.pt iter 26000 | train 2.939 | val 2.954 | lr 5.66e-04 | 102,482 tok/s | 54.64 GB | 3.408B tok checkpoint @ iter 26000 -> checkpoints/500m_latest.pt iter 26020 | train 3.064 | val 2.968 | lr 5.66e-04 | 102,401 tok/s | 54.64 GB | 3.411B tok iter 26040 | train 2.815 | val 2.958 | lr 5.66e-04 | 102,408 tok/s | 54.64 GB | 3.413B tok iter 26060 | train 3.036 | val 2.949 | lr 5.66e-04 | 102,414 tok/s | 54.64 GB | 3.416B tok iter 26080 | train 3.025 | val 2.898 | lr 5.66e-04 | 102,420 tok/s | 54.64 GB | 3.418B tok iter 26100 | train 2.781 | val 2.967 | lr 5.66e-04 | 102,421 tok/s | 54.64 GB | 3.421B tok iter 26120 | train 2.812 | val 2.978 | lr 5.65e-04 | 102,427 tok/s | 54.64 GB | 3.424B tok iter 26140 | train 3.045 | val 2.965 | lr 5.65e-04 | 102,434 tok/s | 54.64 GB | 3.426B tok iter 26160 | train 2.969 | val 2.934 | lr 5.65e-04 | 102,440 tok/s | 54.64 GB | 3.429B tok iter 26180 | train 2.846 | val 2.986 | lr 5.65e-04 | 102,441 tok/s | 54.64 GB | 3.432B tok iter 26200 | train 2.972 | val 2.955 | lr 5.65e-04 | 102,447 tok/s | 54.64 GB | 3.434B tok iter 26220 | train 2.972 | val 3.004 | lr 5.65e-04 | 102,453 tok/s | 54.64 GB | 3.437B tok iter 26240 | train 2.927 | val 2.991 | lr 5.65e-04 | 102,459 tok/s | 54.64 GB | 3.439B tok iter 26260 | train 2.857 | val 2.954 | lr 5.65e-04 | 102,460 tok/s | 54.64 GB | 3.442B tok iter 26280 | train 2.958 | val 2.979 | lr 5.65e-04 | 102,466 tok/s | 54.64 GB | 3.445B tok iter 26300 | train 3.065 | val 2.947 | lr 5.65e-04 | 102,473 tok/s | 54.64 GB | 3.447B tok iter 26320 | train 2.930 | val 2.921 | lr 5.65e-04 | 102,479 tok/s | 54.64 GB | 3.450B tok iter 26340 | train 2.885 | val 2.924 | lr 5.65e-04 | 102,480 tok/s | 54.64 GB | 3.453B tok iter 26360 | train 3.032 | val 2.998 | lr 5.65e-04 | 102,486 tok/s | 54.64 GB | 3.455B tok iter 26380 | train 2.971 | val 2.922 | lr 5.65e-04 | 102,491 tok/s | 54.64 GB | 3.458B tok iter 26400 | train 2.929 | val 2.923 | lr 5.65e-04 | 102,497 tok/s | 54.64 GB | 3.460B tok iter 26420 | train 2.978 | val 2.960 | lr 5.65e-04 | 102,498 tok/s | 54.64 GB | 3.463B tok iter 26440 | train 2.858 | val 2.920 | lr 5.65e-04 | 102,504 tok/s | 54.64 GB | 3.466B tok iter 26460 | train 2.958 | val 2.970 | lr 5.65e-04 | 102,511 tok/s | 54.64 GB | 3.468B tok iter 26480 | train 2.809 | val 2.960 | lr 5.64e-04 | 102,517 tok/s | 54.64 GB | 3.471B tok checkpoint @ iter 26490 -> checkpoints/500m_latest.pt iter 26500 | train 2.875 | val 2.919 | lr 5.64e-04 | 102,479 tok/s | 54.64 GB | 3.474B tok checkpoint @ iter 26500 -> checkpoints/500m_latest.pt iter 26520 | train 2.949 | val 2.943 | lr 5.64e-04 | 102,404 tok/s | 54.64 GB | 3.476B tok iter 26540 | train 2.814 | val 2.984 | lr 5.64e-04 | 102,411 tok/s | 54.64 GB | 3.479B tok iter 26560 | train 2.900 | val 2.965 | lr 5.64e-04 | 102,417 tok/s | 54.64 GB | 3.481B tok iter 26580 | train 2.930 | val 2.932 | lr 5.64e-04 | 102,418 tok/s | 54.64 GB | 3.484B tok iter 26600 | train 2.956 | val 2.908 | lr 5.64e-04 | 102,424 tok/s | 54.64 GB | 3.487B tok iter 26620 | train 2.657 | val 2.940 | lr 5.64e-04 | 102,430 tok/s | 54.64 GB | 3.489B tok iter 26640 | train 2.970 | val 2.931 | lr 5.64e-04 | 102,436 tok/s | 54.64 GB | 3.492B tok iter 26660 | train 2.936 | val 2.960 | lr 5.64e-04 | 102,437 tok/s | 54.64 GB | 3.495B tok iter 26680 | train 3.019 | val 2.980 | lr 5.64e-04 | 102,443 tok/s | 54.64 GB | 3.497B tok iter 26700 | train 2.924 | val 2.938 | lr 5.64e-04 | 102,449 tok/s | 54.64 GB | 3.500B tok iter 26720 | train 2.871 | val 2.976 | lr 5.64e-04 | 102,455 tok/s | 54.64 GB | 3.502B tok iter 26740 | train 2.849 | val 2.887 | lr 5.64e-04 | 102,456 tok/s | 54.64 GB | 3.505B tok iter 26760 | train 2.990 | val 2.996 | lr 5.64e-04 | 102,462 tok/s | 54.64 GB | 3.508B tok iter 26780 | train 2.891 | val 2.907 | lr 5.64e-04 | 102,468 tok/s | 54.64 GB | 3.510B tok iter 26800 | train 2.769 | val 2.945 | lr 5.64e-04 | 102,474 tok/s | 54.64 GB | 3.513B tok iter 26820 | train 2.958 | val 2.958 | lr 5.63e-04 | 102,475 tok/s | 54.64 GB | 3.515B tok iter 26840 | train 2.882 | val 2.917 | lr 5.63e-04 | 102,481 tok/s | 54.64 GB | 3.518B tok iter 26860 | train 2.724 | val 2.953 | lr 5.63e-04 | 102,487 tok/s | 54.64 GB | 3.521B tok iter 26880 | train 2.943 | val 2.946 | lr 5.63e-04 | 102,493 tok/s | 54.64 GB | 3.523B tok iter 26900 | train 3.027 | val 2.927 | lr 5.63e-04 | 102,494 tok/s | 54.64 GB | 3.526B tok iter 26920 | train 2.868 | val 2.976 | lr 5.63e-04 | 102,500 tok/s | 54.64 GB | 3.529B tok iter 26940 | train 2.925 | val 2.948 | lr 5.63e-04 | 102,506 tok/s | 54.64 GB | 3.531B tok iter 26960 | train 2.833 | val 2.923 | lr 5.63e-04 | 102,512 tok/s | 54.64 GB | 3.534B tok iter 26980 | train 2.960 | val 2.946 | lr 5.63e-04 | 102,513 tok/s | 54.64 GB | 3.536B tok checkpoint @ iter 26990 -> checkpoints/500m_latest.pt iter 27000 | train 3.024 | val 2.926 | lr 5.63e-04 | 102,479 tok/s | 54.64 GB | 3.539B tok checkpoint @ iter 27000 -> checkpoints/500m_latest.pt iter 27020 | train 3.091 | val 2.893 | lr 5.63e-04 | 102,407 tok/s | 54.64 GB | 3.542B tok iter 27040 | train 2.801 | val 2.930 | lr 5.63e-04 | 102,413 tok/s | 54.64 GB | 3.544B tok iter 27060 | train 3.002 | val 2.928 | lr 5.63e-04 | 102,414 tok/s | 54.64 GB | 3.547B tok iter 27080 | train 2.927 | val 2.984 | lr 5.63e-04 | 102,420 tok/s | 54.64 GB | 3.550B tok iter 27100 | train 2.951 | val 2.973 | lr 5.63e-04 | 102,426 tok/s | 54.64 GB | 3.552B tok iter 27120 | train 2.957 | val 2.919 | lr 5.63e-04 | 102,432 tok/s | 54.64 GB | 3.555B tok iter 27140 | train 2.853 | val 2.959 | lr 5.63e-04 | 102,433 tok/s | 54.64 GB | 3.557B tok iter 27160 | train 3.060 | val 2.939 | lr 5.63e-04 | 102,439 tok/s | 54.64 GB | 3.560B tok iter 27180 | train 2.974 | val 2.907 | lr 5.62e-04 | 102,445 tok/s | 54.64 GB | 3.563B tok iter 27200 | train 3.097 | val 2.915 | lr 5.62e-04 | 102,451 tok/s | 54.64 GB | 3.565B tok iter 27220 | train 2.967 | val 2.959 | lr 5.62e-04 | 102,451 tok/s | 54.64 GB | 3.568B tok iter 27240 | train 2.853 | val 2.932 | lr 5.62e-04 | 102,457 tok/s | 54.64 GB | 3.571B tok iter 27260 | train 2.963 | val 2.943 | lr 5.62e-04 | 102,463 tok/s | 54.64 GB | 3.573B tok iter 27280 | train 3.013 | val 2.959 | lr 5.62e-04 | 102,469 tok/s | 54.64 GB | 3.576B tok iter 27300 | train 2.953 | val 2.948 | lr 5.62e-04 | 102,470 tok/s | 54.64 GB | 3.578B tok iter 27320 | train 2.804 | val 2.956 | lr 5.62e-04 | 102,476 tok/s | 54.64 GB | 3.581B tok iter 27340 | train 2.919 | val 2.921 | lr 5.62e-04 | 102,482 tok/s | 54.64 GB | 3.584B tok iter 27360 | train 2.941 | val 2.900 | lr 5.62e-04 | 102,488 tok/s | 54.64 GB | 3.586B tok iter 27380 | train 2.939 | val 2.926 | lr 5.62e-04 | 102,489 tok/s | 54.64 GB | 3.589B tok iter 27400 | train 2.930 | val 2.913 | lr 5.62e-04 | 102,495 tok/s | 54.64 GB | 3.592B tok iter 27420 | train 2.935 | val 2.941 | lr 5.62e-04 | 102,501 tok/s | 54.64 GB | 3.594B tok iter 27440 | train 2.904 | val 3.025 | lr 5.62e-04 | 102,506 tok/s | 54.64 GB | 3.597B tok iter 27460 | train 2.893 | val 2.927 | lr 5.62e-04 | 102,507 tok/s | 54.64 GB | 3.599B tok iter 27480 | train 2.887 | val 2.981 | lr 5.62e-04 | 102,513 tok/s | 54.64 GB | 3.602B tok checkpoint @ iter 27490 -> checkpoints/500m_latest.pt iter 27500 | train 2.803 | val 2.967 | lr 5.62e-04 | 102,481 tok/s | 54.64 GB | 3.605B tok checkpoint @ iter 27500 -> checkpoints/500m_latest.pt iter 27520 | train 2.877 | val 2.950 | lr 5.61e-04 | 102,412 tok/s | 54.64 GB | 3.607B tok iter 27540 | train 3.046 | val 2.926 | lr 5.61e-04 | 102,413 tok/s | 54.64 GB | 3.610B tok iter 27560 | train 2.964 | val 2.892 | lr 5.61e-04 | 102,419 tok/s | 54.64 GB | 3.612B tok iter 27580 | train 2.865 | val 2.849 | lr 5.61e-04 | 102,425 tok/s | 54.64 GB | 3.615B tok iter 27600 | train 2.835 | val 2.974 | lr 5.61e-04 | 102,431 tok/s | 54.64 GB | 3.618B tok iter 27620 | train 2.834 | val 2.956 | lr 5.61e-04 | 102,432 tok/s | 54.64 GB | 3.620B tok iter 27640 | train 3.034 | val 2.950 | lr 5.61e-04 | 102,438 tok/s | 54.64 GB | 3.623B tok iter 27660 | train 2.888 | val 2.995 | lr 5.61e-04 | 102,444 tok/s | 54.64 GB | 3.626B tok iter 27680 | train 2.918 | val 2.903 | lr 5.61e-04 | 102,450 tok/s | 54.64 GB | 3.628B tok iter 27700 | train 2.954 | val 2.953 | lr 5.61e-04 | 102,451 tok/s | 54.64 GB | 3.631B tok iter 27720 | train 3.024 | val 2.939 | lr 5.61e-04 | 102,457 tok/s | 54.64 GB | 3.633B tok iter 27740 | train 2.944 | val 2.921 | lr 5.61e-04 | 102,462 tok/s | 54.64 GB | 3.636B tok iter 27760 | train 2.911 | val 2.947 | lr 5.61e-04 | 102,468 tok/s | 54.64 GB | 3.639B tok iter 27780 | train 2.974 | val 2.913 | lr 5.61e-04 | 102,469 tok/s | 54.64 GB | 3.641B tok iter 27800 | train 2.910 | val 2.929 | lr 5.61e-04 | 102,475 tok/s | 54.64 GB | 3.644B tok iter 27820 | train 2.943 | val 2.973 | lr 5.61e-04 | 102,481 tok/s | 54.64 GB | 3.647B tok iter 27840 | train 2.814 | val 2.934 | lr 5.61e-04 | 102,487 tok/s | 54.64 GB | 3.649B tok iter 27860 | train 2.860 | val 2.916 | lr 5.61e-04 | 102,488 tok/s | 54.64 GB | 3.652B tok iter 27880 | train 3.111 | val 2.932 | lr 5.60e-04 | 102,493 tok/s | 54.64 GB | 3.654B tok iter 27900 | train 3.044 | val 2.920 | lr 5.60e-04 | 102,499 tok/s | 54.64 GB | 3.657B tok iter 27920 | train 3.072 | val 2.983 | lr 5.60e-04 | 102,505 tok/s | 54.64 GB | 3.660B tok iter 27940 | train 2.972 | val 2.961 | lr 5.60e-04 | 102,506 tok/s | 54.64 GB | 3.662B tok iter 27960 | train 2.886 | val 2.903 | lr 5.60e-04 | 102,511 tok/s | 54.64 GB | 3.665B tok iter 27980 | train 2.875 | val 2.934 | lr 5.60e-04 | 102,517 tok/s | 54.64 GB | 3.668B tok checkpoint @ iter 27991 -> checkpoints/500m_latest.pt iter 28000 | train 3.058 | val 2.933 | lr 5.60e-04 | 102,486 tok/s | 54.64 GB | 3.670B tok checkpoint @ iter 28000 -> checkpoints/500m_latest.pt iter 28020 | train 2.945 | val 2.925 | lr 5.60e-04 | 102,412 tok/s | 54.64 GB | 3.673B tok iter 28040 | train 3.053 | val 2.954 | lr 5.60e-04 | 102,418 tok/s | 54.64 GB | 3.675B tok iter 28060 | train 2.955 | val 2.962 | lr 5.60e-04 | 102,423 tok/s | 54.64 GB | 3.678B tok iter 28080 | train 2.909 | val 2.974 | lr 5.60e-04 | 102,429 tok/s | 54.64 GB | 3.681B tok iter 28100 | train 2.943 | val 2.956 | lr 5.60e-04 | 102,430 tok/s | 54.64 GB | 3.683B tok iter 28120 | train 3.020 | val 2.920 | lr 5.60e-04 | 102,436 tok/s | 54.64 GB | 3.686B tok iter 28140 | train 2.816 | val 2.933 | lr 5.60e-04 | 102,442 tok/s | 54.64 GB | 3.688B tok iter 28160 | train 2.895 | val 2.935 | lr 5.60e-04 | 102,448 tok/s | 54.64 GB | 3.691B tok iter 28180 | train 2.834 | val 2.971 | lr 5.60e-04 | 102,448 tok/s | 54.64 GB | 3.694B tok iter 28200 | train 3.007 | val 2.929 | lr 5.60e-04 | 102,454 tok/s | 54.64 GB | 3.696B tok iter 28220 | train 2.877 | val 2.897 | lr 5.59e-04 | 102,460 tok/s | 54.64 GB | 3.699B tok iter 28240 | train 2.963 | val 2.932 | lr 5.59e-04 | 102,466 tok/s | 54.64 GB | 3.702B tok iter 28260 | train 3.117 | val 2.903 | lr 5.59e-04 | 102,467 tok/s | 54.64 GB | 3.704B tok iter 28280 | train 2.963 | val 2.953 | lr 5.59e-04 | 102,472 tok/s | 54.64 GB | 3.707B tok iter 28300 | train 2.975 | val 2.938 | lr 5.59e-04 | 102,478 tok/s | 54.64 GB | 3.709B tok iter 28320 | train 3.011 | val 2.996 | lr 5.59e-04 | 102,484 tok/s | 54.64 GB | 3.712B tok iter 28340 | train 2.805 | val 2.880 | lr 5.59e-04 | 102,485 tok/s | 54.64 GB | 3.715B tok iter 28360 | train 3.085 | val 2.965 | lr 5.59e-04 | 102,490 tok/s | 54.64 GB | 3.717B tok iter 28380 | train 3.006 | val 2.944 | lr 5.59e-04 | 102,496 tok/s | 54.64 GB | 3.720B tok iter 28400 | train 3.014 | val 2.937 | lr 5.59e-04 | 102,502 tok/s | 54.64 GB | 3.723B tok iter 28420 | train 2.894 | val 2.915 | lr 5.59e-04 | 102,502 tok/s | 54.64 GB | 3.725B tok iter 28440 | train 2.899 | val 2.914 | lr 5.59e-04 | 102,508 tok/s | 54.64 GB | 3.728B tok iter 28460 | train 2.929 | val 2.915 | lr 5.59e-04 | 102,514 tok/s | 54.64 GB | 3.730B tok iter 28480 | train 2.869 | val 2.930 | lr 5.59e-04 | 102,519 tok/s | 54.64 GB | 3.733B tok checkpoint @ iter 28491 -> checkpoints/500m_latest.pt iter 28500 | train 2.826 | val 2.919 | lr 5.59e-04 | 102,484 tok/s | 54.64 GB | 3.736B tok checkpoint @ iter 28500 -> checkpoints/500m_latest.pt iter 28520 | train 3.058 | val 2.946 | lr 5.59e-04 | 102,416 tok/s | 54.64 GB | 3.738B tok iter 28540 | train 2.802 | val 2.974 | lr 5.58e-04 | 102,422 tok/s | 54.64 GB | 3.741B tok iter 28560 | train 2.970 | val 2.889 | lr 5.58e-04 | 102,428 tok/s | 54.64 GB | 3.744B tok iter 28580 | train 2.964 | val 2.923 | lr 5.58e-04 | 102,429 tok/s | 54.64 GB | 3.746B tok iter 28600 | train 2.917 | val 2.888 | lr 5.58e-04 | 102,434 tok/s | 54.64 GB | 3.749B tok iter 28620 | train 2.941 | val 2.946 | lr 5.58e-04 | 102,440 tok/s | 54.64 GB | 3.751B tok iter 28640 | train 2.799 | val 2.899 | lr 5.58e-04 | 102,446 tok/s | 54.64 GB | 3.754B tok iter 28660 | train 2.816 | val 2.938 | lr 5.58e-04 | 102,447 tok/s | 54.64 GB | 3.757B tok iter 28680 | train 2.917 | val 2.915 | lr 5.58e-04 | 102,452 tok/s | 54.64 GB | 3.759B tok iter 28700 | train 2.825 | val 2.926 | lr 5.58e-04 | 102,458 tok/s | 54.64 GB | 3.762B tok iter 28720 | train 2.999 | val 2.868 | lr 5.58e-04 | 102,463 tok/s | 54.64 GB | 3.765B tok iter 28740 | train 3.051 | val 2.921 | lr 5.58e-04 | 102,464 tok/s | 54.64 GB | 3.767B tok iter 28760 | train 3.028 | val 2.939 | lr 5.58e-04 | 102,470 tok/s | 54.64 GB | 3.770B tok iter 28780 | train 2.992 | val 2.920 | lr 5.58e-04 | 102,475 tok/s | 54.64 GB | 3.772B tok iter 28800 | train 2.916 | val 2.950 | lr 5.58e-04 | 102,481 tok/s | 54.64 GB | 3.775B tok iter 28820 | train 2.916 | val 2.989 | lr 5.58e-04 | 102,482 tok/s | 54.64 GB | 3.778B tok iter 28840 | train 2.822 | val 3.002 | lr 5.58e-04 | 102,488 tok/s | 54.64 GB | 3.780B tok iter 28860 | train 3.032 | val 2.944 | lr 5.58e-04 | 102,493 tok/s | 54.64 GB | 3.783B tok iter 28880 | train 2.861 | val 2.971 | lr 5.57e-04 | 102,499 tok/s | 54.64 GB | 3.785B tok iter 28900 | train 2.889 | val 2.872 | lr 5.57e-04 | 102,500 tok/s | 54.64 GB | 3.788B tok iter 28920 | train 2.871 | val 2.922 | lr 5.57e-04 | 102,501 tok/s | 54.64 GB | 3.791B tok iter 28940 | train 2.809 | val 2.910 | lr 5.57e-04 | 102,507 tok/s | 54.64 GB | 3.793B tok iter 28960 | train 2.985 | val 3.007 | lr 5.57e-04 | 102,512 tok/s | 54.64 GB | 3.796B tok iter 28980 | train 2.929 | val 2.928 | lr 5.57e-04 | 102,513 tok/s | 54.64 GB | 3.799B tok checkpoint @ iter 28989 -> checkpoints/500m_latest.pt iter 29000 | train 2.840 | val 2.915 | lr 5.57e-04 | 102,481 tok/s | 54.64 GB | 3.801B tok checkpoint @ iter 29000 -> checkpoints/500m_latest.pt iter 29020 | train 2.600 | val 2.945 | lr 5.57e-04 | 102,413 tok/s | 54.64 GB | 3.804B tok iter 29040 | train 2.864 | val 2.985 | lr 5.57e-04 | 102,418 tok/s | 54.64 GB | 3.806B tok iter 29060 | train 2.977 | val 2.917 | lr 5.57e-04 | 102,419 tok/s | 54.64 GB | 3.809B tok iter 29080 | train 2.910 | val 2.905 | lr 5.57e-04 | 102,425 tok/s | 54.64 GB | 3.812B tok iter 29100 | train 2.901 | val 2.940 | lr 5.57e-04 | 102,431 tok/s | 54.64 GB | 3.814B tok iter 29120 | train 3.018 | val 2.920 | lr 5.57e-04 | 102,435 tok/s | 54.64 GB | 3.817B tok iter 29140 | train 2.813 | val 2.886 | lr 5.57e-04 | 102,436 tok/s | 54.64 GB | 3.820B tok iter 29160 | train 2.902 | val 2.883 | lr 5.57e-04 | 102,442 tok/s | 54.64 GB | 3.822B tok iter 29180 | train 2.859 | val 2.924 | lr 5.57e-04 | 102,447 tok/s | 54.64 GB | 3.825B tok iter 29200 | train 3.046 | val 2.897 | lr 5.57e-04 | 102,453 tok/s | 54.64 GB | 3.827B tok iter 29220 | train 2.956 | val 2.951 | lr 5.56e-04 | 102,454 tok/s | 54.64 GB | 3.830B tok iter 29240 | train 2.988 | val 2.896 | lr 5.56e-04 | 102,459 tok/s | 54.64 GB | 3.833B tok iter 29260 | train 2.794 | val 2.932 | lr 5.56e-04 | 102,465 tok/s | 54.64 GB | 3.835B tok iter 29280 | train 2.894 | val 2.922 | lr 5.56e-04 | 102,470 tok/s | 54.64 GB | 3.838B tok iter 29300 | train 2.947 | val 2.863 | lr 5.56e-04 | 102,471 tok/s | 54.64 GB | 3.841B tok iter 29320 | train 2.914 | val 2.905 | lr 5.56e-04 | 102,476 tok/s | 54.64 GB | 3.843B tok iter 29340 | train 3.008 | val 2.912 | lr 5.56e-04 | 102,482 tok/s | 54.64 GB | 3.846B tok iter 29360 | train 2.995 | val 2.889 | lr 5.56e-04 | 102,487 tok/s | 54.64 GB | 3.848B tok iter 29380 | train 2.785 | val 2.930 | lr 5.56e-04 | 102,488 tok/s | 54.64 GB | 3.851B tok iter 29400 | train 2.971 | val 2.955 | lr 5.56e-04 | 102,494 tok/s | 54.64 GB | 3.854B tok iter 29420 | train 2.844 | val 2.932 | lr 5.56e-04 | 102,499 tok/s | 54.64 GB | 3.856B tok iter 29440 | train 2.943 | val 2.927 | lr 5.56e-04 | 102,505 tok/s | 54.64 GB | 3.859B tok iter 29460 | train 2.809 | val 2.924 | lr 5.56e-04 | 102,505 tok/s | 54.64 GB | 3.862B tok iter 29480 | train 3.073 | val 2.902 | lr 5.56e-04 | 102,511 tok/s | 54.64 GB | 3.864B tok checkpoint @ iter 29490 -> checkpoints/500m_latest.pt iter 29500 | train 3.070 | val 2.936 | lr 5.56e-04 | 102,479 tok/s | 54.64 GB | 3.867B tok checkpoint @ iter 29500 -> checkpoints/500m_latest.pt iter 29520 | train 3.122 | val 2.902 | lr 5.56e-04 | 102,413 tok/s | 54.64 GB | 3.869B tok iter 29540 | train 2.973 | val 2.886 | lr 5.55e-04 | 102,414 tok/s | 54.64 GB | 3.872B tok iter 29560 | train 2.748 | val 2.928 | lr 5.55e-04 | 102,419 tok/s | 54.64 GB | 3.875B tok iter 29580 | train 2.955 | val 2.911 | lr 5.55e-04 | 102,425 tok/s | 54.64 GB | 3.877B tok iter 29600 | train 2.902 | val 2.909 | lr 5.55e-04 | 102,430 tok/s | 54.64 GB | 3.880B tok iter 29620 | train 2.912 | val 2.916 | lr 5.55e-04 | 102,431 tok/s | 54.64 GB | 3.882B tok iter 29640 | train 3.099 | val 2.922 | lr 5.55e-04 | 102,437 tok/s | 54.64 GB | 3.885B tok iter 29660 | train 2.962 | val 2.927 | lr 5.55e-04 | 102,442 tok/s | 54.64 GB | 3.888B tok iter 29680 | train 2.954 | val 2.909 | lr 5.55e-04 | 102,447 tok/s | 54.64 GB | 3.890B tok iter 29700 | train 3.062 | val 2.911 | lr 5.55e-04 | 102,448 tok/s | 54.64 GB | 3.893B tok iter 29720 | train 2.945 | val 2.930 | lr 5.55e-04 | 102,454 tok/s | 54.64 GB | 3.896B tok iter 29740 | train 2.917 | val 2.936 | lr 5.55e-04 | 102,459 tok/s | 54.64 GB | 3.898B tok iter 29760 | train 2.815 | val 2.929 | lr 5.55e-04 | 102,464 tok/s | 54.64 GB | 3.901B tok iter 29780 | train 3.051 | val 2.924 | lr 5.55e-04 | 102,465 tok/s | 54.64 GB | 3.903B tok iter 29800 | train 2.884 | val 2.901 | lr 5.55e-04 | 102,471 tok/s | 54.64 GB | 3.906B tok iter 29820 | train 2.771 | val 2.981 | lr 5.55e-04 | 102,476 tok/s | 54.64 GB | 3.909B tok iter 29840 | train 2.997 | val 2.939 | lr 5.55e-04 | 102,481 tok/s | 54.64 GB | 3.911B tok iter 29860 | train 2.846 | val 2.941 | lr 5.54e-04 | 102,482 tok/s | 54.64 GB | 3.914B tok iter 29880 | train 2.921 | val 2.929 | lr 5.54e-04 | 102,488 tok/s | 54.64 GB | 3.917B tok iter 29900 | train 2.940 | val 2.958 | lr 5.54e-04 | 102,493 tok/s | 54.64 GB | 3.919B tok iter 29920 | train 2.918 | val 2.913 | lr 5.54e-04 | 102,498 tok/s | 54.64 GB | 3.922B tok iter 29940 | train 2.937 | val 2.891 | lr 5.54e-04 | 102,499 tok/s | 54.64 GB | 3.924B tok iter 29960 | train 2.808 | val 2.881 | lr 5.54e-04 | 102,505 tok/s | 54.64 GB | 3.927B tok iter 29980 | train 2.936 | val 2.937 | lr 5.54e-04 | 102,510 tok/s | 54.64 GB | 3.930B tok checkpoint @ iter 29990 -> checkpoints/500m_latest.pt iter 30000 | train 2.921 | val 2.924 | lr 5.54e-04 | 102,479 tok/s | 54.64 GB | 3.932B tok checkpoint @ iter 30000 -> checkpoints/500m_latest.pt iter 30020 | train 2.966 | val 2.922 | lr 5.54e-04 | 102,408 tok/s | 54.64 GB | 3.935B tok iter 30040 | train 2.859 | val 2.927 | lr 5.54e-04 | 102,413 tok/s | 54.64 GB | 3.938B tok iter 30060 | train 2.910 | val 2.924 | lr 5.54e-04 | 102,419 tok/s | 54.64 GB | 3.940B tok iter 30080 | train 2.987 | val 2.907 | lr 5.54e-04 | 102,424 tok/s | 54.64 GB | 3.943B tok iter 30100 | train 2.910 | val 2.942 | lr 5.54e-04 | 102,425 tok/s | 54.64 GB | 3.945B tok iter 30120 | train 2.943 | val 2.926 | lr 5.54e-04 | 102,431 tok/s | 54.64 GB | 3.948B tok iter 30140 | train 2.999 | val 2.918 | lr 5.54e-04 | 102,436 tok/s | 54.64 GB | 3.951B tok iter 30160 | train 2.938 | val 2.898 | lr 5.54e-04 | 102,441 tok/s | 54.64 GB | 3.953B tok iter 30180 | train 2.931 | val 2.883 | lr 5.53e-04 | 102,442 tok/s | 54.64 GB | 3.956B tok iter 30200 | train 3.008 | val 2.918 | lr 5.53e-04 | 102,448 tok/s | 54.64 GB | 3.959B tok iter 30220 | train 2.860 | val 2.892 | lr 5.53e-04 | 102,453 tok/s | 54.64 GB | 3.961B tok iter 30240 | train 3.000 | val 2.887 | lr 5.53e-04 | 102,458 tok/s | 54.64 GB | 3.964B tok iter 30260 | train 2.998 | val 2.932 | lr 5.53e-04 | 102,459 tok/s | 54.64 GB | 3.966B tok iter 30280 | train 2.893 | val 2.949 | lr 5.53e-04 | 102,465 tok/s | 54.64 GB | 3.969B tok iter 30300 | train 2.934 | val 2.981 | lr 5.53e-04 | 102,470 tok/s | 54.64 GB | 3.972B tok iter 30320 | train 2.991 | val 2.916 | lr 5.53e-04 | 102,475 tok/s | 54.64 GB | 3.974B tok iter 30340 | train 2.880 | val 2.945 | lr 5.53e-04 | 102,476 tok/s | 54.64 GB | 3.977B tok iter 30360 | train 2.853 | val 2.911 | lr 5.53e-04 | 102,481 tok/s | 54.64 GB | 3.979B tok iter 30380 | train 2.824 | val 2.923 | lr 5.53e-04 | 102,487 tok/s | 54.64 GB | 3.982B tok iter 30400 | train 2.944 | val 2.929 | lr 5.53e-04 | 102,492 tok/s | 54.64 GB | 3.985B tok iter 30420 | train 2.854 | val 2.892 | lr 5.53e-04 | 102,493 tok/s | 54.64 GB | 3.987B tok iter 30440 | train 3.001 | val 2.896 | lr 5.53e-04 | 102,498 tok/s | 54.64 GB | 3.990B tok iter 30460 | train 2.870 | val 2.901 | lr 5.53e-04 | 102,503 tok/s | 54.64 GB | 3.993B tok iter 30480 | train 2.943 | val 2.953 | lr 5.53e-04 | 102,509 tok/s | 54.64 GB | 3.995B tok checkpoint @ iter 30490 -> checkpoints/500m_latest.pt iter 30500 | train 2.869 | val 2.938 | lr 5.52e-04 | 102,475 tok/s | 54.64 GB | 3.998B tok checkpoint @ iter 30500 -> checkpoints/500m_latest.pt iter 30520 | train 2.900 | val 2.949 | lr 5.52e-04 | 102,410 tok/s | 54.64 GB | 4.000B tok iter 30540 | train 2.976 | val 2.856 | lr 5.52e-04 | 102,416 tok/s | 54.64 GB | 4.003B tok iter 30560 | train 2.994 | val 2.911 | lr 5.52e-04 | 102,421 tok/s | 54.64 GB | 4.006B tok iter 30580 | train 2.876 | val 2.908 | lr 5.52e-04 | 102,422 tok/s | 54.64 GB | 4.008B tok iter 30600 | train 2.824 | val 2.912 | lr 5.52e-04 | 102,427 tok/s | 54.64 GB | 4.011B tok iter 30620 | train 2.935 | val 2.912 | lr 5.52e-04 | 102,432 tok/s | 54.64 GB | 4.014B tok iter 30640 | train 2.927 | val 2.929 | lr 5.52e-04 | 102,438 tok/s | 54.64 GB | 4.016B tok iter 30660 | train 3.074 | val 2.973 | lr 5.52e-04 | 102,438 tok/s | 54.64 GB | 4.019B tok iter 30680 | train 3.085 | val 2.925 | lr 5.52e-04 | 102,443 tok/s | 54.64 GB | 4.021B tok iter 30700 | train 2.912 | val 2.949 | lr 5.52e-04 | 102,449 tok/s | 54.64 GB | 4.024B tok iter 30720 | train 2.952 | val 2.910 | lr 5.52e-04 | 102,454 tok/s | 54.64 GB | 4.027B tok iter 30740 | train 2.935 | val 2.925 | lr 5.52e-04 | 102,455 tok/s | 54.64 GB | 4.029B tok iter 30760 | train 3.020 | val 2.928 | lr 5.52e-04 | 102,460 tok/s | 54.64 GB | 4.032B tok iter 30780 | train 2.940 | val 2.948 | lr 5.52e-04 | 102,465 tok/s | 54.64 GB | 4.035B tok iter 30800 | train 2.738 | val 2.950 | lr 5.51e-04 | 102,470 tok/s | 54.64 GB | 4.037B tok iter 30820 | train 2.871 | val 2.945 | lr 5.51e-04 | 102,471 tok/s | 54.64 GB | 4.040B tok iter 30840 | train 2.920 | val 2.951 | lr 5.51e-04 | 102,477 tok/s | 54.64 GB | 4.042B tok iter 30860 | train 2.798 | val 2.916 | lr 5.51e-04 | 102,482 tok/s | 54.64 GB | 4.045B tok iter 30880 | train 2.949 | val 2.911 | lr 5.51e-04 | 102,486 tok/s | 54.64 GB | 4.048B tok iter 30900 | train 2.963 | val 2.913 | lr 5.51e-04 | 102,487 tok/s | 54.64 GB | 4.050B tok iter 30920 | train 2.875 | val 2.915 | lr 5.51e-04 | 102,492 tok/s | 54.64 GB | 4.053B tok iter 30940 | train 2.988 | val 2.900 | lr 5.51e-04 | 102,497 tok/s | 54.64 GB | 4.055B tok iter 30960 | train 2.930 | val 2.932 | lr 5.51e-04 | 102,502 tok/s | 54.64 GB | 4.058B tok iter 30980 | train 2.948 | val 2.932 | lr 5.51e-04 | 102,503 tok/s | 54.64 GB | 4.061B tok checkpoint @ iter 30990 -> checkpoints/500m_latest.pt iter 31000 | train 3.065 | val 2.929 | lr 5.51e-04 | 102,473 tok/s | 54.64 GB | 4.063B tok checkpoint @ iter 31000 -> checkpoints/500m_latest.pt iter 31020 | train 2.918 | val 2.944 | lr 5.51e-04 | 102,409 tok/s | 54.64 GB | 4.066B tok iter 31040 | train 3.192 | val 2.978 | lr 5.51e-04 | 102,414 tok/s | 54.64 GB | 4.069B tok iter 31060 | train 2.946 | val 2.948 | lr 5.51e-04 | 102,415 tok/s | 54.64 GB | 4.071B tok iter 31080 | train 2.919 | val 2.920 | lr 5.51e-04 | 102,421 tok/s | 54.64 GB | 4.074B tok iter 31100 | train 2.906 | val 2.858 | lr 5.51e-04 | 102,426 tok/s | 54.64 GB | 4.076B tok iter 31120 | train 2.974 | val 2.861 | lr 5.50e-04 | 102,431 tok/s | 54.64 GB | 4.079B tok iter 31140 | train 2.800 | val 2.923 | lr 5.50e-04 | 102,432 tok/s | 54.64 GB | 4.082B tok iter 31160 | train 2.828 | val 2.897 | lr 5.50e-04 | 102,437 tok/s | 54.64 GB | 4.084B tok iter 31180 | train 2.946 | val 2.899 | lr 5.50e-04 | 102,442 tok/s | 54.64 GB | 4.087B tok iter 31200 | train 2.895 | val 2.936 | lr 5.50e-04 | 102,447 tok/s | 54.64 GB | 4.090B tok iter 31220 | train 2.891 | val 2.915 | lr 5.50e-04 | 102,448 tok/s | 54.64 GB | 4.092B tok iter 31240 | train 2.898 | val 2.920 | lr 5.50e-04 | 102,453 tok/s | 54.64 GB | 4.095B tok iter 31260 | train 3.013 | val 2.876 | lr 5.50e-04 | 102,458 tok/s | 54.64 GB | 4.097B tok iter 31280 | train 2.926 | val 2.915 | lr 5.50e-04 | 102,464 tok/s | 54.64 GB | 4.100B tok iter 31300 | train 2.927 | val 2.929 | lr 5.50e-04 | 102,465 tok/s | 54.64 GB | 4.103B tok iter 31320 | train 2.800 | val 2.903 | lr 5.50e-04 | 102,470 tok/s | 54.64 GB | 4.105B tok iter 31340 | train 2.806 | val 2.912 | lr 5.50e-04 | 102,475 tok/s | 54.64 GB | 4.108B tok iter 31360 | train 2.885 | val 2.950 | lr 5.50e-04 | 102,480 tok/s | 54.64 GB | 4.111B tok iter 31380 | train 2.845 | val 2.940 | lr 5.50e-04 | 102,481 tok/s | 54.64 GB | 4.113B tok iter 31400 | train 2.922 | val 2.952 | lr 5.50e-04 | 102,486 tok/s | 54.64 GB | 4.116B tok iter 31420 | train 3.017 | val 2.930 | lr 5.49e-04 | 102,491 tok/s | 54.64 GB | 4.118B tok iter 31440 | train 2.851 | val 2.936 | lr 5.49e-04 | 102,496 tok/s | 54.64 GB | 4.121B tok iter 31460 | train 2.825 | val 2.893 | lr 5.49e-04 | 102,496 tok/s | 54.64 GB | 4.124B tok iter 31480 | train 2.821 | val 2.932 | lr 5.49e-04 | 102,502 tok/s | 54.64 GB | 4.126B tok checkpoint @ iter 31490 -> checkpoints/500m_latest.pt iter 31500 | train 2.860 | val 2.965 | lr 5.49e-04 | 102,473 tok/s | 54.64 GB | 4.129B tok checkpoint @ iter 31500 -> checkpoints/500m_latest.pt iter 31520 | train 2.964 | val 2.858 | lr 5.49e-04 | 102,411 tok/s | 54.64 GB | 4.132B tok iter 31540 | train 2.943 | val 2.894 | lr 5.49e-04 | 102,412 tok/s | 54.64 GB | 4.134B tok iter 31560 | train 2.855 | val 2.873 | lr 5.49e-04 | 102,417 tok/s | 54.64 GB | 4.137B tok iter 31580 | train 2.926 | val 2.912 | lr 5.49e-04 | 102,422 tok/s | 54.64 GB | 4.139B tok iter 31600 | train 2.981 | val 2.893 | lr 5.49e-04 | 102,428 tok/s | 54.64 GB | 4.142B tok iter 31620 | train 3.001 | val 2.949 | lr 5.49e-04 | 102,428 tok/s | 54.64 GB | 4.145B tok iter 31640 | train 2.914 | val 2.904 | lr 5.49e-04 | 102,434 tok/s | 54.64 GB | 4.147B tok iter 31660 | train 2.941 | val 2.916 | lr 5.49e-04 | 102,439 tok/s | 54.64 GB | 4.150B tok iter 31680 | train 2.912 | val 2.904 | lr 5.49e-04 | 102,444 tok/s | 54.64 GB | 4.152B tok iter 31700 | train 2.933 | val 2.885 | lr 5.49e-04 | 102,445 tok/s | 54.64 GB | 4.155B tok iter 31720 | train 2.847 | val 2.902 | lr 5.48e-04 | 102,450 tok/s | 54.64 GB | 4.158B tok iter 31740 | train 2.936 | val 2.852 | lr 5.48e-04 | 102,455 tok/s | 54.64 GB | 4.160B tok iter 31760 | train 2.732 | val 2.918 | lr 5.48e-04 | 102,460 tok/s | 54.64 GB | 4.163B tok iter 31780 | train 2.896 | val 2.924 | lr 5.48e-04 | 102,461 tok/s | 54.64 GB | 4.166B tok iter 31800 | train 2.855 | val 2.907 | lr 5.48e-04 | 102,466 tok/s | 54.64 GB | 4.168B tok iter 31820 | train 2.619 | val 2.939 | lr 5.48e-04 | 102,471 tok/s | 54.64 GB | 4.171B tok iter 31840 | train 2.829 | val 2.923 | lr 5.48e-04 | 102,476 tok/s | 54.64 GB | 4.173B tok iter 31860 | train 2.851 | val 2.977 | lr 5.48e-04 | 102,477 tok/s | 54.64 GB | 4.176B tok iter 31880 | train 2.924 | val 2.999 | lr 5.48e-04 | 102,482 tok/s | 54.64 GB | 4.179B tok iter 31900 | train 2.997 | val 2.902 | lr 5.48e-04 | 102,487 tok/s | 54.64 GB | 4.181B tok iter 31920 | train 2.876 | val 2.951 | lr 5.48e-04 | 102,492 tok/s | 54.64 GB | 4.184B tok iter 31940 | train 2.843 | val 2.938 | lr 5.48e-04 | 102,493 tok/s | 54.64 GB | 4.187B tok iter 31960 | train 2.868 | val 2.981 | lr 5.48e-04 | 102,498 tok/s | 54.64 GB | 4.189B tok iter 31980 | train 3.015 | val 2.889 | lr 5.48e-04 | 102,503 tok/s | 54.64 GB | 4.192B tok checkpoint @ iter 31990 -> checkpoints/500m_latest.pt iter 32000 | train 2.929 | val 2.916 | lr 5.48e-04 | 102,475 tok/s | 54.64 GB | 4.194B tok checkpoint @ iter 32000 -> checkpoints/500m_latest.pt iter 32020 | train 2.810 | val 2.929 | lr 5.48e-04 | 102,409 tok/s | 54.64 GB | 4.197B tok iter 32040 | train 3.061 | val 2.890 | lr 5.47e-04 | 102,414 tok/s | 54.64 GB | 4.200B tok iter 32060 | train 2.930 | val 2.949 | lr 5.47e-04 | 102,419 tok/s | 54.64 GB | 4.202B tok iter 32080 | train 3.023 | val 2.922 | lr 5.47e-04 | 102,424 tok/s | 54.64 GB | 4.205B tok iter 32100 | train 2.915 | val 2.938 | lr 5.47e-04 | 102,425 tok/s | 54.64 GB | 4.208B tok iter 32120 | train 2.874 | val 2.984 | lr 5.47e-04 | 102,430 tok/s | 54.64 GB | 4.210B tok iter 32140 | train 2.901 | val 2.908 | lr 5.47e-04 | 102,435 tok/s | 54.64 GB | 4.213B tok iter 32160 | train 2.827 | val 2.889 | lr 5.47e-04 | 102,440 tok/s | 54.64 GB | 4.215B tok iter 32180 | train 2.798 | val 2.929 | lr 5.47e-04 | 102,441 tok/s | 54.64 GB | 4.218B tok iter 32200 | train 2.923 | val 2.900 | lr 5.47e-04 | 102,446 tok/s | 54.64 GB | 4.221B tok iter 32220 | train 3.009 | val 2.916 | lr 5.47e-04 | 102,451 tok/s | 54.64 GB | 4.223B tok iter 32240 | train 2.831 | val 2.908 | lr 5.47e-04 | 102,456 tok/s | 54.64 GB | 4.226B tok iter 32260 | train 2.962 | val 2.952 | lr 5.47e-04 | 102,457 tok/s | 54.64 GB | 4.229B tok iter 32280 | train 3.095 | val 2.902 | lr 5.47e-04 | 102,462 tok/s | 54.64 GB | 4.231B tok iter 32300 | train 3.104 | val 2.919 | lr 5.47e-04 | 102,467 tok/s | 54.64 GB | 4.234B tok iter 32320 | train 2.910 | val 2.913 | lr 5.46e-04 | 102,472 tok/s | 54.64 GB | 4.236B tok iter 32340 | train 2.942 | val 2.919 | lr 5.46e-04 | 102,472 tok/s | 54.64 GB | 4.239B tok iter 32360 | train 2.869 | val 2.926 | lr 5.46e-04 | 102,477 tok/s | 54.64 GB | 4.242B tok iter 32380 | train 2.876 | val 2.955 | lr 5.46e-04 | 102,482 tok/s | 54.64 GB | 4.244B tok iter 32400 | train 2.821 | val 2.904 | lr 5.46e-04 | 102,487 tok/s | 54.64 GB | 4.247B tok iter 32420 | train 2.940 | val 2.894 | lr 5.46e-04 | 102,488 tok/s | 54.64 GB | 4.249B tok iter 32440 | train 3.013 | val 2.901 | lr 5.46e-04 | 102,493 tok/s | 54.64 GB | 4.252B tok iter 32460 | train 2.967 | val 2.931 | lr 5.46e-04 | 102,498 tok/s | 54.64 GB | 4.255B tok iter 32480 | train 2.872 | val 2.915 | lr 5.46e-04 | 102,503 tok/s | 54.64 GB | 4.257B tok checkpoint @ iter 32490 -> checkpoints/500m_latest.pt iter 32500 | train 2.760 | val 2.892 | lr 5.46e-04 | 102,471 tok/s | 54.64 GB | 4.260B tok checkpoint @ iter 32500 -> checkpoints/500m_latest.pt iter 32520 | train 2.923 | val 2.914 | lr 5.46e-04 | 102,411 tok/s | 54.64 GB | 4.263B tok iter 32540 | train 2.985 | val 2.953 | lr 5.46e-04 | 102,416 tok/s | 54.64 GB | 4.265B tok iter 32560 | train 2.917 | val 2.962 | lr 5.46e-04 | 102,421 tok/s | 54.64 GB | 4.268B tok iter 32580 | train 2.961 | val 2.906 | lr 5.46e-04 | 102,422 tok/s | 54.64 GB | 4.270B tok iter 32600 | train 2.977 | val 2.949 | lr 5.46e-04 | 102,427 tok/s | 54.64 GB | 4.273B tok iter 32620 | train 3.005 | val 2.940 | lr 5.45e-04 | 102,432 tok/s | 54.64 GB | 4.276B tok iter 32640 | train 2.864 | val 2.956 | lr 5.45e-04 | 102,437 tok/s | 54.64 GB | 4.278B tok iter 32660 | train 2.768 | val 2.939 | lr 5.45e-04 | 102,438 tok/s | 54.64 GB | 4.281B tok iter 32680 | train 2.929 | val 2.905 | lr 5.45e-04 | 102,442 tok/s | 54.64 GB | 4.284B tok iter 32700 | train 2.789 | val 2.941 | lr 5.45e-04 | 102,447 tok/s | 54.64 GB | 4.286B tok iter 32720 | train 2.827 | val 2.961 | lr 5.45e-04 | 102,452 tok/s | 54.64 GB | 4.289B tok iter 32740 | train 2.916 | val 2.916 | lr 5.45e-04 | 102,453 tok/s | 54.64 GB | 4.291B tok iter 32760 | train 2.780 | val 2.964 | lr 5.45e-04 | 102,458 tok/s | 54.64 GB | 4.294B tok iter 32780 | train 2.935 | val 2.887 | lr 5.45e-04 | 102,463 tok/s | 54.64 GB | 4.297B tok iter 32800 | train 3.001 | val 2.892 | lr 5.45e-04 | 102,468 tok/s | 54.64 GB | 4.299B tok iter 32820 | train 2.911 | val 2.901 | lr 5.45e-04 | 102,469 tok/s | 54.64 GB | 4.302B tok iter 32840 | train 2.928 | val 2.977 | lr 5.45e-04 | 102,474 tok/s | 54.64 GB | 4.305B tok iter 32860 | train 2.968 | val 2.981 | lr 5.45e-04 | 102,479 tok/s | 54.64 GB | 4.307B tok iter 32880 | train 3.000 | val 2.866 | lr 5.45e-04 | 102,484 tok/s | 54.64 GB | 4.310B tok iter 32900 | train 2.981 | val 2.906 | lr 5.45e-04 | 102,484 tok/s | 54.64 GB | 4.312B tok iter 32920 | train 2.874 | val 2.869 | lr 5.44e-04 | 102,489 tok/s | 54.64 GB | 4.315B tok iter 32940 | train 3.029 | val 2.928 | lr 5.44e-04 | 102,494 tok/s | 54.64 GB | 4.318B tok iter 32960 | train 2.930 | val 2.922 | lr 5.44e-04 | 102,499 tok/s | 54.64 GB | 4.320B tok iter 32980 | train 2.810 | val 2.899 | lr 5.44e-04 | 102,500 tok/s | 54.64 GB | 4.323B tok checkpoint @ iter 32990 -> checkpoints/500m_latest.pt iter 33000 | train 3.003 | val 2.950 | lr 5.44e-04 | 102,472 tok/s | 54.64 GB | 4.326B tok checkpoint @ iter 33000 -> checkpoints/500m_latest.pt iter 33020 | train 2.915 | val 2.930 | lr 5.44e-04 | 102,412 tok/s | 54.64 GB | 4.328B tok iter 33040 | train 3.188 | val 2.891 | lr 5.44e-04 | 102,417 tok/s | 54.64 GB | 4.331B tok iter 33060 | train 2.936 | val 2.954 | lr 5.44e-04 | 102,418 tok/s | 54.64 GB | 4.333B tok iter 33080 | train 2.795 | val 2.919 | lr 5.44e-04 | 102,423 tok/s | 54.64 GB | 4.336B tok iter 33100 | train 2.888 | val 2.884 | lr 5.44e-04 | 102,428 tok/s | 54.64 GB | 4.339B tok iter 33120 | train 2.818 | val 2.878 | lr 5.44e-04 | 102,433 tok/s | 54.64 GB | 4.341B tok iter 33140 | train 2.948 | val 2.898 | lr 5.44e-04 | 102,433 tok/s | 54.64 GB | 4.344B tok iter 33160 | train 2.906 | val 2.931 | lr 5.44e-04 | 102,438 tok/s | 54.64 GB | 4.346B tok iter 33180 | train 2.852 | val 2.888 | lr 5.44e-04 | 102,443 tok/s | 54.64 GB | 4.349B tok iter 33200 | train 2.939 | val 2.916 | lr 5.44e-04 | 102,448 tok/s | 54.64 GB | 4.352B tok iter 33220 | train 2.958 | val 2.891 | lr 5.43e-04 | 102,449 tok/s | 54.64 GB | 4.354B tok iter 33240 | train 2.716 | val 2.953 | lr 5.43e-04 | 102,453 tok/s | 54.64 GB | 4.357B tok iter 33260 | train 2.896 | val 2.937 | lr 5.43e-04 | 102,458 tok/s | 54.64 GB | 4.360B tok iter 33280 | train 3.021 | val 2.908 | lr 5.43e-04 | 102,463 tok/s | 54.64 GB | 4.362B tok iter 33300 | train 2.749 | val 2.964 | lr 5.43e-04 | 102,464 tok/s | 54.64 GB | 4.365B tok iter 33320 | train 2.944 | val 2.895 | lr 5.43e-04 | 102,469 tok/s | 54.64 GB | 4.367B tok iter 33340 | train 2.921 | val 2.877 | lr 5.43e-04 | 102,474 tok/s | 54.64 GB | 4.370B tok iter 33360 | train 3.015 | val 2.895 | lr 5.43e-04 | 102,478 tok/s | 54.64 GB | 4.373B tok iter 33380 | train 2.869 | val 2.920 | lr 5.43e-04 | 102,479 tok/s | 54.64 GB | 4.375B tok iter 33400 | train 2.772 | val 2.940 | lr 5.43e-04 | 102,484 tok/s | 54.64 GB | 4.378B tok iter 33420 | train 2.985 | val 2.929 | lr 5.43e-04 | 102,489 tok/s | 54.64 GB | 4.381B tok iter 33440 | train 2.694 | val 2.896 | lr 5.43e-04 | 102,493 tok/s | 54.64 GB | 4.383B tok iter 33460 | train 2.915 | val 2.943 | lr 5.43e-04 | 102,494 tok/s | 54.64 GB | 4.386B tok iter 33480 | train 3.087 | val 2.909 | lr 5.43e-04 | 102,499 tok/s | 54.64 GB | 4.388B tok checkpoint @ iter 33490 -> checkpoints/500m_latest.pt iter 33500 | train 2.965 | val 2.957 | lr 5.42e-04 | 102,471 tok/s | 54.64 GB | 4.391B tok checkpoint @ iter 33500 -> checkpoints/500m_latest.pt iter 33520 | train 2.896 | val 2.915 | lr 5.42e-04 | 102,412 tok/s | 54.64 GB | 4.394B tok iter 33540 | train 3.031 | val 2.943 | lr 5.42e-04 | 102,413 tok/s | 54.64 GB | 4.396B tok iter 33560 | train 2.834 | val 2.913 | lr 5.42e-04 | 102,418 tok/s | 54.64 GB | 4.399B tok iter 33580 | train 2.992 | val 2.926 | lr 5.42e-04 | 102,422 tok/s | 54.64 GB | 4.402B tok iter 33600 | train 2.870 | val 2.950 | lr 5.42e-04 | 102,427 tok/s | 54.64 GB | 4.404B tok iter 33620 | train 2.896 | val 2.900 | lr 5.42e-04 | 102,428 tok/s | 54.64 GB | 4.407B tok iter 33640 | train 2.948 | val 2.937 | lr 5.42e-04 | 102,433 tok/s | 54.64 GB | 4.409B tok iter 33660 | train 2.862 | val 2.927 | lr 5.42e-04 | 102,438 tok/s | 54.64 GB | 4.412B tok iter 33680 | train 2.914 | val 2.915 | lr 5.42e-04 | 102,443 tok/s | 54.64 GB | 4.415B tok iter 33700 | train 2.862 | val 2.861 | lr 5.42e-04 | 102,443 tok/s | 54.64 GB | 4.417B tok iter 33720 | train 2.816 | val 2.930 | lr 5.42e-04 | 102,448 tok/s | 54.64 GB | 4.420B tok iter 33740 | train 2.855 | val 2.903 | lr 5.42e-04 | 102,453 tok/s | 54.64 GB | 4.423B tok iter 33760 | train 2.911 | val 2.896 | lr 5.42e-04 | 102,458 tok/s | 54.64 GB | 4.425B tok iter 33780 | train 2.831 | val 2.893 | lr 5.42e-04 | 102,459 tok/s | 54.64 GB | 4.428B tok iter 33800 | train 2.995 | val 2.938 | lr 5.41e-04 | 102,463 tok/s | 54.64 GB | 4.430B tok iter 33820 | train 2.937 | val 2.895 | lr 5.41e-04 | 102,468 tok/s | 54.64 GB | 4.433B tok iter 33840 | train 2.890 | val 2.890 | lr 5.41e-04 | 102,473 tok/s | 54.64 GB | 4.436B tok iter 33860 | train 2.778 | val 2.926 | lr 5.41e-04 | 102,474 tok/s | 54.64 GB | 4.438B tok iter 33880 | train 2.965 | val 2.951 | lr 5.41e-04 | 102,479 tok/s | 54.64 GB | 4.441B tok iter 33900 | train 2.970 | val 2.905 | lr 5.41e-04 | 102,484 tok/s | 54.64 GB | 4.443B tok iter 33920 | train 2.877 | val 2.879 | lr 5.41e-04 | 102,488 tok/s | 54.64 GB | 4.446B tok iter 33940 | train 2.922 | val 2.957 | lr 5.41e-04 | 102,489 tok/s | 54.64 GB | 4.449B tok iter 33960 | train 2.923 | val 2.871 | lr 5.41e-04 | 102,494 tok/s | 54.64 GB | 4.451B tok iter 33980 | train 2.918 | val 2.876 | lr 5.41e-04 | 102,499 tok/s | 54.64 GB | 4.454B tok checkpoint @ iter 33990 -> checkpoints/500m_latest.pt iter 34000 | train 2.932 | val 2.889 | lr 5.41e-04 | 102,471 tok/s | 54.64 GB | 4.457B tok checkpoint @ iter 34000 -> checkpoints/500m_latest.pt iter 34020 | train 2.899 | val 2.927 | lr 5.41e-04 | 102,408 tok/s | 54.64 GB | 4.459B tok iter 34040 | train 2.912 | val 2.919 | lr 5.41e-04 | 102,413 tok/s | 54.64 GB | 4.462B tok iter 34060 | train 2.833 | val 2.941 | lr 5.41e-04 | 102,418 tok/s | 54.64 GB | 4.464B tok iter 34080 | train 2.904 | val 2.877 | lr 5.40e-04 | 102,423 tok/s | 54.64 GB | 4.467B tok iter 34100 | train 2.992 | val 2.910 | lr 5.40e-04 | 102,424 tok/s | 54.64 GB | 4.470B tok iter 34120 | train 3.010 | val 2.926 | lr 5.40e-04 | 102,428 tok/s | 54.64 GB | 4.472B tok iter 34140 | train 2.794 | val 2.907 | lr 5.40e-04 | 102,433 tok/s | 54.64 GB | 4.475B tok iter 34160 | train 3.041 | val 2.927 | lr 5.40e-04 | 102,438 tok/s | 54.64 GB | 4.478B tok iter 34180 | train 2.923 | val 2.870 | lr 5.40e-04 | 102,439 tok/s | 54.64 GB | 4.480B tok iter 34200 | train 2.964 | val 2.876 | lr 5.40e-04 | 102,444 tok/s | 54.64 GB | 4.483B tok iter 34220 | train 2.785 | val 2.866 | lr 5.40e-04 | 102,448 tok/s | 54.64 GB | 4.485B tok iter 34240 | train 2.856 | val 2.939 | lr 5.40e-04 | 102,453 tok/s | 54.64 GB | 4.488B tok iter 34260 | train 2.852 | val 2.909 | lr 5.40e-04 | 102,454 tok/s | 54.64 GB | 4.491B tok iter 34280 | train 3.031 | val 2.871 | lr 5.40e-04 | 102,459 tok/s | 54.64 GB | 4.493B tok iter 34300 | train 2.871 | val 2.902 | lr 5.40e-04 | 102,463 tok/s | 54.64 GB | 4.496B tok iter 34320 | train 2.892 | val 2.899 | lr 5.40e-04 | 102,468 tok/s | 54.64 GB | 4.499B tok iter 34340 | train 2.865 | val 2.931 | lr 5.40e-04 | 102,469 tok/s | 54.64 GB | 4.501B tok iter 34360 | train 2.839 | val 2.862 | lr 5.39e-04 | 102,473 tok/s | 54.64 GB | 4.504B tok iter 34380 | train 2.937 | val 2.923 | lr 5.39e-04 | 102,478 tok/s | 54.64 GB | 4.506B tok iter 34400 | train 3.009 | val 2.922 | lr 5.39e-04 | 102,483 tok/s | 54.64 GB | 4.509B tok iter 34420 | train 3.025 | val 2.910 | lr 5.39e-04 | 102,484 tok/s | 54.64 GB | 4.512B tok iter 34440 | train 2.958 | val 2.924 | lr 5.39e-04 | 102,488 tok/s | 54.64 GB | 4.514B tok iter 34460 | train 2.927 | val 2.892 | lr 5.39e-04 | 102,493 tok/s | 54.64 GB | 4.517B tok iter 34480 | train 2.881 | val 2.922 | lr 5.39e-04 | 102,498 tok/s | 54.64 GB | 4.519B tok checkpoint @ iter 34490 -> checkpoints/500m_latest.pt iter 34500 | train 2.910 | val 2.896 | lr 5.39e-04 | 102,468 tok/s | 54.64 GB | 4.522B tok checkpoint @ iter 34500 -> checkpoints/500m_latest.pt iter 34520 | train 2.968 | val 2.919 | lr 5.39e-04 | 102,410 tok/s | 54.64 GB | 4.525B tok iter 34540 | train 2.815 | val 2.929 | lr 5.39e-04 | 102,415 tok/s | 54.64 GB | 4.527B tok iter 34560 | train 2.910 | val 2.899 | lr 5.39e-04 | 102,420 tok/s | 54.64 GB | 4.530B tok iter 34580 | train 3.023 | val 2.934 | lr 5.39e-04 | 102,421 tok/s | 54.64 GB | 4.533B tok iter 34600 | train 2.908 | val 2.930 | lr 5.39e-04 | 102,426 tok/s | 54.64 GB | 4.535B tok iter 34620 | train 3.037 | val 2.893 | lr 5.39e-04 | 102,430 tok/s | 54.64 GB | 4.538B tok iter 34640 | train 3.059 | val 2.920 | lr 5.38e-04 | 102,435 tok/s | 54.64 GB | 4.540B tok iter 34660 | train 2.933 | val 2.930 | lr 5.38e-04 | 102,436 tok/s | 54.64 GB | 4.543B tok iter 34680 | train 2.884 | val 2.875 | lr 5.38e-04 | 102,440 tok/s | 54.64 GB | 4.546B tok iter 34700 | train 2.923 | val 2.886 | lr 5.38e-04 | 102,445 tok/s | 54.64 GB | 4.548B tok iter 34720 | train 2.740 | val 2.933 | lr 5.38e-04 | 102,450 tok/s | 54.64 GB | 4.551B tok iter 34740 | train 2.932 | val 2.889 | lr 5.38e-04 | 102,450 tok/s | 54.64 GB | 4.554B tok iter 34760 | train 2.710 | val 2.907 | lr 5.38e-04 | 102,455 tok/s | 54.64 GB | 4.556B tok iter 34780 | train 2.842 | val 2.876 | lr 5.38e-04 | 102,460 tok/s | 54.64 GB | 4.559B tok iter 34800 | train 2.844 | val 2.864 | lr 5.38e-04 | 102,464 tok/s | 54.64 GB | 4.561B tok iter 34820 | train 2.800 | val 2.899 | lr 5.38e-04 | 102,465 tok/s | 54.64 GB | 4.564B tok iter 34840 | train 2.841 | val 2.917 | lr 5.38e-04 | 102,469 tok/s | 54.64 GB | 4.567B tok iter 34860 | train 2.924 | val 2.906 | lr 5.38e-04 | 102,474 tok/s | 54.64 GB | 4.569B tok iter 34880 | train 2.981 | val 2.910 | lr 5.38e-04 | 102,479 tok/s | 54.64 GB | 4.572B tok iter 34900 | train 2.956 | val 2.944 | lr 5.38e-04 | 102,479 tok/s | 54.64 GB | 4.575B tok iter 34920 | train 2.827 | val 2.890 | lr 5.37e-04 | 102,484 tok/s | 54.64 GB | 4.577B tok iter 34940 | train 2.983 | val 2.928 | lr 5.37e-04 | 102,489 tok/s | 54.64 GB | 4.580B tok iter 34960 | train 2.920 | val 2.920 | lr 5.37e-04 | 102,493 tok/s | 54.64 GB | 4.582B tok iter 34980 | train 2.975 | val 2.918 | lr 5.37e-04 | 102,494 tok/s | 54.64 GB | 4.585B tok checkpoint @ iter 34990 -> checkpoints/500m_latest.pt iter 35000 | train 2.810 | val 2.910 | lr 5.37e-04 | 102,470 tok/s | 54.64 GB | 4.588B tok checkpoint @ iter 35000 -> checkpoints/500m_latest.pt iter 35020 | train 3.000 | val 2.944 | lr 5.37e-04 | 102,414 tok/s | 54.64 GB | 4.590B tok iter 35040 | train 2.882 | val 2.898 | lr 5.37e-04 | 102,419 tok/s | 54.64 GB | 4.593B tok iter 35060 | train 3.033 | val 2.897 | lr 5.37e-04 | 102,420 tok/s | 54.64 GB | 4.596B tok iter 35080 | train 2.749 | val 2.895 | lr 5.37e-04 | 102,424 tok/s | 54.64 GB | 4.598B tok iter 35100 | train 2.932 | val 2.928 | lr 5.37e-04 | 102,429 tok/s | 54.64 GB | 4.601B tok iter 35120 | train 2.835 | val 2.866 | lr 5.37e-04 | 102,434 tok/s | 54.64 GB | 4.603B tok iter 35140 | train 2.816 | val 2.923 | lr 5.37e-04 | 102,434 tok/s | 54.64 GB | 4.606B tok iter 35160 | train 2.829 | val 2.918 | lr 5.37e-04 | 102,439 tok/s | 54.64 GB | 4.609B tok iter 35180 | train 2.869 | val 2.900 | lr 5.37e-04 | 102,444 tok/s | 54.64 GB | 4.611B tok iter 35200 | train 2.870 | val 2.853 | lr 5.36e-04 | 102,448 tok/s | 54.64 GB | 4.614B tok iter 35220 | train 2.793 | val 2.897 | lr 5.36e-04 | 102,449 tok/s | 54.64 GB | 4.616B tok iter 35240 | train 3.017 | val 2.894 | lr 5.36e-04 | 102,454 tok/s | 54.64 GB | 4.619B tok iter 35260 | train 3.037 | val 2.938 | lr 5.36e-04 | 102,458 tok/s | 54.64 GB | 4.622B tok iter 35280 | train 2.776 | val 2.882 | lr 5.36e-04 | 102,463 tok/s | 54.64 GB | 4.624B tok iter 35300 | train 2.822 | val 2.877 | lr 5.36e-04 | 102,464 tok/s | 54.64 GB | 4.627B tok iter 35320 | train 2.860 | val 2.887 | lr 5.36e-04 | 102,468 tok/s | 54.64 GB | 4.630B tok iter 35340 | train 2.827 | val 2.954 | lr 5.36e-04 | 102,473 tok/s | 54.64 GB | 4.632B tok iter 35360 | train 3.028 | val 2.872 | lr 5.36e-04 | 102,477 tok/s | 54.64 GB | 4.635B tok iter 35380 | train 2.846 | val 2.879 | lr 5.36e-04 | 102,478 tok/s | 54.64 GB | 4.637B tok iter 35400 | train 2.913 | val 2.857 | lr 5.36e-04 | 102,483 tok/s | 54.64 GB | 4.640B tok iter 35420 | train 2.904 | val 2.835 | lr 5.36e-04 | 102,487 tok/s | 54.64 GB | 4.643B tok iter 35440 | train 2.810 | val 2.871 | lr 5.36e-04 | 102,492 tok/s | 54.64 GB | 4.645B tok iter 35460 | train 2.870 | val 2.934 | lr 5.36e-04 | 102,492 tok/s | 54.64 GB | 4.648B tok iter 35480 | train 2.840 | val 2.922 | lr 5.35e-04 | 102,497 tok/s | 54.64 GB | 4.651B tok checkpoint @ iter 35490 -> checkpoints/500m_latest.pt iter 35500 | train 2.923 | val 2.897 | lr 5.35e-04 | 102,471 tok/s | 54.64 GB | 4.653B tok checkpoint @ iter 35500 -> checkpoints/500m_latest.pt iter 35520 | train 2.938 | val 2.907 | lr 5.35e-04 | 102,414 tok/s | 54.64 GB | 4.656B tok iter 35540 | train 2.856 | val 2.950 | lr 5.35e-04 | 102,415 tok/s | 54.64 GB | 4.658B tok iter 35560 | train 2.827 | val 2.898 | lr 5.35e-04 | 102,420 tok/s | 54.64 GB | 4.661B tok iter 35580 | train 2.927 | val 2.953 | lr 5.35e-04 | 102,424 tok/s | 54.64 GB | 4.664B tok iter 35600 | train 2.951 | val 2.887 | lr 5.35e-04 | 102,429 tok/s | 54.64 GB | 4.666B tok iter 35620 | train 2.968 | val 2.885 | lr 5.35e-04 | 102,430 tok/s | 54.64 GB | 4.669B tok iter 35640 | train 2.791 | val 2.888 | lr 5.35e-04 | 102,434 tok/s | 54.64 GB | 4.672B tok iter 35660 | train 2.757 | val 2.895 | lr 5.35e-04 | 102,439 tok/s | 54.64 GB | 4.674B tok iter 35680 | train 3.017 | val 2.885 | lr 5.35e-04 | 102,443 tok/s | 54.64 GB | 4.677B tok iter 35700 | train 2.932 | val 2.885 | lr 5.35e-04 | 102,444 tok/s | 54.64 GB | 4.679B tok iter 35720 | train 2.882 | val 2.895 | lr 5.35e-04 | 102,448 tok/s | 54.64 GB | 4.682B tok iter 35740 | train 2.795 | val 2.889 | lr 5.34e-04 | 102,453 tok/s | 54.64 GB | 4.685B tok iter 35760 | train 2.808 | val 2.849 | lr 5.34e-04 | 102,458 tok/s | 54.64 GB | 4.687B tok iter 35780 | train 2.910 | val 2.929 | lr 5.34e-04 | 102,458 tok/s | 54.64 GB | 4.690B tok iter 35800 | train 2.980 | val 2.897 | lr 5.34e-04 | 102,463 tok/s | 54.64 GB | 4.693B tok iter 35820 | train 2.821 | val 2.886 | lr 5.34e-04 | 102,467 tok/s | 54.64 GB | 4.695B tok iter 35840 | train 2.828 | val 2.942 | lr 5.34e-04 | 102,472 tok/s | 54.64 GB | 4.698B tok iter 35860 | train 2.960 | val 2.885 | lr 5.34e-04 | 102,473 tok/s | 54.64 GB | 4.700B tok iter 35880 | train 2.862 | val 2.905 | lr 5.34e-04 | 102,477 tok/s | 54.64 GB | 4.703B tok iter 35900 | train 3.034 | val 2.917 | lr 5.34e-04 | 102,482 tok/s | 54.64 GB | 4.706B tok iter 35920 | train 2.782 | val 2.928 | lr 5.34e-04 | 102,486 tok/s | 54.64 GB | 4.708B tok iter 35940 | train 2.849 | val 2.929 | lr 5.34e-04 | 102,487 tok/s | 54.64 GB | 4.711B tok iter 35960 | train 2.879 | val 2.922 | lr 5.34e-04 | 102,492 tok/s | 54.64 GB | 4.713B tok iter 35980 | train 3.006 | val 2.904 | lr 5.34e-04 | 102,496 tok/s | 54.64 GB | 4.716B tok checkpoint @ iter 35990 -> checkpoints/500m_latest.pt iter 36000 | train 2.908 | val 2.919 | lr 5.34e-04 | 102,471 tok/s | 54.64 GB | 4.719B tok checkpoint @ iter 36000 -> checkpoints/500m_latest.pt iter 36020 | train 2.988 | val 2.909 | lr 5.33e-04 | 102,411 tok/s | 54.64 GB | 4.721B tok iter 36040 | train 3.018 | val 2.914 | lr 5.33e-04 | 102,416 tok/s | 54.64 GB | 4.724B tok iter 36060 | train 2.972 | val 2.895 | lr 5.33e-04 | 102,420 tok/s | 54.64 GB | 4.727B tok iter 36080 | train 2.816 | val 2.888 | lr 5.33e-04 | 102,425 tok/s | 54.64 GB | 4.729B tok iter 36100 | train 2.931 | val 2.868 | lr 5.33e-04 | 102,426 tok/s | 54.64 GB | 4.732B tok iter 36120 | train 2.925 | val 2.907 | lr 5.33e-04 | 102,430 tok/s | 54.64 GB | 4.734B tok iter 36140 | train 2.905 | val 2.874 | lr 5.33e-04 | 102,435 tok/s | 54.64 GB | 4.737B tok iter 36160 | train 2.850 | val 2.849 | lr 5.33e-04 | 102,439 tok/s | 54.64 GB | 4.740B tok iter 36180 | train 2.844 | val 2.910 | lr 5.33e-04 | 102,440 tok/s | 54.64 GB | 4.742B tok iter 36200 | train 2.811 | val 2.877 | lr 5.33e-04 | 102,445 tok/s | 54.64 GB | 4.745B tok iter 36220 | train 2.908 | val 2.928 | lr 5.33e-04 | 102,449 tok/s | 54.64 GB | 4.748B tok iter 36240 | train 2.904 | val 2.916 | lr 5.33e-04 | 102,454 tok/s | 54.64 GB | 4.750B tok iter 36260 | train 2.918 | val 2.867 | lr 5.33e-04 | 102,454 tok/s | 54.64 GB | 4.753B tok iter 36280 | train 2.932 | val 2.849 | lr 5.32e-04 | 102,459 tok/s | 54.64 GB | 4.755B tok iter 36300 | train 2.890 | val 2.932 | lr 5.32e-04 | 102,463 tok/s | 54.64 GB | 4.758B tok iter 36320 | train 2.914 | val 2.873 | lr 5.32e-04 | 102,468 tok/s | 54.64 GB | 4.761B tok iter 36340 | train 2.948 | val 2.943 | lr 5.32e-04 | 102,469 tok/s | 54.64 GB | 4.763B tok iter 36360 | train 2.944 | val 2.915 | lr 5.32e-04 | 102,473 tok/s | 54.64 GB | 4.766B tok iter 36380 | train 2.985 | val 2.896 | lr 5.32e-04 | 102,477 tok/s | 54.64 GB | 4.769B tok iter 36400 | train 2.877 | val 2.839 | lr 5.32e-04 | 102,482 tok/s | 54.64 GB | 4.771B tok iter 36420 | train 2.906 | val 2.918 | lr 5.32e-04 | 102,483 tok/s | 54.64 GB | 4.774B tok iter 36440 | train 2.952 | val 2.937 | lr 5.32e-04 | 102,487 tok/s | 54.64 GB | 4.776B tok iter 36460 | train 2.883 | val 2.860 | lr 5.32e-04 | 102,491 tok/s | 54.64 GB | 4.779B tok iter 36480 | train 3.101 | val 2.901 | lr 5.32e-04 | 102,496 tok/s | 54.64 GB | 4.782B tok checkpoint @ iter 36490 -> checkpoints/500m_latest.pt iter 36500 | train 2.778 | val 2.953 | lr 5.32e-04 | 102,466 tok/s | 54.64 GB | 4.784B tok checkpoint @ iter 36500 -> checkpoints/500m_latest.pt iter 36520 | train 2.903 | val 2.900 | lr 5.32e-04 | 102,412 tok/s | 54.64 GB | 4.787B tok iter 36540 | train 3.087 | val 2.945 | lr 5.32e-04 | 102,416 tok/s | 54.64 GB | 4.790B tok iter 36560 | train 2.893 | val 2.915 | lr 5.31e-04 | 102,421 tok/s | 54.64 GB | 4.792B tok iter 36580 | train 2.930 | val 2.883 | lr 5.31e-04 | 102,422 tok/s | 54.64 GB | 4.795B tok iter 36600 | train 2.910 | val 2.929 | lr 5.31e-04 | 102,426 tok/s | 54.64 GB | 4.797B tok iter 36620 | train 2.800 | val 2.940 | lr 5.31e-04 | 102,431 tok/s | 54.64 GB | 4.800B tok iter 36640 | train 2.997 | val 2.934 | lr 5.31e-04 | 102,435 tok/s | 54.64 GB | 4.803B tok iter 36660 | train 2.932 | val 2.895 | lr 5.31e-04 | 102,436 tok/s | 54.64 GB | 4.805B tok iter 36680 | train 2.713 | val 2.916 | lr 5.31e-04 | 102,440 tok/s | 54.64 GB | 4.808B tok iter 36700 | train 2.835 | val 2.911 | lr 5.31e-04 | 102,445 tok/s | 54.64 GB | 4.810B tok iter 36720 | train 3.012 | val 2.891 | lr 5.31e-04 | 102,449 tok/s | 54.64 GB | 4.813B tok iter 36740 | train 2.915 | val 2.963 | lr 5.31e-04 | 102,450 tok/s | 54.64 GB | 4.816B tok iter 36760 | train 2.821 | val 2.936 | lr 5.31e-04 | 102,454 tok/s | 54.64 GB | 4.818B tok iter 36780 | train 2.921 | val 2.904 | lr 5.31e-04 | 102,459 tok/s | 54.64 GB | 4.821B tok iter 36800 | train 2.810 | val 2.896 | lr 5.31e-04 | 102,463 tok/s | 54.64 GB | 4.824B tok iter 36820 | train 2.940 | val 2.870 | lr 5.30e-04 | 102,464 tok/s | 54.64 GB | 4.826B tok iter 36840 | train 2.899 | val 2.870 | lr 5.30e-04 | 102,468 tok/s | 54.64 GB | 4.829B tok iter 36860 | train 2.865 | val 2.930 | lr 5.30e-04 | 102,473 tok/s | 54.64 GB | 4.831B tok iter 36880 | train 2.830 | val 2.913 | lr 5.30e-04 | 102,477 tok/s | 54.64 GB | 4.834B tok iter 36900 | train 2.896 | val 2.937 | lr 5.30e-04 | 102,478 tok/s | 54.64 GB | 4.837B tok iter 36920 | train 2.831 | val 2.889 | lr 5.30e-04 | 102,482 tok/s | 54.64 GB | 4.839B tok iter 36940 | train 2.937 | val 2.948 | lr 5.30e-04 | 102,486 tok/s | 54.64 GB | 4.842B tok iter 36960 | train 2.987 | val 2.907 | lr 5.30e-04 | 102,491 tok/s | 54.64 GB | 4.845B tok iter 36980 | train 2.781 | val 2.885 | lr 5.30e-04 | 102,491 tok/s | 54.64 GB | 4.847B tok checkpoint @ iter 36990 -> checkpoints/500m_latest.pt iter 37000 | train 2.978 | val 2.891 | lr 5.30e-04 | 102,467 tok/s | 54.64 GB | 4.850B tok checkpoint @ iter 37000 -> checkpoints/500m_latest.pt iter 37020 | train 2.940 | val 2.916 | lr 5.30e-04 | 102,414 tok/s | 54.64 GB | 4.852B tok iter 37040 | train 3.058 | val 2.899 | lr 5.30e-04 | 102,418 tok/s | 54.64 GB | 4.855B tok iter 37060 | train 2.910 | val 2.924 | lr 5.30e-04 | 102,419 tok/s | 54.64 GB | 4.858B tok iter 37080 | train 2.821 | val 2.870 | lr 5.29e-04 | 102,424 tok/s | 54.64 GB | 4.860B tok iter 37100 | train 2.907 | val 2.862 | lr 5.29e-04 | 102,428 tok/s | 54.64 GB | 4.863B tok iter 37120 | train 2.951 | val 2.902 | lr 5.29e-04 | 102,432 tok/s | 54.64 GB | 4.866B tok iter 37140 | train 2.912 | val 2.940 | lr 5.29e-04 | 102,433 tok/s | 54.64 GB | 4.868B tok iter 37160 | train 2.898 | val 2.864 | lr 5.29e-04 | 102,437 tok/s | 54.64 GB | 4.871B tok iter 37180 | train 2.844 | val 2.924 | lr 5.29e-04 | 102,442 tok/s | 54.64 GB | 4.873B tok iter 37200 | train 2.964 | val 2.912 | lr 5.29e-04 | 102,446 tok/s | 54.64 GB | 4.876B tok iter 37220 | train 2.996 | val 2.895 | lr 5.29e-04 | 102,447 tok/s | 54.64 GB | 4.879B tok iter 37240 | train 2.820 | val 2.919 | lr 5.29e-04 | 102,451 tok/s | 54.64 GB | 4.881B tok iter 37260 | train 2.963 | val 2.940 | lr 5.29e-04 | 102,456 tok/s | 54.64 GB | 4.884B tok iter 37280 | train 3.017 | val 2.862 | lr 5.29e-04 | 102,460 tok/s | 54.64 GB | 4.886B tok iter 37300 | train 2.959 | val 2.894 | lr 5.29e-04 | 102,461 tok/s | 54.64 GB | 4.889B tok iter 37320 | train 2.851 | val 2.893 | lr 5.29e-04 | 102,465 tok/s | 54.64 GB | 4.892B tok iter 37340 | train 2.684 | val 2.841 | lr 5.29e-04 | 102,469 tok/s | 54.64 GB | 4.894B tok iter 37360 | train 2.878 | val 2.919 | lr 5.28e-04 | 102,474 tok/s | 54.64 GB | 4.897B tok iter 37380 | train 2.945 | val 2.870 | lr 5.28e-04 | 102,474 tok/s | 54.64 GB | 4.900B tok iter 37400 | train 2.940 | val 2.916 | lr 5.28e-04 | 102,478 tok/s | 54.64 GB | 4.902B tok iter 37420 | train 2.822 | val 2.925 | lr 5.28e-04 | 102,483 tok/s | 54.64 GB | 4.905B tok iter 37440 | train 2.871 | val 2.888 | lr 5.28e-04 | 102,487 tok/s | 54.64 GB | 4.907B tok iter 37460 | train 2.939 | val 2.897 | lr 5.28e-04 | 102,488 tok/s | 54.64 GB | 4.910B tok iter 37480 | train 2.996 | val 2.887 | lr 5.28e-04 | 102,492 tok/s | 54.64 GB | 4.913B tok checkpoint @ iter 37490 -> checkpoints/500m_latest.pt iter 37500 | train 2.941 | val 2.911 | lr 5.28e-04 | 102,467 tok/s | 54.64 GB | 4.915B tok checkpoint @ iter 37500 -> checkpoints/500m_latest.pt iter 37520 | train 2.870 | val 2.946 | lr 5.28e-04 | 102,412 tok/s | 54.64 GB | 4.918B tok iter 37540 | train 2.758 | val 2.871 | lr 5.28e-04 | 102,413 tok/s | 54.64 GB | 4.921B tok iter 37560 | train 3.029 | val 2.895 | lr 5.28e-04 | 102,417 tok/s | 54.64 GB | 4.923B tok iter 37580 | train 3.004 | val 2.902 | lr 5.28e-04 | 102,422 tok/s | 54.64 GB | 4.926B tok iter 37600 | train 3.044 | val 2.907 | lr 5.28e-04 | 102,426 tok/s | 54.64 GB | 4.928B tok iter 37620 | train 2.850 | val 2.907 | lr 5.27e-04 | 102,427 tok/s | 54.64 GB | 4.931B tok iter 37640 | train 2.955 | val 2.918 | lr 5.27e-04 | 102,430 tok/s | 54.64 GB | 4.934B tok iter 37660 | train 2.917 | val 2.892 | lr 5.27e-04 | 102,435 tok/s | 54.64 GB | 4.936B tok iter 37680 | train 2.818 | val 2.867 | lr 5.27e-04 | 102,439 tok/s | 54.64 GB | 4.939B tok iter 37700 | train 2.965 | val 2.883 | lr 5.27e-04 | 102,440 tok/s | 54.64 GB | 4.942B tok iter 37720 | train 2.945 | val 2.872 | lr 5.27e-04 | 102,444 tok/s | 54.64 GB | 4.944B tok iter 37740 | train 3.101 | val 2.902 | lr 5.27e-04 | 102,449 tok/s | 54.64 GB | 4.947B tok iter 37760 | train 2.985 | val 2.847 | lr 5.27e-04 | 102,453 tok/s | 54.64 GB | 4.949B tok iter 37780 | train 2.949 | val 2.950 | lr 5.27e-04 | 102,454 tok/s | 54.64 GB | 4.952B tok iter 37800 | train 2.929 | val 2.908 | lr 5.27e-04 | 102,458 tok/s | 54.64 GB | 4.955B tok iter 37820 | train 2.943 | val 2.889 | lr 5.27e-04 | 102,462 tok/s | 54.64 GB | 4.957B tok iter 37840 | train 2.809 | val 2.860 | lr 5.27e-04 | 102,466 tok/s | 54.64 GB | 4.960B tok iter 37860 | train 2.907 | val 2.907 | lr 5.27e-04 | 102,467 tok/s | 54.64 GB | 4.963B tok iter 37880 | train 2.803 | val 2.907 | lr 5.26e-04 | 102,471 tok/s | 54.64 GB | 4.965B tok iter 37900 | train 2.868 | val 2.887 | lr 5.26e-04 | 102,475 tok/s | 54.64 GB | 4.968B tok iter 37920 | train 2.889 | val 2.869 | lr 5.26e-04 | 102,480 tok/s | 54.64 GB | 4.970B tok iter 37940 | train 2.966 | val 2.913 | lr 5.26e-04 | 102,481 tok/s | 54.64 GB | 4.973B tok iter 37960 | train 2.875 | val 2.878 | lr 5.26e-04 | 102,485 tok/s | 54.64 GB | 4.976B tok iter 37980 | train 3.018 | val 2.870 | lr 5.26e-04 | 102,489 tok/s | 54.64 GB | 4.978B tok checkpoint @ iter 37989 -> checkpoints/500m_latest.pt iter 38000 | train 2.840 | val 2.900 | lr 5.26e-04 | 102,465 tok/s | 54.64 GB | 4.981B tok checkpoint @ iter 38000 -> checkpoints/500m_latest.pt iter 38020 | train 2.796 | val 2.926 | lr 5.26e-04 | 102,408 tok/s | 54.64 GB | 4.983B tok iter 38040 | train 2.876 | val 2.853 | lr 5.26e-04 | 102,412 tok/s | 54.64 GB | 4.986B tok iter 38060 | train 3.035 | val 2.901 | lr 5.26e-04 | 102,417 tok/s | 54.64 GB | 4.989B tok iter 38080 | train 2.919 | val 2.899 | lr 5.26e-04 | 102,421 tok/s | 54.64 GB | 4.991B tok iter 38100 | train 2.896 | val 2.869 | lr 5.26e-04 | 102,422 tok/s | 54.64 GB | 4.994B tok iter 38120 | train 2.623 | val 2.915 | lr 5.26e-04 | 102,426 tok/s | 54.64 GB | 4.997B tok iter 38140 | train 3.043 | val 2.898 | lr 5.25e-04 | 102,430 tok/s | 54.64 GB | 4.999B tok iter 38160 | train 2.797 | val 2.863 | lr 5.25e-04 | 102,434 tok/s | 54.64 GB | 5.002B tok iter 38180 | train 2.964 | val 2.849 | lr 5.25e-04 | 102,435 tok/s | 54.64 GB | 5.004B tok iter 38200 | train 2.808 | val 2.942 | lr 5.25e-04 | 102,439 tok/s | 54.64 GB | 5.007B tok iter 38220 | train 2.974 | val 2.909 | lr 5.25e-04 | 102,443 tok/s | 54.64 GB | 5.010B tok iter 38240 | train 2.810 | val 2.897 | lr 5.25e-04 | 102,447 tok/s | 54.64 GB | 5.012B tok iter 38260 | train 2.920 | val 2.924 | lr 5.25e-04 | 102,448 tok/s | 54.64 GB | 5.015B tok iter 38280 | train 2.945 | val 2.868 | lr 5.25e-04 | 102,452 tok/s | 54.64 GB | 5.018B tok iter 38300 | train 2.728 | val 2.875 | lr 5.25e-04 | 102,457 tok/s | 54.64 GB | 5.020B tok iter 38320 | train 2.969 | val 2.929 | lr 5.25e-04 | 102,461 tok/s | 54.64 GB | 5.023B tok iter 38340 | train 2.878 | val 2.910 | lr 5.25e-04 | 102,462 tok/s | 54.64 GB | 5.025B tok iter 38360 | train 3.016 | val 2.874 | lr 5.25e-04 | 102,466 tok/s | 54.64 GB | 5.028B tok iter 38380 | train 2.981 | val 2.914 | lr 5.25e-04 | 102,470 tok/s | 54.64 GB | 5.031B tok iter 38400 | train 2.920 | val 2.918 | lr 5.24e-04 | 102,474 tok/s | 54.64 GB | 5.033B tok iter 38420 | train 2.941 | val 2.931 | lr 5.24e-04 | 102,475 tok/s | 54.64 GB | 5.036B tok iter 38440 | train 2.809 | val 2.873 | lr 5.24e-04 | 102,479 tok/s | 54.64 GB | 5.039B tok iter 38460 | train 2.915 | val 2.873 | lr 5.24e-04 | 102,483 tok/s | 54.64 GB | 5.041B tok iter 38480 | train 2.779 | val 2.904 | lr 5.24e-04 | 102,488 tok/s | 54.64 GB | 5.044B tok checkpoint @ iter 38490 -> checkpoints/500m_latest.pt iter 38500 | train 2.954 | val 2.853 | lr 5.24e-04 | 102,460 tok/s | 54.64 GB | 5.046B tok checkpoint @ iter 38500 -> checkpoints/500m_latest.pt iter 38520 | train 2.856 | val 2.920 | lr 5.24e-04 | 102,409 tok/s | 54.64 GB | 5.049B tok iter 38540 | train 2.903 | val 2.862 | lr 5.24e-04 | 102,413 tok/s | 54.64 GB | 5.052B tok iter 38560 | train 2.908 | val 2.859 | lr 5.24e-04 | 102,418 tok/s | 54.64 GB | 5.054B tok iter 38580 | train 2.869 | val 2.883 | lr 5.24e-04 | 102,418 tok/s | 54.64 GB | 5.057B tok iter 38600 | train 2.961 | val 2.923 | lr 5.24e-04 | 102,423 tok/s | 54.64 GB | 5.060B tok iter 38620 | train 3.000 | val 2.883 | lr 5.24e-04 | 102,427 tok/s | 54.64 GB | 5.062B tok iter 38640 | train 2.790 | val 2.908 | lr 5.23e-04 | 102,431 tok/s | 54.64 GB | 5.065B tok iter 38660 | train 2.869 | val 2.928 | lr 5.23e-04 | 102,432 tok/s | 54.64 GB | 5.067B tok iter 38680 | train 2.852 | val 2.906 | lr 5.23e-04 | 102,436 tok/s | 54.64 GB | 5.070B tok iter 38700 | train 2.858 | val 2.870 | lr 5.23e-04 | 102,440 tok/s | 54.64 GB | 5.073B tok iter 38720 | train 2.927 | val 2.881 | lr 5.23e-04 | 102,444 tok/s | 54.64 GB | 5.075B tok iter 38740 | train 2.870 | val 2.908 | lr 5.23e-04 | 102,445 tok/s | 54.64 GB | 5.078B tok iter 38760 | train 2.877 | val 2.909 | lr 5.23e-04 | 102,449 tok/s | 54.64 GB | 5.080B tok iter 38780 | train 2.883 | val 2.886 | lr 5.23e-04 | 102,453 tok/s | 54.64 GB | 5.083B tok iter 38800 | train 2.923 | val 2.876 | lr 5.23e-04 | 102,457 tok/s | 54.64 GB | 5.086B tok iter 38820 | train 2.909 | val 2.930 | lr 5.23e-04 | 102,458 tok/s | 54.64 GB | 5.088B tok iter 38840 | train 2.772 | val 2.934 | lr 5.23e-04 | 102,462 tok/s | 54.64 GB | 5.091B tok iter 38860 | train 2.793 | val 2.905 | lr 5.23e-04 | 102,467 tok/s | 54.64 GB | 5.094B tok iter 38880 | train 2.884 | val 2.892 | lr 5.23e-04 | 102,471 tok/s | 54.64 GB | 5.096B tok iter 38900 | train 2.848 | val 2.901 | lr 5.22e-04 | 102,471 tok/s | 54.64 GB | 5.099B tok iter 38920 | train 2.948 | val 2.921 | lr 5.22e-04 | 102,476 tok/s | 54.64 GB | 5.101B tok iter 38940 | train 2.808 | val 2.874 | lr 5.22e-04 | 102,480 tok/s | 54.64 GB | 5.104B tok iter 38960 | train 2.910 | val 2.904 | lr 5.22e-04 | 102,484 tok/s | 54.64 GB | 5.107B tok iter 38980 | train 3.002 | val 2.910 | lr 5.22e-04 | 102,485 tok/s | 54.64 GB | 5.109B tok checkpoint @ iter 38990 -> checkpoints/500m_latest.pt iter 39000 | train 2.919 | val 2.853 | lr 5.22e-04 | 102,461 tok/s | 54.64 GB | 5.112B tok checkpoint @ iter 39000 -> checkpoints/500m_latest.pt iter 39020 | train 3.031 | val 2.861 | lr 5.22e-04 | 102,410 tok/s | 54.64 GB | 5.115B tok iter 39040 | train 2.980 | val 2.884 | lr 5.22e-04 | 102,414 tok/s | 54.64 GB | 5.117B tok iter 39060 | train 2.899 | val 2.909 | lr 5.22e-04 | 102,415 tok/s | 54.64 GB | 5.120B tok iter 39080 | train 2.866 | val 2.932 | lr 5.22e-04 | 102,419 tok/s | 54.64 GB | 5.122B tok iter 39100 | train 3.039 | val 2.923 | lr 5.22e-04 | 102,423 tok/s | 54.64 GB | 5.125B tok iter 39120 | train 3.118 | val 2.930 | lr 5.22e-04 | 102,427 tok/s | 54.64 GB | 5.128B tok iter 39140 | train 2.881 | val 2.847 | lr 5.22e-04 | 102,428 tok/s | 54.64 GB | 5.130B tok iter 39160 | train 2.914 | val 2.907 | lr 5.21e-04 | 102,432 tok/s | 54.64 GB | 5.133B tok iter 39180 | train 2.913 | val 2.873 | lr 5.21e-04 | 102,436 tok/s | 54.64 GB | 5.136B tok iter 39200 | train 2.885 | val 2.848 | lr 5.21e-04 | 102,440 tok/s | 54.64 GB | 5.138B tok iter 39220 | train 3.046 | val 2.810 | lr 5.21e-04 | 102,441 tok/s | 54.64 GB | 5.141B tok iter 39240 | train 2.957 | val 2.916 | lr 5.21e-04 | 102,445 tok/s | 54.64 GB | 5.143B tok iter 39260 | train 2.771 | val 2.928 | lr 5.21e-04 | 102,449 tok/s | 54.64 GB | 5.146B tok iter 39280 | train 2.921 | val 2.892 | lr 5.21e-04 | 102,453 tok/s | 54.64 GB | 5.149B tok iter 39300 | train 2.885 | val 2.859 | lr 5.21e-04 | 102,454 tok/s | 54.64 GB | 5.151B tok iter 39320 | train 2.960 | val 2.873 | lr 5.21e-04 | 102,458 tok/s | 54.64 GB | 5.154B tok iter 39340 | train 2.974 | val 2.908 | lr 5.21e-04 | 102,462 tok/s | 54.64 GB | 5.157B tok iter 39360 | train 2.774 | val 2.814 | lr 5.21e-04 | 102,466 tok/s | 54.64 GB | 5.159B tok iter 39380 | train 2.939 | val 2.856 | lr 5.21e-04 | 102,467 tok/s | 54.64 GB | 5.162B tok iter 39400 | train 2.847 | val 2.866 | lr 5.20e-04 | 102,471 tok/s | 54.64 GB | 5.164B tok iter 39420 | train 2.776 | val 2.879 | lr 5.20e-04 | 102,475 tok/s | 54.64 GB | 5.167B tok iter 39440 | train 2.887 | val 2.890 | lr 5.20e-04 | 102,479 tok/s | 54.64 GB | 5.170B tok iter 39460 | train 2.884 | val 2.895 | lr 5.20e-04 | 102,480 tok/s | 54.64 GB | 5.172B tok iter 39480 | train 2.826 | val 2.888 | lr 5.20e-04 | 102,484 tok/s | 54.64 GB | 5.175B tok checkpoint @ iter 39490 -> checkpoints/500m_latest.pt iter 39500 | train 2.882 | val 2.926 | lr 5.20e-04 | 102,460 tok/s | 54.64 GB | 5.177B tok checkpoint @ iter 39500 -> checkpoints/500m_latest.pt iter 39520 | train 3.024 | val 2.898 | lr 5.20e-04 | 102,410 tok/s | 54.64 GB | 5.180B tok iter 39540 | train 3.012 | val 2.909 | lr 5.20e-04 | 102,411 tok/s | 54.64 GB | 5.183B tok iter 39560 | train 2.896 | val 2.887 | lr 5.20e-04 | 102,415 tok/s | 54.64 GB | 5.185B tok iter 39580 | train 2.793 | val 2.888 | lr 5.20e-04 | 102,419 tok/s | 54.64 GB | 5.188B tok iter 39600 | train 2.924 | val 2.887 | lr 5.20e-04 | 102,423 tok/s | 54.64 GB | 5.191B tok iter 39620 | train 2.786 | val 2.850 | lr 5.20e-04 | 102,424 tok/s | 54.64 GB | 5.193B tok iter 39640 | train 2.964 | val 2.867 | lr 5.20e-04 | 102,428 tok/s | 54.64 GB | 5.196B tok iter 39660 | train 2.953 | val 2.922 | lr 5.19e-04 | 102,432 tok/s | 54.64 GB | 5.198B tok iter 39680 | train 2.957 | val 2.905 | lr 5.19e-04 | 102,436 tok/s | 54.64 GB | 5.201B tok iter 39700 | train 2.893 | val 2.841 | lr 5.19e-04 | 102,437 tok/s | 54.64 GB | 5.204B tok iter 39720 | train 2.844 | val 2.849 | lr 5.19e-04 | 102,441 tok/s | 54.64 GB | 5.206B tok iter 39740 | train 2.756 | val 2.918 | lr 5.19e-04 | 102,445 tok/s | 54.64 GB | 5.209B tok iter 39760 | train 2.837 | val 2.888 | lr 5.19e-04 | 102,449 tok/s | 54.64 GB | 5.212B tok iter 39780 | train 2.774 | val 2.867 | lr 5.19e-04 | 102,450 tok/s | 54.64 GB | 5.214B tok iter 39800 | train 2.789 | val 2.885 | lr 5.19e-04 | 102,454 tok/s | 54.64 GB | 5.217B tok iter 39820 | train 2.971 | val 2.842 | lr 5.19e-04 | 102,458 tok/s | 54.64 GB | 5.219B tok iter 39840 | train 2.891 | val 2.870 | lr 5.19e-04 | 102,462 tok/s | 54.64 GB | 5.222B tok iter 39860 | train 2.905 | val 2.898 | lr 5.19e-04 | 102,463 tok/s | 54.64 GB | 5.225B tok iter 39880 | train 2.803 | val 2.901 | lr 5.19e-04 | 102,467 tok/s | 54.64 GB | 5.227B tok iter 39900 | train 2.994 | val 2.873 | lr 5.18e-04 | 102,470 tok/s | 54.64 GB | 5.230B tok iter 39920 | train 2.920 | val 2.920 | lr 5.18e-04 | 102,474 tok/s | 54.64 GB | 5.233B tok iter 39940 | train 3.031 | val 2.879 | lr 5.18e-04 | 102,475 tok/s | 54.64 GB | 5.235B tok iter 39960 | train 2.807 | val 2.896 | lr 5.18e-04 | 102,479 tok/s | 54.64 GB | 5.238B tok iter 39980 | train 2.821 | val 2.876 | lr 5.18e-04 | 102,483 tok/s | 54.64 GB | 5.240B tok checkpoint @ iter 39990 -> checkpoints/500m_latest.pt iter 40000 | train 2.841 | val 2.931 | lr 5.18e-04 | 102,460 tok/s | 54.64 GB | 5.243B tok checkpoint @ iter 40000 -> checkpoints/500m_latest.pt iter 40020 | train 2.869 | val 2.895 | lr 5.18e-04 | 102,406 tok/s | 54.64 GB | 5.246B tok iter 40040 | train 2.738 | val 2.888 | lr 5.18e-04 | 102,410 tok/s | 54.64 GB | 5.248B tok iter 40060 | train 2.957 | val 2.897 | lr 5.18e-04 | 102,414 tok/s | 54.64 GB | 5.251B tok iter 40080 | train 2.884 | val 2.890 | lr 5.18e-04 | 102,418 tok/s | 54.64 GB | 5.253B tok iter 40100 | train 2.868 | val 2.868 | lr 5.18e-04 | 102,419 tok/s | 54.64 GB | 5.256B tok iter 40120 | train 2.868 | val 2.893 | lr 5.18e-04 | 102,423 tok/s | 54.64 GB | 5.259B tok iter 40140 | train 2.752 | val 2.912 | lr 5.18e-04 | 102,427 tok/s | 54.64 GB | 5.261B tok iter 40160 | train 2.902 | val 2.902 | lr 5.17e-04 | 102,431 tok/s | 54.64 GB | 5.264B tok iter 40180 | train 2.955 | val 2.886 | lr 5.17e-04 | 102,431 tok/s | 54.64 GB | 5.267B tok iter 40200 | train 2.901 | val 2.910 | lr 5.17e-04 | 102,435 tok/s | 54.64 GB | 5.269B tok iter 40220 | train 2.871 | val 2.869 | lr 5.17e-04 | 102,439 tok/s | 54.64 GB | 5.272B tok iter 40240 | train 2.894 | val 2.840 | lr 5.17e-04 | 102,444 tok/s | 54.64 GB | 5.274B tok iter 40260 | train 2.983 | val 2.894 | lr 5.17e-04 | 102,444 tok/s | 54.64 GB | 5.277B tok iter 40280 | train 2.802 | val 2.888 | lr 5.17e-04 | 102,448 tok/s | 54.64 GB | 5.280B tok iter 40300 | train 2.925 | val 2.887 | lr 5.17e-04 | 102,452 tok/s | 54.64 GB | 5.282B tok iter 40320 | train 3.026 | val 2.888 | lr 5.17e-04 | 102,456 tok/s | 54.64 GB | 5.285B tok iter 40340 | train 2.961 | val 2.880 | lr 5.17e-04 | 102,457 tok/s | 54.64 GB | 5.288B tok iter 40360 | train 2.823 | val 2.943 | lr 5.17e-04 | 102,461 tok/s | 54.64 GB | 5.290B tok iter 40380 | train 2.661 | val 2.917 | lr 5.17e-04 | 102,465 tok/s | 54.64 GB | 5.293B tok iter 40400 | train 2.980 | val 2.879 | lr 5.16e-04 | 102,469 tok/s | 54.64 GB | 5.295B tok iter 40420 | train 2.954 | val 2.891 | lr 5.16e-04 | 102,469 tok/s | 54.64 GB | 5.298B tok iter 40440 | train 2.855 | val 2.885 | lr 5.16e-04 | 102,474 tok/s | 54.64 GB | 5.301B tok iter 40460 | train 2.881 | val 2.880 | lr 5.16e-04 | 102,477 tok/s | 54.64 GB | 5.303B tok iter 40480 | train 2.910 | val 2.936 | lr 5.16e-04 | 102,481 tok/s | 54.64 GB | 5.306B tok checkpoint @ iter 40490 -> checkpoints/500m_latest.pt iter 40500 | train 2.905 | val 2.891 | lr 5.16e-04 | 102,456 tok/s | 54.64 GB | 5.309B tok checkpoint @ iter 40500 -> checkpoints/500m_latest.pt iter 40520 | train 2.829 | val 2.807 | lr 5.16e-04 | 102,407 tok/s | 54.64 GB | 5.311B tok iter 40540 | train 2.956 | val 2.928 | lr 5.16e-04 | 102,411 tok/s | 54.64 GB | 5.314B tok iter 40560 | train 2.854 | val 2.841 | lr 5.16e-04 | 102,415 tok/s | 54.64 GB | 5.316B tok iter 40580 | train 2.914 | val 2.847 | lr 5.16e-04 | 102,416 tok/s | 54.64 GB | 5.319B tok iter 40600 | train 2.974 | val 2.900 | lr 5.16e-04 | 102,420 tok/s | 54.64 GB | 5.322B tok iter 40620 | train 2.879 | val 2.843 | lr 5.16e-04 | 102,424 tok/s | 54.64 GB | 5.324B tok iter 40640 | train 2.946 | val 2.861 | lr 5.15e-04 | 102,428 tok/s | 54.64 GB | 5.327B tok iter 40660 | train 2.926 | val 2.883 | lr 5.15e-04 | 102,428 tok/s | 54.64 GB | 5.330B tok iter 40680 | train 2.944 | val 2.900 | lr 5.15e-04 | 102,432 tok/s | 54.64 GB | 5.332B tok iter 40700 | train 2.880 | val 2.914 | lr 5.15e-04 | 102,436 tok/s | 54.64 GB | 5.335B tok iter 40720 | train 2.940 | val 2.879 | lr 5.15e-04 | 102,440 tok/s | 54.64 GB | 5.337B tok iter 40740 | train 3.029 | val 2.891 | lr 5.15e-04 | 102,441 tok/s | 54.64 GB | 5.340B tok iter 40760 | train 2.944 | val 2.910 | lr 5.15e-04 | 102,444 tok/s | 54.64 GB | 5.343B tok iter 40780 | train 2.737 | val 2.880 | lr 5.15e-04 | 102,449 tok/s | 54.64 GB | 5.345B tok iter 40800 | train 2.934 | val 2.884 | lr 5.15e-04 | 102,452 tok/s | 54.64 GB | 5.348B tok iter 40820 | train 3.008 | val 2.865 | lr 5.15e-04 | 102,453 tok/s | 54.64 GB | 5.350B tok iter 40840 | train 2.853 | val 2.916 | lr 5.15e-04 | 102,457 tok/s | 54.64 GB | 5.353B tok iter 40860 | train 2.953 | val 2.880 | lr 5.15e-04 | 102,461 tok/s | 54.64 GB | 5.356B tok iter 40880 | train 2.789 | val 2.919 | lr 5.15e-04 | 102,465 tok/s | 54.64 GB | 5.358B tok iter 40900 | train 2.844 | val 2.874 | lr 5.14e-04 | 102,465 tok/s | 54.64 GB | 5.361B tok iter 40920 | train 2.902 | val 2.893 | lr 5.14e-04 | 102,469 tok/s | 54.64 GB | 5.364B tok iter 40940 | train 2.878 | val 2.884 | lr 5.14e-04 | 102,473 tok/s | 54.64 GB | 5.366B tok iter 40960 | train 2.864 | val 2.849 | lr 5.14e-04 | 102,477 tok/s | 54.64 GB | 5.369B tok iter 40980 | train 2.894 | val 2.889 | lr 5.14e-04 | 102,478 tok/s | 54.64 GB | 5.371B tok checkpoint @ iter 40990 -> checkpoints/500m_latest.pt iter 41000 | train 2.960 | val 2.858 | lr 5.14e-04 | 102,456 tok/s | 54.64 GB | 5.374B tok checkpoint @ iter 41000 -> checkpoints/500m_latest.pt iter 41020 | train 2.875 | val 2.873 | lr 5.14e-04 | 102,408 tok/s | 54.64 GB | 5.377B tok iter 41040 | train 2.901 | val 2.903 | lr 5.14e-04 | 102,412 tok/s | 54.64 GB | 5.379B tok iter 41060 | train 2.927 | val 2.877 | lr 5.14e-04 | 102,413 tok/s | 54.64 GB | 5.382B tok iter 41080 | train 2.914 | val 2.864 | lr 5.14e-04 | 102,417 tok/s | 54.64 GB | 5.385B tok iter 41100 | train 2.874 | val 2.903 | lr 5.14e-04 | 102,421 tok/s | 54.64 GB | 5.387B tok iter 41120 | train 2.899 | val 2.860 | lr 5.14e-04 | 102,425 tok/s | 54.64 GB | 5.390B tok iter 41140 | train 2.672 | val 2.874 | lr 5.13e-04 | 102,425 tok/s | 54.64 GB | 5.392B tok iter 41160 | train 2.916 | val 2.868 | lr 5.13e-04 | 102,429 tok/s | 54.64 GB | 5.395B tok iter 41180 | train 2.904 | val 2.904 | lr 5.13e-04 | 102,433 tok/s | 54.64 GB | 5.398B tok iter 41200 | train 2.795 | val 2.938 | lr 5.13e-04 | 102,437 tok/s | 54.64 GB | 5.400B tok iter 41220 | train 2.916 | val 2.839 | lr 5.13e-04 | 102,438 tok/s | 54.64 GB | 5.403B tok iter 41240 | train 2.756 | val 2.914 | lr 5.13e-04 | 102,442 tok/s | 54.64 GB | 5.406B tok iter 41260 | train 2.791 | val 2.886 | lr 5.13e-04 | 102,445 tok/s | 54.64 GB | 5.408B tok iter 41280 | train 2.937 | val 2.894 | lr 5.13e-04 | 102,449 tok/s | 54.64 GB | 5.411B tok iter 41300 | train 2.876 | val 2.908 | lr 5.13e-04 | 102,450 tok/s | 54.64 GB | 5.413B tok iter 41320 | train 2.753 | val 2.857 | lr 5.13e-04 | 102,454 tok/s | 54.64 GB | 5.416B tok iter 41340 | train 2.715 | val 2.863 | lr 5.13e-04 | 102,458 tok/s | 54.64 GB | 5.419B tok iter 41360 | train 2.841 | val 2.883 | lr 5.13e-04 | 102,462 tok/s | 54.64 GB | 5.421B tok iter 41380 | train 2.864 | val 2.889 | lr 5.12e-04 | 102,462 tok/s | 54.64 GB | 5.424B tok iter 41400 | train 2.838 | val 2.907 | lr 5.12e-04 | 102,466 tok/s | 54.64 GB | 5.427B tok iter 41420 | train 2.845 | val 2.875 | lr 5.12e-04 | 102,470 tok/s | 54.64 GB | 5.429B tok iter 41440 | train 2.902 | val 2.906 | lr 5.12e-04 | 102,474 tok/s | 54.64 GB | 5.432B tok iter 41460 | train 2.875 | val 2.878 | lr 5.12e-04 | 102,475 tok/s | 54.64 GB | 5.434B tok iter 41480 | train 2.853 | val 2.860 | lr 5.12e-04 | 102,479 tok/s | 54.64 GB | 5.437B tok checkpoint @ iter 41490 -> checkpoints/500m_latest.pt iter 41500 | train 2.772 | val 2.903 | lr 5.12e-04 | 102,457 tok/s | 54.64 GB | 5.440B tok checkpoint @ iter 41500 -> checkpoints/500m_latest.pt iter 41520 | train 2.990 | val 2.874 | lr 5.12e-04 | 102,409 tok/s | 54.64 GB | 5.442B tok iter 41540 | train 3.076 | val 2.864 | lr 5.12e-04 | 102,409 tok/s | 54.64 GB | 5.445B tok iter 41560 | train 2.913 | val 2.906 | lr 5.12e-04 | 102,413 tok/s | 54.64 GB | 5.447B tok iter 41580 | train 2.901 | val 2.920 | lr 5.12e-04 | 102,417 tok/s | 54.64 GB | 5.450B tok iter 41600 | train 3.079 | val 2.854 | lr 5.12e-04 | 102,421 tok/s | 54.64 GB | 5.453B tok iter 41620 | train 2.789 | val 2.871 | lr 5.11e-04 | 102,422 tok/s | 54.64 GB | 5.455B tok iter 41640 | train 2.977 | val 2.843 | lr 5.11e-04 | 102,426 tok/s | 54.64 GB | 5.458B tok iter 41660 | train 2.886 | val 2.840 | lr 5.11e-04 | 102,430 tok/s | 54.64 GB | 5.461B tok iter 41680 | train 2.853 | val 2.843 | lr 5.11e-04 | 102,434 tok/s | 54.64 GB | 5.463B tok iter 41700 | train 2.845 | val 2.935 | lr 5.11e-04 | 102,434 tok/s | 54.64 GB | 5.466B tok iter 41720 | train 3.020 | val 2.842 | lr 5.11e-04 | 102,438 tok/s | 54.64 GB | 5.468B tok iter 41740 | train 2.818 | val 2.888 | lr 5.11e-04 | 102,442 tok/s | 54.64 GB | 5.471B tok iter 41760 | train 2.881 | val 2.878 | lr 5.11e-04 | 102,446 tok/s | 54.64 GB | 5.474B tok iter 41780 | train 3.028 | val 2.861 | lr 5.11e-04 | 102,447 tok/s | 54.64 GB | 5.476B tok iter 41800 | train 2.818 | val 2.852 | lr 5.11e-04 | 102,451 tok/s | 54.64 GB | 5.479B tok iter 41820 | train 2.792 | val 2.882 | lr 5.11e-04 | 102,454 tok/s | 54.64 GB | 5.482B tok iter 41840 | train 2.925 | val 2.917 | lr 5.11e-04 | 102,458 tok/s | 54.64 GB | 5.484B tok iter 41860 | train 3.055 | val 2.871 | lr 5.10e-04 | 102,459 tok/s | 54.64 GB | 5.487B tok iter 41880 | train 2.857 | val 2.884 | lr 5.10e-04 | 102,463 tok/s | 54.64 GB | 5.489B tok iter 41900 | train 2.860 | val 2.888 | lr 5.10e-04 | 102,467 tok/s | 54.64 GB | 5.492B tok iter 41920 | train 2.769 | val 2.895 | lr 5.10e-04 | 102,471 tok/s | 54.64 GB | 5.495B tok iter 41940 | train 2.856 | val 2.848 | lr 5.10e-04 | 102,471 tok/s | 54.64 GB | 5.497B tok iter 41960 | train 2.910 | val 2.889 | lr 5.10e-04 | 102,475 tok/s | 54.64 GB | 5.500B tok iter 41980 | train 2.941 | val 2.834 | lr 5.10e-04 | 102,479 tok/s | 54.64 GB | 5.503B tok checkpoint @ iter 41990 -> checkpoints/500m_latest.pt iter 42000 | train 2.959 | val 2.853 | lr 5.10e-04 | 102,457 tok/s | 54.64 GB | 5.505B tok checkpoint @ iter 42000 -> checkpoints/500m_latest.pt iter 42020 | train 2.857 | val 2.907 | lr 5.10e-04 | 102,407 tok/s | 54.64 GB | 5.508B tok iter 42040 | train 2.973 | val 2.873 | lr 5.10e-04 | 102,411 tok/s | 54.64 GB | 5.510B tok iter 42060 | train 2.895 | val 2.889 | lr 5.10e-04 | 102,415 tok/s | 54.64 GB | 5.513B tok iter 42080 | train 2.909 | val 2.862 | lr 5.10e-04 | 102,419 tok/s | 54.64 GB | 5.516B tok iter 42100 | train 2.764 | val 2.862 | lr 5.09e-04 | 102,419 tok/s | 54.64 GB | 5.518B tok iter 42120 | train 2.900 | val 2.886 | lr 5.09e-04 | 102,423 tok/s | 54.64 GB | 5.521B tok iter 42140 | train 2.884 | val 2.884 | lr 5.09e-04 | 102,427 tok/s | 54.64 GB | 5.524B tok iter 42160 | train 2.784 | val 2.848 | lr 5.09e-04 | 102,431 tok/s | 54.64 GB | 5.526B tok iter 42180 | train 2.769 | val 2.877 | lr 5.09e-04 | 102,432 tok/s | 54.64 GB | 5.529B tok iter 42200 | train 2.825 | val 2.890 | lr 5.09e-04 | 102,435 tok/s | 54.64 GB | 5.531B tok iter 42220 | train 2.948 | val 2.894 | lr 5.09e-04 | 102,439 tok/s | 54.64 GB | 5.534B tok iter 42240 | train 2.900 | val 2.922 | lr 5.09e-04 | 102,443 tok/s | 54.64 GB | 5.537B tok iter 42260 | train 3.018 | val 2.901 | lr 5.09e-04 | 102,444 tok/s | 54.64 GB | 5.539B tok iter 42280 | train 2.698 | val 2.865 | lr 5.09e-04 | 102,448 tok/s | 54.64 GB | 5.542B tok iter 42300 | train 2.985 | val 2.881 | lr 5.09e-04 | 102,451 tok/s | 54.64 GB | 5.544B tok iter 42320 | train 2.923 | val 2.897 | lr 5.09e-04 | 102,455 tok/s | 54.64 GB | 5.547B tok iter 42340 | train 2.954 | val 2.864 | lr 5.08e-04 | 102,456 tok/s | 54.64 GB | 5.550B tok iter 42360 | train 2.909 | val 2.849 | lr 5.08e-04 | 102,460 tok/s | 54.64 GB | 5.552B tok iter 42380 | train 2.983 | val 2.860 | lr 5.08e-04 | 102,464 tok/s | 54.64 GB | 5.555B tok iter 42400 | train 2.995 | val 2.911 | lr 5.08e-04 | 102,467 tok/s | 54.64 GB | 5.558B tok iter 42420 | train 2.885 | val 2.881 | lr 5.08e-04 | 102,468 tok/s | 54.64 GB | 5.560B tok iter 42440 | train 2.805 | val 2.895 | lr 5.08e-04 | 102,472 tok/s | 54.64 GB | 5.563B tok iter 42460 | train 2.908 | val 2.869 | lr 5.08e-04 | 102,475 tok/s | 54.64 GB | 5.565B tok iter 42480 | train 2.921 | val 2.871 | lr 5.08e-04 | 102,479 tok/s | 54.64 GB | 5.568B tok checkpoint @ iter 42490 -> checkpoints/500m_latest.pt iter 42500 | train 2.892 | val 2.834 | lr 5.08e-04 | 102,453 tok/s | 54.64 GB | 5.571B tok checkpoint @ iter 42500 -> checkpoints/500m_latest.pt iter 42520 | train 2.700 | val 2.940 | lr 5.08e-04 | 102,408 tok/s | 54.64 GB | 5.573B tok iter 42540 | train 2.852 | val 2.893 | lr 5.08e-04 | 102,411 tok/s | 54.64 GB | 5.576B tok iter 42560 | train 2.891 | val 2.845 | lr 5.08e-04 | 102,415 tok/s | 54.64 GB | 5.579B tok iter 42580 | train 2.971 | val 2.913 | lr 5.07e-04 | 102,416 tok/s | 54.64 GB | 5.581B tok iter 42600 | train 2.966 | val 2.834 | lr 5.07e-04 | 102,420 tok/s | 54.64 GB | 5.584B tok iter 42620 | train 2.864 | val 2.890 | lr 5.07e-04 | 102,424 tok/s | 54.64 GB | 5.586B tok iter 42640 | train 2.841 | val 2.907 | lr 5.07e-04 | 102,427 tok/s | 54.64 GB | 5.589B tok iter 42660 | train 2.921 | val 2.863 | lr 5.07e-04 | 102,428 tok/s | 54.64 GB | 5.592B tok iter 42680 | train 2.947 | val 2.878 | lr 5.07e-04 | 102,432 tok/s | 54.64 GB | 5.594B tok iter 42700 | train 2.900 | val 2.880 | lr 5.07e-04 | 102,436 tok/s | 54.64 GB | 5.597B tok iter 42720 | train 2.752 | val 2.902 | lr 5.07e-04 | 102,440 tok/s | 54.64 GB | 5.600B tok iter 42740 | train 2.900 | val 2.918 | lr 5.07e-04 | 102,440 tok/s | 54.64 GB | 5.602B tok iter 42760 | train 2.813 | val 2.854 | lr 5.07e-04 | 102,444 tok/s | 54.64 GB | 5.605B tok iter 42780 | train 2.890 | val 2.903 | lr 5.07e-04 | 102,448 tok/s | 54.64 GB | 5.607B tok iter 42800 | train 2.927 | val 2.874 | lr 5.06e-04 | 102,452 tok/s | 54.64 GB | 5.610B tok iter 42820 | train 3.015 | val 2.880 | lr 5.06e-04 | 102,452 tok/s | 54.64 GB | 5.613B tok iter 42840 | train 2.920 | val 2.896 | lr 5.06e-04 | 102,456 tok/s | 54.64 GB | 5.615B tok iter 42860 | train 2.807 | val 2.842 | lr 5.06e-04 | 102,460 tok/s | 54.64 GB | 5.618B tok iter 42880 | train 2.880 | val 2.872 | lr 5.06e-04 | 102,463 tok/s | 54.64 GB | 5.620B tok iter 42900 | train 2.918 | val 2.869 | lr 5.06e-04 | 102,464 tok/s | 54.64 GB | 5.623B tok iter 42920 | train 2.957 | val 2.906 | lr 5.06e-04 | 102,468 tok/s | 54.64 GB | 5.626B tok iter 42940 | train 2.943 | val 2.852 | lr 5.06e-04 | 102,471 tok/s | 54.64 GB | 5.628B tok iter 42960 | train 3.066 | val 2.879 | lr 5.06e-04 | 102,475 tok/s | 54.64 GB | 5.631B tok iter 42980 | train 2.909 | val 2.847 | lr 5.06e-04 | 102,476 tok/s | 54.64 GB | 5.634B tok checkpoint @ iter 42990 -> checkpoints/500m_latest.pt iter 43000 | train 2.935 | val 2.848 | lr 5.06e-04 | 102,455 tok/s | 54.64 GB | 5.636B tok checkpoint @ iter 43000 -> checkpoints/500m_latest.pt iter 43020 | train 2.968 | val 2.913 | lr 5.06e-04 | 102,409 tok/s | 54.64 GB | 5.639B tok iter 43040 | train 2.997 | val 2.886 | lr 5.05e-04 | 102,413 tok/s | 54.64 GB | 5.641B tok iter 43060 | train 2.841 | val 2.929 | lr 5.05e-04 | 102,414 tok/s | 54.64 GB | 5.644B tok iter 43080 | train 2.852 | val 2.857 | lr 5.05e-04 | 102,418 tok/s | 54.64 GB | 5.647B tok iter 43100 | train 2.790 | val 2.859 | lr 5.05e-04 | 102,421 tok/s | 54.64 GB | 5.649B tok iter 43120 | train 2.714 | val 2.890 | lr 5.05e-04 | 102,425 tok/s | 54.64 GB | 5.652B tok iter 43140 | train 2.925 | val 2.896 | lr 5.05e-04 | 102,426 tok/s | 54.64 GB | 5.655B tok iter 43160 | train 2.993 | val 2.914 | lr 5.05e-04 | 102,430 tok/s | 54.64 GB | 5.657B tok iter 43180 | train 2.890 | val 2.865 | lr 5.05e-04 | 102,433 tok/s | 54.64 GB | 5.660B tok iter 43200 | train 2.887 | val 2.907 | lr 5.05e-04 | 102,437 tok/s | 54.64 GB | 5.662B tok iter 43220 | train 2.809 | val 2.877 | lr 5.05e-04 | 102,438 tok/s | 54.64 GB | 5.665B tok iter 43240 | train 2.853 | val 2.864 | lr 5.05e-04 | 102,442 tok/s | 54.64 GB | 5.668B tok iter 43260 | train 2.833 | val 2.896 | lr 5.05e-04 | 102,445 tok/s | 54.64 GB | 5.670B tok iter 43280 | train 2.917 | val 2.883 | lr 5.04e-04 | 102,449 tok/s | 54.64 GB | 5.673B tok iter 43300 | train 2.855 | val 2.888 | lr 5.04e-04 | 102,449 tok/s | 54.64 GB | 5.676B tok iter 43320 | train 2.828 | val 2.874 | lr 5.04e-04 | 102,453 tok/s | 54.64 GB | 5.678B tok iter 43340 | train 3.043 | val 2.871 | lr 5.04e-04 | 102,457 tok/s | 54.64 GB | 5.681B tok iter 43360 | train 2.778 | val 2.905 | lr 5.04e-04 | 102,460 tok/s | 54.64 GB | 5.683B tok iter 43380 | train 2.741 | val 2.926 | lr 5.04e-04 | 102,461 tok/s | 54.64 GB | 5.686B tok iter 43400 | train 2.932 | val 2.888 | lr 5.04e-04 | 102,465 tok/s | 54.64 GB | 5.689B tok iter 43420 | train 2.926 | val 2.838 | lr 5.04e-04 | 102,469 tok/s | 54.64 GB | 5.691B tok iter 43440 | train 2.926 | val 2.880 | lr 5.04e-04 | 102,472 tok/s | 54.64 GB | 5.694B tok iter 43460 | train 2.898 | val 2.897 | lr 5.04e-04 | 102,473 tok/s | 54.64 GB | 5.697B tok iter 43480 | train 2.911 | val 2.859 | lr 5.04e-04 | 102,477 tok/s | 54.64 GB | 5.699B tok checkpoint @ iter 43490 -> checkpoints/500m_latest.pt iter 43500 | train 2.830 | val 2.889 | lr 5.03e-04 | 102,456 tok/s | 54.64 GB | 5.702B tok checkpoint @ iter 43500 -> checkpoints/500m_latest.pt iter 43520 | train 2.777 | val 2.886 | lr 5.03e-04 | 102,410 tok/s | 54.64 GB | 5.704B tok iter 43540 | train 2.856 | val 2.893 | lr 5.03e-04 | 102,410 tok/s | 54.64 GB | 5.707B tok iter 43560 | train 2.926 | val 2.839 | lr 5.03e-04 | 102,414 tok/s | 54.64 GB | 5.710B tok iter 43580 | train 2.917 | val 2.843 | lr 5.03e-04 | 102,418 tok/s | 54.64 GB | 5.712B tok iter 43600 | train 2.940 | val 2.863 | lr 5.03e-04 | 102,422 tok/s | 54.64 GB | 5.715B tok iter 43620 | train 2.917 | val 2.851 | lr 5.03e-04 | 102,422 tok/s | 54.64 GB | 5.717B tok iter 43640 | train 2.976 | val 2.921 | lr 5.03e-04 | 102,426 tok/s | 54.64 GB | 5.720B tok iter 43660 | train 2.870 | val 2.878 | lr 5.03e-04 | 102,430 tok/s | 54.64 GB | 5.723B tok iter 43680 | train 2.959 | val 2.835 | lr 5.03e-04 | 102,433 tok/s | 54.64 GB | 5.725B tok iter 43700 | train 2.997 | val 2.885 | lr 5.03e-04 | 102,434 tok/s | 54.64 GB | 5.728B tok iter 43720 | train 2.760 | val 2.863 | lr 5.03e-04 | 102,438 tok/s | 54.64 GB | 5.731B tok iter 43740 | train 2.805 | val 2.848 | lr 5.02e-04 | 102,441 tok/s | 54.64 GB | 5.733B tok iter 43760 | train 2.927 | val 2.874 | lr 5.02e-04 | 102,445 tok/s | 54.64 GB | 5.736B tok iter 43780 | train 2.823 | val 2.888 | lr 5.02e-04 | 102,446 tok/s | 54.64 GB | 5.738B tok iter 43800 | train 2.848 | val 2.854 | lr 5.02e-04 | 102,450 tok/s | 54.64 GB | 5.741B tok iter 43820 | train 3.095 | val 2.912 | lr 5.02e-04 | 102,453 tok/s | 54.64 GB | 5.744B tok iter 43840 | train 2.941 | val 2.883 | lr 5.02e-04 | 102,457 tok/s | 54.64 GB | 5.746B tok iter 43860 | train 2.764 | val 2.873 | lr 5.02e-04 | 102,458 tok/s | 54.64 GB | 5.749B tok iter 43880 | train 2.675 | val 2.850 | lr 5.02e-04 | 102,461 tok/s | 54.64 GB | 5.752B tok iter 43900 | train 2.746 | val 2.849 | lr 5.02e-04 | 102,465 tok/s | 54.64 GB | 5.754B tok iter 43920 | train 2.993 | val 2.882 | lr 5.02e-04 | 102,469 tok/s | 54.64 GB | 5.757B tok iter 43940 | train 2.918 | val 2.880 | lr 5.02e-04 | 102,469 tok/s | 54.64 GB | 5.759B tok iter 43960 | train 2.806 | val 2.883 | lr 5.02e-04 | 102,473 tok/s | 54.64 GB | 5.762B tok iter 43980 | train 2.838 | val 2.894 | lr 5.01e-04 | 102,476 tok/s | 54.64 GB | 5.765B tok checkpoint @ iter 43990 -> checkpoints/500m_latest.pt iter 44000 | train 2.906 | val 2.875 | lr 5.01e-04 | 102,455 tok/s | 54.64 GB | 5.767B tok checkpoint @ iter 44000 -> checkpoints/500m_latest.pt iter 44020 | train 2.856 | val 2.872 | lr 5.01e-04 | 102,406 tok/s | 54.64 GB | 5.770B tok iter 44040 | train 2.834 | val 2.889 | lr 5.01e-04 | 102,409 tok/s | 54.64 GB | 5.773B tok iter 44060 | train 2.962 | val 2.890 | lr 5.01e-04 | 102,413 tok/s | 54.64 GB | 5.775B tok iter 44080 | train 2.837 | val 2.922 | lr 5.01e-04 | 102,417 tok/s | 54.64 GB | 5.778B tok iter 44100 | train 2.924 | val 2.849 | lr 5.01e-04 | 102,417 tok/s | 54.64 GB | 5.780B tok iter 44120 | train 2.872 | val 2.903 | lr 5.01e-04 | 102,421 tok/s | 54.64 GB | 5.783B tok iter 44140 | train 2.876 | val 2.876 | lr 5.01e-04 | 102,425 tok/s | 54.64 GB | 5.786B tok iter 44160 | train 2.835 | val 2.869 | lr 5.01e-04 | 102,428 tok/s | 54.64 GB | 5.788B tok iter 44180 | train 2.930 | val 2.908 | lr 5.01e-04 | 102,429 tok/s | 54.64 GB | 5.791B tok iter 44200 | train 2.764 | val 2.848 | lr 5.00e-04 | 102,432 tok/s | 54.64 GB | 5.794B tok iter 44220 | train 2.797 | val 2.909 | lr 5.00e-04 | 102,436 tok/s | 54.64 GB | 5.796B tok iter 44240 | train 2.901 | val 2.859 | lr 5.00e-04 | 102,440 tok/s | 54.64 GB | 5.799B tok iter 44260 | train 2.794 | val 2.855 | lr 5.00e-04 | 102,440 tok/s | 54.64 GB | 5.801B tok iter 44280 | train 2.788 | val 2.922 | lr 5.00e-04 | 102,444 tok/s | 54.64 GB | 5.804B tok iter 44300 | train 2.922 | val 2.883 | lr 5.00e-04 | 102,448 tok/s | 54.64 GB | 5.807B tok iter 44320 | train 2.883 | val 2.878 | lr 5.00e-04 | 102,451 tok/s | 54.64 GB | 5.809B tok iter 44340 | train 2.912 | val 2.939 | lr 5.00e-04 | 102,452 tok/s | 54.64 GB | 5.812B tok iter 44360 | train 2.785 | val 2.863 | lr 5.00e-04 | 102,456 tok/s | 54.64 GB | 5.814B tok iter 44380 | train 2.803 | val 2.920 | lr 5.00e-04 | 102,459 tok/s | 54.64 GB | 5.817B tok iter 44400 | train 3.031 | val 2.849 | lr 5.00e-04 | 102,463 tok/s | 54.64 GB | 5.820B tok iter 44420 | train 2.821 | val 2.842 | lr 5.00e-04 | 102,464 tok/s | 54.64 GB | 5.822B tok iter 44440 | train 2.838 | val 2.855 | lr 4.99e-04 | 102,467 tok/s | 54.64 GB | 5.825B tok iter 44460 | train 2.803 | val 2.893 | lr 4.99e-04 | 102,471 tok/s | 54.64 GB | 5.828B tok iter 44480 | train 2.771 | val 2.863 | lr 4.99e-04 | 102,475 tok/s | 54.64 GB | 5.830B tok checkpoint @ iter 44490 -> checkpoints/500m_latest.pt iter 44500 | train 2.764 | val 2.873 | lr 4.99e-04 | 102,451 tok/s | 54.64 GB | 5.833B tok checkpoint @ iter 44500 -> checkpoints/500m_latest.pt iter 44520 | train 2.954 | val 2.905 | lr 4.99e-04 | 102,407 tok/s | 54.64 GB | 5.835B tok iter 44540 | train 2.835 | val 2.867 | lr 4.99e-04 | 102,410 tok/s | 54.64 GB | 5.838B tok iter 44560 | train 2.831 | val 2.888 | lr 4.99e-04 | 102,414 tok/s | 54.64 GB | 5.841B tok iter 44580 | train 2.914 | val 2.902 | lr 4.99e-04 | 102,415 tok/s | 54.64 GB | 5.843B tok iter 44600 | train 2.857 | val 2.882 | lr 4.99e-04 | 102,418 tok/s | 54.64 GB | 5.846B tok iter 44620 | train 2.786 | val 2.891 | lr 4.99e-04 | 102,422 tok/s | 54.64 GB | 5.849B tok iter 44640 | train 2.864 | val 2.894 | lr 4.99e-04 | 102,426 tok/s | 54.64 GB | 5.851B tok iter 44660 | train 2.903 | val 2.912 | lr 4.98e-04 | 102,426 tok/s | 54.64 GB | 5.854B tok iter 44680 | train 2.896 | val 2.871 | lr 4.98e-04 | 102,430 tok/s | 54.64 GB | 5.856B tok iter 44700 | train 2.963 | val 2.877 | lr 4.98e-04 | 102,433 tok/s | 54.64 GB | 5.859B tok iter 44720 | train 2.890 | val 2.897 | lr 4.98e-04 | 102,437 tok/s | 54.64 GB | 5.862B tok iter 44740 | train 2.905 | val 2.880 | lr 4.98e-04 | 102,438 tok/s | 54.64 GB | 5.864B tok iter 44760 | train 2.879 | val 2.858 | lr 4.98e-04 | 102,441 tok/s | 54.64 GB | 5.867B tok iter 44780 | train 2.965 | val 2.919 | lr 4.98e-04 | 102,445 tok/s | 54.64 GB | 5.870B tok iter 44800 | train 2.734 | val 2.906 | lr 4.98e-04 | 102,449 tok/s | 54.64 GB | 5.872B tok iter 44820 | train 2.884 | val 2.855 | lr 4.98e-04 | 102,449 tok/s | 54.64 GB | 5.875B tok iter 44840 | train 2.893 | val 2.902 | lr 4.98e-04 | 102,453 tok/s | 54.64 GB | 5.877B tok iter 44860 | train 3.009 | val 2.888 | lr 4.98e-04 | 102,456 tok/s | 54.64 GB | 5.880B tok iter 44880 | train 2.943 | val 2.871 | lr 4.97e-04 | 102,460 tok/s | 54.64 GB | 5.883B tok iter 44900 | train 2.926 | val 2.854 | lr 4.97e-04 | 102,461 tok/s | 54.64 GB | 5.885B tok iter 44920 | train 2.932 | val 2.887 | lr 4.97e-04 | 102,464 tok/s | 54.64 GB | 5.888B tok iter 44940 | train 2.712 | val 2.889 | lr 4.97e-04 | 102,468 tok/s | 54.64 GB | 5.891B tok iter 44960 | train 2.816 | val 2.870 | lr 4.97e-04 | 102,471 tok/s | 54.64 GB | 5.893B tok iter 44980 | train 2.885 | val 2.880 | lr 4.97e-04 | 102,472 tok/s | 54.64 GB | 5.896B tok checkpoint @ iter 44990 -> checkpoints/500m_latest.pt iter 45000 | train 2.800 | val 2.859 | lr 4.97e-04 | 102,451 tok/s | 54.64 GB | 5.898B tok checkpoint @ iter 45000 -> checkpoints/500m_latest.pt iter 45020 | train 2.885 | val 2.850 | lr 4.97e-04 | 102,406 tok/s | 54.64 GB | 5.901B tok iter 45040 | train 2.994 | val 2.892 | lr 4.97e-04 | 102,410 tok/s | 54.64 GB | 5.904B tok iter 45060 | train 2.865 | val 2.836 | lr 4.97e-04 | 102,411 tok/s | 54.64 GB | 5.906B tok iter 45080 | train 2.809 | val 2.882 | lr 4.97e-04 | 102,414 tok/s | 54.64 GB | 5.909B tok iter 45100 | train 2.804 | val 2.862 | lr 4.97e-04 | 102,418 tok/s | 54.64 GB | 5.911B tok iter 45120 | train 2.859 | val 2.894 | lr 4.96e-04 | 102,421 tok/s | 54.64 GB | 5.914B tok iter 45140 | train 2.938 | val 2.827 | lr 4.96e-04 | 102,422 tok/s | 54.64 GB | 5.917B tok iter 45160 | train 2.766 | val 2.883 | lr 4.96e-04 | 102,426 tok/s | 54.64 GB | 5.919B tok iter 45180 | train 3.007 | val 2.907 | lr 4.96e-04 | 102,429 tok/s | 54.64 GB | 5.922B tok iter 45200 | train 2.857 | val 2.854 | lr 4.96e-04 | 102,433 tok/s | 54.64 GB | 5.925B tok iter 45220 | train 2.919 | val 2.850 | lr 4.96e-04 | 102,433 tok/s | 54.64 GB | 5.927B tok iter 45240 | train 2.919 | val 2.915 | lr 4.96e-04 | 102,437 tok/s | 54.64 GB | 5.930B tok iter 45260 | train 2.843 | val 2.865 | lr 4.96e-04 | 102,440 tok/s | 54.64 GB | 5.932B tok iter 45280 | train 2.791 | val 2.842 | lr 4.96e-04 | 102,444 tok/s | 54.64 GB | 5.935B tok iter 45300 | train 2.984 | val 2.854 | lr 4.96e-04 | 102,445 tok/s | 54.64 GB | 5.938B tok iter 45320 | train 2.869 | val 2.825 | lr 4.96e-04 | 102,448 tok/s | 54.64 GB | 5.940B tok iter 45340 | train 2.808 | val 2.919 | lr 4.95e-04 | 102,452 tok/s | 54.64 GB | 5.943B tok iter 45360 | train 2.916 | val 2.861 | lr 4.95e-04 | 102,455 tok/s | 54.64 GB | 5.946B tok iter 45380 | train 2.998 | val 2.854 | lr 4.95e-04 | 102,456 tok/s | 54.64 GB | 5.948B tok iter 45400 | train 2.948 | val 2.866 | lr 4.95e-04 | 102,459 tok/s | 54.64 GB | 5.951B tok iter 45420 | train 2.847 | val 2.870 | lr 4.95e-04 | 102,463 tok/s | 54.64 GB | 5.953B tok iter 45440 | train 2.853 | val 2.853 | lr 4.95e-04 | 102,466 tok/s | 54.64 GB | 5.956B tok iter 45460 | train 2.821 | val 2.862 | lr 4.95e-04 | 102,467 tok/s | 54.64 GB | 5.959B tok iter 45480 | train 2.943 | val 2.852 | lr 4.95e-04 | 102,471 tok/s | 54.64 GB | 5.961B tok checkpoint @ iter 45490 -> checkpoints/500m_latest.pt iter 45500 | train 2.866 | val 2.908 | lr 4.95e-04 | 102,451 tok/s | 54.64 GB | 5.964B tok checkpoint @ iter 45500 -> checkpoints/500m_latest.pt iter 45520 | train 2.867 | val 2.846 | lr 4.95e-04 | 102,408 tok/s | 54.64 GB | 5.967B tok iter 45540 | train 2.941 | val 2.886 | lr 4.95e-04 | 102,409 tok/s | 54.64 GB | 5.969B tok iter 45560 | train 2.988 | val 2.826 | lr 4.94e-04 | 102,412 tok/s | 54.64 GB | 5.972B tok iter 45580 | train 2.824 | val 2.856 | lr 4.94e-04 | 102,416 tok/s | 54.64 GB | 5.974B tok iter 45600 | train 2.762 | val 2.811 | lr 4.94e-04 | 102,419 tok/s | 54.64 GB | 5.977B tok iter 45620 | train 3.049 | val 2.851 | lr 4.94e-04 | 102,420 tok/s | 54.64 GB | 5.980B tok iter 45640 | train 3.062 | val 2.901 | lr 4.94e-04 | 102,424 tok/s | 54.64 GB | 5.982B tok iter 45660 | train 2.814 | val 2.831 | lr 4.94e-04 | 102,427 tok/s | 54.64 GB | 5.985B tok iter 45680 | train 2.918 | val 2.907 | lr 4.94e-04 | 102,431 tok/s | 54.64 GB | 5.988B tok iter 45700 | train 2.832 | val 2.919 | lr 4.94e-04 | 102,431 tok/s | 54.64 GB | 5.990B tok iter 45720 | train 2.803 | val 2.878 | lr 4.94e-04 | 102,435 tok/s | 54.64 GB | 5.993B tok iter 45740 | train 2.904 | val 2.884 | lr 4.94e-04 | 102,438 tok/s | 54.64 GB | 5.995B tok iter 45760 | train 2.786 | val 2.888 | lr 4.94e-04 | 102,442 tok/s | 54.64 GB | 5.998B tok iter 45780 | train 2.825 | val 2.909 | lr 4.93e-04 | 102,442 tok/s | 54.64 GB | 6.001B tok iter 45800 | train 2.906 | val 2.844 | lr 4.93e-04 | 102,446 tok/s | 54.64 GB | 6.003B tok iter 45820 | train 2.861 | val 2.883 | lr 4.93e-04 | 102,449 tok/s | 54.64 GB | 6.006B tok iter 45840 | train 2.887 | val 2.910 | lr 4.93e-04 | 102,453 tok/s | 54.64 GB | 6.008B tok iter 45860 | train 2.782 | val 2.875 | lr 4.93e-04 | 102,453 tok/s | 54.64 GB | 6.011B tok iter 45880 | train 2.989 | val 2.888 | lr 4.93e-04 | 102,457 tok/s | 54.64 GB | 6.014B tok iter 45900 | train 2.861 | val 2.864 | lr 4.93e-04 | 102,460 tok/s | 54.64 GB | 6.016B tok iter 45920 | train 2.843 | val 2.867 | lr 4.93e-04 | 102,464 tok/s | 54.64 GB | 6.019B tok iter 45940 | train 2.863 | val 2.908 | lr 4.93e-04 | 102,464 tok/s | 54.64 GB | 6.022B tok iter 45960 | train 2.799 | val 2.853 | lr 4.93e-04 | 102,468 tok/s | 54.64 GB | 6.024B tok iter 45980 | train 2.715 | val 2.861 | lr 4.93e-04 | 102,471 tok/s | 54.64 GB | 6.027B tok checkpoint @ iter 45990 -> checkpoints/500m_latest.pt iter 46000 | train 2.947 | val 2.903 | lr 4.93e-04 | 102,451 tok/s | 54.64 GB | 6.029B tok checkpoint @ iter 46000 -> checkpoints/500m_latest.pt iter 46020 | train 2.845 | val 2.873 | lr 4.92e-04 | 102,406 tok/s | 54.64 GB | 6.032B tok iter 46040 | train 2.819 | val 2.837 | lr 4.92e-04 | 102,409 tok/s | 54.64 GB | 6.035B tok iter 46060 | train 2.920 | val 2.860 | lr 4.92e-04 | 102,413 tok/s | 54.64 GB | 6.037B tok iter 46080 | train 2.980 | val 2.869 | lr 4.92e-04 | 102,416 tok/s | 54.64 GB | 6.040B tok iter 46100 | train 2.799 | val 2.903 | lr 4.92e-04 | 102,417 tok/s | 54.64 GB | 6.043B tok iter 46120 | train 2.853 | val 2.871 | lr 4.92e-04 | 102,420 tok/s | 54.64 GB | 6.045B tok iter 46140 | train 2.964 | val 2.871 | lr 4.92e-04 | 102,424 tok/s | 54.64 GB | 6.048B tok iter 46160 | train 2.809 | val 2.841 | lr 4.92e-04 | 102,427 tok/s | 54.64 GB | 6.050B tok iter 46180 | train 2.931 | val 2.854 | lr 4.92e-04 | 102,428 tok/s | 54.64 GB | 6.053B tok iter 46200 | train 2.809 | val 2.829 | lr 4.92e-04 | 102,432 tok/s | 54.64 GB | 6.056B tok iter 46220 | train 2.891 | val 2.912 | lr 4.92e-04 | 102,435 tok/s | 54.64 GB | 6.058B tok iter 46240 | train 2.722 | val 2.877 | lr 4.91e-04 | 102,438 tok/s | 54.64 GB | 6.061B tok iter 46260 | train 2.935 | val 2.847 | lr 4.91e-04 | 102,439 tok/s | 54.64 GB | 6.064B tok iter 46280 | train 3.005 | val 2.882 | lr 4.91e-04 | 102,443 tok/s | 54.64 GB | 6.066B tok iter 46300 | train 2.858 | val 2.882 | lr 4.91e-04 | 102,446 tok/s | 54.64 GB | 6.069B tok iter 46320 | train 2.774 | val 2.812 | lr 4.91e-04 | 102,450 tok/s | 54.64 GB | 6.071B tok iter 46340 | train 2.727 | val 2.825 | lr 4.91e-04 | 102,450 tok/s | 54.64 GB | 6.074B tok iter 46360 | train 2.927 | val 2.828 | lr 4.91e-04 | 102,454 tok/s | 54.64 GB | 6.077B tok iter 46380 | train 2.728 | val 2.891 | lr 4.91e-04 | 102,457 tok/s | 54.64 GB | 6.079B tok iter 46400 | train 2.926 | val 2.830 | lr 4.91e-04 | 102,461 tok/s | 54.64 GB | 6.082B tok iter 46420 | train 2.764 | val 2.899 | lr 4.91e-04 | 102,461 tok/s | 54.64 GB | 6.084B tok iter 46440 | train 2.719 | val 2.892 | lr 4.91e-04 | 102,465 tok/s | 54.64 GB | 6.087B tok iter 46460 | train 2.918 | val 2.903 | lr 4.90e-04 | 102,468 tok/s | 54.64 GB | 6.090B tok iter 46480 | train 2.990 | val 2.887 | lr 4.90e-04 | 102,472 tok/s | 54.64 GB | 6.092B tok checkpoint @ iter 46490 -> checkpoints/500m_latest.pt iter 46500 | train 2.871 | val 2.854 | lr 4.90e-04 | 102,449 tok/s | 54.64 GB | 6.095B tok checkpoint @ iter 46500 -> checkpoints/500m_latest.pt iter 46520 | train 2.729 | val 2.847 | lr 4.90e-04 | 102,408 tok/s | 54.64 GB | 6.098B tok iter 46540 | train 2.688 | val 2.870 | lr 4.90e-04 | 102,411 tok/s | 54.64 GB | 6.100B tok iter 46560 | train 2.891 | val 2.845 | lr 4.90e-04 | 102,415 tok/s | 54.64 GB | 6.103B tok iter 46580 | train 2.772 | val 2.868 | lr 4.90e-04 | 102,415 tok/s | 54.64 GB | 6.105B tok iter 46600 | train 2.836 | val 2.875 | lr 4.90e-04 | 102,418 tok/s | 54.64 GB | 6.108B tok iter 46620 | train 2.835 | val 2.871 | lr 4.90e-04 | 102,422 tok/s | 54.64 GB | 6.111B tok iter 46640 | train 2.928 | val 2.876 | lr 4.90e-04 | 102,425 tok/s | 54.64 GB | 6.113B tok iter 46660 | train 2.962 | val 2.852 | lr 4.90e-04 | 102,426 tok/s | 54.64 GB | 6.116B tok iter 46680 | train 2.876 | val 2.909 | lr 4.89e-04 | 102,429 tok/s | 54.64 GB | 6.119B tok iter 46700 | train 3.031 | val 2.863 | lr 4.89e-04 | 102,433 tok/s | 54.64 GB | 6.121B tok iter 46720 | train 2.855 | val 2.840 | lr 4.89e-04 | 102,436 tok/s | 54.64 GB | 6.124B tok iter 46740 | train 2.916 | val 2.845 | lr 4.89e-04 | 102,437 tok/s | 54.64 GB | 6.126B tok iter 46760 | train 2.880 | val 2.874 | lr 4.89e-04 | 102,440 tok/s | 54.64 GB | 6.129B tok iter 46780 | train 2.903 | val 2.895 | lr 4.89e-04 | 102,444 tok/s | 54.64 GB | 6.132B tok iter 46800 | train 2.727 | val 2.891 | lr 4.89e-04 | 102,447 tok/s | 54.64 GB | 6.134B tok iter 46820 | train 2.863 | val 2.875 | lr 4.89e-04 | 102,448 tok/s | 54.64 GB | 6.137B tok iter 46840 | train 2.840 | val 2.865 | lr 4.89e-04 | 102,451 tok/s | 54.64 GB | 6.140B tok iter 46860 | train 2.586 | val 2.818 | lr 4.89e-04 | 102,455 tok/s | 54.64 GB | 6.142B tok iter 46880 | train 2.928 | val 2.861 | lr 4.89e-04 | 102,458 tok/s | 54.64 GB | 6.145B tok iter 46900 | train 2.932 | val 2.884 | lr 4.88e-04 | 102,459 tok/s | 54.64 GB | 6.147B tok iter 46920 | train 2.789 | val 2.845 | lr 4.88e-04 | 102,462 tok/s | 54.64 GB | 6.150B tok iter 46940 | train 2.911 | val 2.828 | lr 4.88e-04 | 102,466 tok/s | 54.64 GB | 6.153B tok iter 46960 | train 2.773 | val 2.882 | lr 4.88e-04 | 102,469 tok/s | 54.64 GB | 6.155B tok iter 46980 | train 2.930 | val 2.873 | lr 4.88e-04 | 102,470 tok/s | 54.64 GB | 6.158B tok checkpoint @ iter 46990 -> checkpoints/500m_latest.pt iter 47000 | train 2.884 | val 2.891 | lr 4.88e-04 | 102,451 tok/s | 54.64 GB | 6.161B tok checkpoint @ iter 47000 -> checkpoints/500m_latest.pt iter 47020 | train 2.803 | val 2.916 | lr 4.88e-04 | 102,407 tok/s | 54.64 GB | 6.163B tok iter 47040 | train 2.903 | val 2.885 | lr 4.88e-04 | 102,410 tok/s | 54.64 GB | 6.166B tok iter 47060 | train 2.809 | val 2.817 | lr 4.88e-04 | 102,411 tok/s | 54.64 GB | 6.168B tok iter 47080 | train 2.773 | val 2.866 | lr 4.88e-04 | 102,414 tok/s | 54.64 GB | 6.171B tok iter 47100 | train 2.731 | val 2.874 | lr 4.88e-04 | 102,418 tok/s | 54.64 GB | 6.174B tok iter 47120 | train 2.781 | val 2.816 | lr 4.87e-04 | 102,421 tok/s | 54.64 GB | 6.176B tok iter 47140 | train 2.984 | val 2.897 | lr 4.87e-04 | 102,422 tok/s | 54.64 GB | 6.179B tok iter 47160 | train 2.870 | val 2.914 | lr 4.87e-04 | 102,425 tok/s | 54.64 GB | 6.181B tok iter 47180 | train 2.884 | val 2.841 | lr 4.87e-04 | 102,429 tok/s | 54.64 GB | 6.184B tok iter 47200 | train 2.717 | val 2.862 | lr 4.87e-04 | 102,432 tok/s | 54.64 GB | 6.187B tok iter 47220 | train 2.870 | val 2.856 | lr 4.87e-04 | 102,433 tok/s | 54.64 GB | 6.189B tok iter 47240 | train 2.937 | val 2.848 | lr 4.87e-04 | 102,436 tok/s | 54.64 GB | 6.192B tok iter 47260 | train 2.715 | val 2.909 | lr 4.87e-04 | 102,440 tok/s | 54.64 GB | 6.195B tok iter 47280 | train 2.707 | val 2.842 | lr 4.87e-04 | 102,443 tok/s | 54.64 GB | 6.197B tok iter 47300 | train 2.996 | val 2.865 | lr 4.87e-04 | 102,444 tok/s | 54.64 GB | 6.200B tok iter 47320 | train 2.872 | val 2.879 | lr 4.87e-04 | 102,447 tok/s | 54.64 GB | 6.202B tok iter 47340 | train 2.837 | val 2.847 | lr 4.86e-04 | 102,450 tok/s | 54.64 GB | 6.205B tok iter 47360 | train 2.841 | val 2.893 | lr 4.86e-04 | 102,454 tok/s | 54.64 GB | 6.208B tok iter 47380 | train 2.886 | val 2.896 | lr 4.86e-04 | 102,455 tok/s | 54.64 GB | 6.210B tok iter 47400 | train 2.802 | val 2.881 | lr 4.86e-04 | 102,458 tok/s | 54.64 GB | 6.213B tok iter 47420 | train 2.982 | val 2.888 | lr 4.86e-04 | 102,461 tok/s | 54.64 GB | 6.216B tok iter 47440 | train 2.807 | val 2.861 | lr 4.86e-04 | 102,465 tok/s | 54.64 GB | 6.218B tok iter 47460 | train 2.953 | val 2.880 | lr 4.86e-04 | 102,465 tok/s | 54.64 GB | 6.221B tok iter 47480 | train 2.837 | val 2.894 | lr 4.86e-04 | 102,468 tok/s | 54.64 GB | 6.223B tok checkpoint @ iter 47489 -> checkpoints/500m_latest.pt iter 47500 | train 2.846 | val 2.820 | lr 4.86e-04 | 102,449 tok/s | 54.64 GB | 6.226B tok checkpoint @ iter 47500 -> checkpoints/500m_latest.pt iter 47520 | train 2.898 | val 2.835 | lr 4.86e-04 | 102,407 tok/s | 54.64 GB | 6.229B tok iter 47540 | train 3.006 | val 2.878 | lr 4.86e-04 | 102,407 tok/s | 54.64 GB | 6.231B tok iter 47560 | train 2.857 | val 2.911 | lr 4.85e-04 | 102,411 tok/s | 54.64 GB | 6.234B tok iter 47580 | train 2.917 | val 2.862 | lr 4.85e-04 | 102,414 tok/s | 54.64 GB | 6.237B tok iter 47600 | train 2.807 | val 2.844 | lr 4.85e-04 | 102,417 tok/s | 54.64 GB | 6.239B tok iter 47620 | train 2.719 | val 2.861 | lr 4.85e-04 | 102,418 tok/s | 54.64 GB | 6.242B tok iter 47640 | train 2.884 | val 2.865 | lr 4.85e-04 | 102,421 tok/s | 54.64 GB | 6.244B tok iter 47660 | train 2.910 | val 2.885 | lr 4.85e-04 | 102,425 tok/s | 54.64 GB | 6.247B tok iter 47680 | train 2.918 | val 2.829 | lr 4.85e-04 | 102,428 tok/s | 54.64 GB | 6.250B tok iter 47700 | train 2.717 | val 2.866 | lr 4.85e-04 | 102,429 tok/s | 54.64 GB | 6.252B tok iter 47720 | train 2.902 | val 2.874 | lr 4.85e-04 | 102,432 tok/s | 54.64 GB | 6.255B tok iter 47740 | train 2.924 | val 2.846 | lr 4.85e-04 | 102,435 tok/s | 54.64 GB | 6.258B tok iter 47760 | train 2.903 | val 2.885 | lr 4.85e-04 | 102,439 tok/s | 54.64 GB | 6.260B tok iter 47780 | train 2.884 | val 2.840 | lr 4.84e-04 | 102,439 tok/s | 54.64 GB | 6.263B tok iter 47800 | train 2.825 | val 2.862 | lr 4.84e-04 | 102,443 tok/s | 54.64 GB | 6.265B tok iter 47820 | train 2.863 | val 2.869 | lr 4.84e-04 | 102,446 tok/s | 54.64 GB | 6.268B tok iter 47840 | train 2.739 | val 2.833 | lr 4.84e-04 | 102,449 tok/s | 54.64 GB | 6.271B tok iter 47860 | train 2.787 | val 2.850 | lr 4.84e-04 | 102,450 tok/s | 54.64 GB | 6.273B tok iter 47880 | train 2.929 | val 2.855 | lr 4.84e-04 | 102,453 tok/s | 54.64 GB | 6.276B tok iter 47900 | train 2.912 | val 2.846 | lr 4.84e-04 | 102,457 tok/s | 54.64 GB | 6.278B tok iter 47920 | train 2.810 | val 2.868 | lr 4.84e-04 | 102,460 tok/s | 54.64 GB | 6.281B tok iter 47940 | train 2.738 | val 2.873 | lr 4.84e-04 | 102,461 tok/s | 54.64 GB | 6.284B tok iter 47960 | train 2.831 | val 2.873 | lr 4.84e-04 | 102,464 tok/s | 54.64 GB | 6.286B tok iter 47980 | train 3.036 | val 2.859 | lr 4.83e-04 | 102,467 tok/s | 54.64 GB | 6.289B tok checkpoint @ iter 47990 -> checkpoints/500m_latest.pt iter 48000 | train 2.953 | val 2.886 | lr 4.83e-04 | 102,448 tok/s | 54.64 GB | 6.292B tok checkpoint @ iter 48000 -> checkpoints/500m_latest.pt iter 48020 | train 2.861 | val 2.853 | lr 4.83e-04 | 102,406 tok/s | 54.64 GB | 6.294B tok iter 48040 | train 2.836 | val 2.867 | lr 4.83e-04 | 102,410 tok/s | 54.64 GB | 6.297B tok iter 48060 | train 2.755 | val 2.868 | lr 4.83e-04 | 102,413 tok/s | 54.64 GB | 6.299B tok iter 48080 | train 2.838 | val 2.832 | lr 4.83e-04 | 102,416 tok/s | 54.64 GB | 6.302B tok iter 48100 | train 2.911 | val 2.855 | lr 4.83e-04 | 102,417 tok/s | 54.64 GB | 6.305B tok iter 48120 | train 2.800 | val 2.896 | lr 4.83e-04 | 102,420 tok/s | 54.64 GB | 6.307B tok iter 48140 | train 2.768 | val 2.891 | lr 4.83e-04 | 102,424 tok/s | 54.64 GB | 6.310B tok iter 48160 | train 2.832 | val 2.915 | lr 4.83e-04 | 102,427 tok/s | 54.64 GB | 6.313B tok iter 48180 | train 2.842 | val 2.862 | lr 4.83e-04 | 102,427 tok/s | 54.64 GB | 6.315B tok iter 48200 | train 3.015 | val 2.861 | lr 4.82e-04 | 102,431 tok/s | 54.64 GB | 6.318B tok iter 48220 | train 2.874 | val 2.850 | lr 4.82e-04 | 102,434 tok/s | 54.64 GB | 6.320B tok iter 48240 | train 3.012 | val 2.835 | lr 4.82e-04 | 102,437 tok/s | 54.64 GB | 6.323B tok iter 48260 | train 2.788 | val 2.920 | lr 4.82e-04 | 102,438 tok/s | 54.64 GB | 6.326B tok iter 48280 | train 2.837 | val 2.843 | lr 4.82e-04 | 102,441 tok/s | 54.64 GB | 6.328B tok iter 48300 | train 2.842 | val 2.856 | lr 4.82e-04 | 102,444 tok/s | 54.64 GB | 6.331B tok iter 48320 | train 2.964 | val 2.865 | lr 4.82e-04 | 102,448 tok/s | 54.64 GB | 6.334B tok iter 48340 | train 2.933 | val 2.925 | lr 4.82e-04 | 102,448 tok/s | 54.64 GB | 6.336B tok iter 48360 | train 2.850 | val 2.859 | lr 4.82e-04 | 102,452 tok/s | 54.64 GB | 6.339B tok iter 48380 | train 2.982 | val 2.898 | lr 4.82e-04 | 102,455 tok/s | 54.64 GB | 6.341B tok iter 48400 | train 2.859 | val 2.852 | lr 4.82e-04 | 102,458 tok/s | 54.64 GB | 6.344B tok iter 48420 | train 2.701 | val 2.835 | lr 4.81e-04 | 102,459 tok/s | 54.64 GB | 6.347B tok iter 48440 | train 2.773 | val 2.883 | lr 4.81e-04 | 102,462 tok/s | 54.64 GB | 6.349B tok iter 48460 | train 2.764 | val 2.857 | lr 4.81e-04 | 102,465 tok/s | 54.64 GB | 6.352B tok iter 48480 | train 2.914 | val 2.840 | lr 4.81e-04 | 102,469 tok/s | 54.64 GB | 6.355B tok checkpoint @ iter 48490 -> checkpoints/500m_latest.pt iter 48500 | train 2.895 | val 2.856 | lr 4.81e-04 | 102,447 tok/s | 54.64 GB | 6.357B tok checkpoint @ iter 48500 -> checkpoints/500m_latest.pt iter 48520 | train 2.859 | val 2.863 | lr 4.81e-04 | 102,406 tok/s | 54.64 GB | 6.360B tok iter 48540 | train 2.848 | val 2.833 | lr 4.81e-04 | 102,409 tok/s | 54.64 GB | 6.362B tok iter 48560 | train 2.879 | val 2.904 | lr 4.81e-04 | 102,413 tok/s | 54.64 GB | 6.365B tok iter 48580 | train 2.910 | val 2.848 | lr 4.81e-04 | 102,413 tok/s | 54.64 GB | 6.368B tok iter 48600 | train 2.987 | val 2.841 | lr 4.81e-04 | 102,417 tok/s | 54.64 GB | 6.370B tok iter 48620 | train 2.897 | val 2.894 | lr 4.81e-04 | 102,420 tok/s | 54.64 GB | 6.373B tok iter 48640 | train 2.907 | val 2.857 | lr 4.80e-04 | 102,423 tok/s | 54.64 GB | 6.375B tok iter 48660 | train 2.854 | val 2.879 | lr 4.80e-04 | 102,424 tok/s | 54.64 GB | 6.378B tok iter 48680 | train 2.951 | val 2.868 | lr 4.80e-04 | 102,427 tok/s | 54.64 GB | 6.381B tok iter 48700 | train 2.881 | val 2.847 | lr 4.80e-04 | 102,431 tok/s | 54.64 GB | 6.383B tok iter 48720 | train 2.809 | val 2.883 | lr 4.80e-04 | 102,434 tok/s | 54.64 GB | 6.386B tok iter 48740 | train 2.794 | val 2.847 | lr 4.80e-04 | 102,435 tok/s | 54.64 GB | 6.389B tok iter 48760 | train 2.883 | val 2.876 | lr 4.80e-04 | 102,438 tok/s | 54.64 GB | 6.391B tok iter 48780 | train 2.761 | val 2.821 | lr 4.80e-04 | 102,441 tok/s | 54.64 GB | 6.394B tok iter 48800 | train 2.914 | val 2.868 | lr 4.80e-04 | 102,444 tok/s | 54.64 GB | 6.396B tok iter 48820 | train 2.671 | val 2.873 | lr 4.80e-04 | 102,445 tok/s | 54.64 GB | 6.399B tok iter 48840 | train 2.847 | val 2.854 | lr 4.80e-04 | 102,448 tok/s | 54.64 GB | 6.402B tok iter 48860 | train 2.805 | val 2.886 | lr 4.79e-04 | 102,452 tok/s | 54.64 GB | 6.404B tok iter 48880 | train 2.821 | val 2.847 | lr 4.79e-04 | 102,455 tok/s | 54.64 GB | 6.407B tok iter 48900 | train 2.771 | val 2.865 | lr 4.79e-04 | 102,455 tok/s | 54.64 GB | 6.410B tok iter 48920 | train 2.854 | val 2.903 | lr 4.79e-04 | 102,459 tok/s | 54.64 GB | 6.412B tok iter 48940 | train 2.881 | val 2.826 | lr 4.79e-04 | 102,462 tok/s | 54.64 GB | 6.415B tok iter 48960 | train 2.857 | val 2.850 | lr 4.79e-04 | 102,465 tok/s | 54.64 GB | 6.417B tok iter 48980 | train 2.959 | val 2.866 | lr 4.79e-04 | 102,466 tok/s | 54.64 GB | 6.420B tok checkpoint @ iter 48990 -> checkpoints/500m_latest.pt iter 49000 | train 2.818 | val 2.821 | lr 4.79e-04 | 102,448 tok/s | 54.64 GB | 6.423B tok checkpoint @ iter 49000 -> checkpoints/500m_latest.pt iter 49020 | train 2.727 | val 2.861 | lr 4.79e-04 | 102,409 tok/s | 54.64 GB | 6.425B tok iter 49040 | train 2.902 | val 2.881 | lr 4.79e-04 | 102,412 tok/s | 54.64 GB | 6.428B tok iter 49060 | train 2.885 | val 2.894 | lr 4.78e-04 | 102,413 tok/s | 54.64 GB | 6.431B tok iter 49080 | train 2.955 | val 2.900 | lr 4.78e-04 | 102,416 tok/s | 54.64 GB | 6.433B tok iter 49100 | train 2.898 | val 2.883 | lr 4.78e-04 | 102,420 tok/s | 54.64 GB | 6.436B tok iter 49120 | train 2.928 | val 2.853 | lr 4.78e-04 | 102,423 tok/s | 54.64 GB | 6.438B tok iter 49140 | train 2.875 | val 2.830 | lr 4.78e-04 | 102,423 tok/s | 54.64 GB | 6.441B tok iter 49160 | train 2.946 | val 2.861 | lr 4.78e-04 | 102,426 tok/s | 54.64 GB | 6.444B tok iter 49180 | train 2.827 | val 2.888 | lr 4.78e-04 | 102,430 tok/s | 54.64 GB | 6.446B tok iter 49200 | train 2.737 | val 2.875 | lr 4.78e-04 | 102,433 tok/s | 54.64 GB | 6.449B tok iter 49220 | train 2.799 | val 2.858 | lr 4.78e-04 | 102,434 tok/s | 54.64 GB | 6.451B tok iter 49240 | train 2.859 | val 2.822 | lr 4.78e-04 | 102,437 tok/s | 54.64 GB | 6.454B tok iter 49260 | train 2.900 | val 2.869 | lr 4.78e-04 | 102,440 tok/s | 54.64 GB | 6.457B tok iter 49280 | train 2.742 | val 2.871 | lr 4.77e-04 | 102,443 tok/s | 54.64 GB | 6.459B tok iter 49300 | train 3.000 | val 2.827 | lr 4.77e-04 | 102,444 tok/s | 54.64 GB | 6.462B tok iter 49320 | train 2.854 | val 2.881 | lr 4.77e-04 | 102,447 tok/s | 54.64 GB | 6.465B tok iter 49340 | train 2.948 | val 2.859 | lr 4.77e-04 | 102,450 tok/s | 54.64 GB | 6.467B tok iter 49360 | train 2.898 | val 2.839 | lr 4.77e-04 | 102,454 tok/s | 54.64 GB | 6.470B tok iter 49380 | train 2.802 | val 2.856 | lr 4.77e-04 | 102,454 tok/s | 54.64 GB | 6.472B tok iter 49400 | train 3.027 | val 2.904 | lr 4.77e-04 | 102,458 tok/s | 54.64 GB | 6.475B tok iter 49420 | train 2.861 | val 2.846 | lr 4.77e-04 | 102,461 tok/s | 54.64 GB | 6.478B tok iter 49440 | train 2.929 | val 2.845 | lr 4.77e-04 | 102,464 tok/s | 54.64 GB | 6.480B tok iter 49460 | train 2.841 | val 2.841 | lr 4.77e-04 | 102,465 tok/s | 54.64 GB | 6.483B tok iter 49480 | train 2.934 | val 2.859 | lr 4.76e-04 | 102,468 tok/s | 54.64 GB | 6.486B tok checkpoint @ iter 49490 -> checkpoints/500m_latest.pt iter 49500 | train 2.923 | val 2.844 | lr 4.76e-04 | 102,450 tok/s | 54.64 GB | 6.488B tok checkpoint @ iter 49500 -> checkpoints/500m_latest.pt iter 49520 | train 2.896 | val 2.860 | lr 4.76e-04 | 102,411 tok/s | 54.64 GB | 6.491B tok iter 49540 | train 2.832 | val 2.851 | lr 4.76e-04 | 102,412 tok/s | 54.64 GB | 6.493B tok iter 49560 | train 2.942 | val 2.852 | lr 4.76e-04 | 102,415 tok/s | 54.64 GB | 6.496B tok iter 49580 | train 2.791 | val 2.895 | lr 4.76e-04 | 102,418 tok/s | 54.64 GB | 6.499B tok iter 49600 | train 2.879 | val 2.851 | lr 4.76e-04 | 102,421 tok/s | 54.64 GB | 6.501B tok iter 49620 | train 2.999 | val 2.874 | lr 4.76e-04 | 102,422 tok/s | 54.64 GB | 6.504B tok iter 49640 | train 2.858 | val 2.861 | lr 4.76e-04 | 102,425 tok/s | 54.64 GB | 6.507B tok iter 49660 | train 2.821 | val 2.819 | lr 4.76e-04 | 102,428 tok/s | 54.64 GB | 6.509B tok iter 49680 | train 2.852 | val 2.838 | lr 4.76e-04 | 102,432 tok/s | 54.64 GB | 6.512B tok iter 49700 | train 2.878 | val 2.868 | lr 4.75e-04 | 102,432 tok/s | 54.64 GB | 6.514B tok iter 49720 | train 2.796 | val 2.841 | lr 4.75e-04 | 102,435 tok/s | 54.64 GB | 6.517B tok iter 49740 | train 2.876 | val 2.878 | lr 4.75e-04 | 102,439 tok/s | 54.64 GB | 6.520B tok iter 49760 | train 2.947 | val 2.854 | lr 4.75e-04 | 102,442 tok/s | 54.64 GB | 6.522B tok iter 49780 | train 2.746 | val 2.860 | lr 4.75e-04 | 102,442 tok/s | 54.64 GB | 6.525B tok iter 49800 | train 2.886 | val 2.860 | lr 4.75e-04 | 102,446 tok/s | 54.64 GB | 6.528B tok iter 49820 | train 2.786 | val 2.892 | lr 4.75e-04 | 102,449 tok/s | 54.64 GB | 6.530B tok iter 49840 | train 2.949 | val 2.811 | lr 4.75e-04 | 102,452 tok/s | 54.64 GB | 6.533B tok iter 49860 | train 2.863 | val 2.837 | lr 4.75e-04 | 102,453 tok/s | 54.64 GB | 6.535B tok iter 49880 | train 2.948 | val 2.885 | lr 4.75e-04 | 102,456 tok/s | 54.64 GB | 6.538B tok iter 49900 | train 2.835 | val 2.816 | lr 4.75e-04 | 102,459 tok/s | 54.64 GB | 6.541B tok iter 49920 | train 2.866 | val 2.875 | lr 4.74e-04 | 102,462 tok/s | 54.64 GB | 6.543B tok iter 49940 | train 2.831 | val 2.837 | lr 4.74e-04 | 102,463 tok/s | 54.64 GB | 6.546B tok iter 49960 | train 2.880 | val 2.872 | lr 4.74e-04 | 102,466 tok/s | 54.64 GB | 6.548B tok iter 49980 | train 2.832 | val 2.889 | lr 4.74e-04 | 102,469 tok/s | 54.64 GB | 6.551B tok checkpoint @ iter 49990 -> checkpoints/500m_latest.pt iter 50000 | train 2.882 | val 2.830 | lr 4.74e-04 | 102,451 tok/s | 54.64 GB | 6.554B tok checkpoint @ iter 50000 -> checkpoints/500m_latest.pt iter 50020 | train 2.868 | val 2.895 | lr 4.74e-04 | 102,409 tok/s | 54.64 GB | 6.556B tok iter 50040 | train 2.964 | val 2.875 | lr 4.74e-04 | 102,412 tok/s | 54.64 GB | 6.559B tok iter 50060 | train 2.805 | val 2.874 | lr 4.74e-04 | 102,416 tok/s | 54.64 GB | 6.562B tok iter 50080 | train 2.796 | val 2.894 | lr 4.74e-04 | 102,419 tok/s | 54.64 GB | 6.564B tok iter 50100 | train 2.788 | val 2.883 | lr 4.74e-04 | 102,419 tok/s | 54.64 GB | 6.567B tok iter 50120 | train 2.940 | val 2.816 | lr 4.73e-04 | 102,423 tok/s | 54.64 GB | 6.569B tok iter 50140 | train 2.798 | val 2.830 | lr 4.73e-04 | 102,426 tok/s | 54.64 GB | 6.572B tok iter 50160 | train 2.818 | val 2.810 | lr 4.73e-04 | 102,429 tok/s | 54.64 GB | 6.575B tok iter 50180 | train 2.824 | val 2.861 | lr 4.73e-04 | 102,430 tok/s | 54.64 GB | 6.577B tok iter 50200 | train 2.934 | val 2.882 | lr 4.73e-04 | 102,433 tok/s | 54.64 GB | 6.580B tok iter 50220 | train 2.764 | val 2.862 | lr 4.73e-04 | 102,436 tok/s | 54.64 GB | 6.583B tok iter 50240 | train 2.712 | val 2.899 | lr 4.73e-04 | 102,439 tok/s | 54.64 GB | 6.585B tok iter 50260 | train 2.839 | val 2.887 | lr 4.73e-04 | 102,440 tok/s | 54.64 GB | 6.588B tok iter 50280 | train 2.815 | val 2.882 | lr 4.73e-04 | 102,443 tok/s | 54.64 GB | 6.590B tok iter 50300 | train 2.916 | val 2.886 | lr 4.73e-04 | 102,446 tok/s | 54.64 GB | 6.593B tok iter 50320 | train 2.948 | val 2.873 | lr 4.73e-04 | 102,450 tok/s | 54.64 GB | 6.596B tok iter 50340 | train 2.825 | val 2.869 | lr 4.72e-04 | 102,450 tok/s | 54.64 GB | 6.598B tok iter 50360 | train 2.915 | val 2.844 | lr 4.72e-04 | 102,453 tok/s | 54.64 GB | 6.601B tok iter 50380 | train 2.760 | val 2.915 | lr 4.72e-04 | 102,456 tok/s | 54.64 GB | 6.604B tok iter 50400 | train 2.819 | val 2.869 | lr 4.72e-04 | 102,460 tok/s | 54.64 GB | 6.606B tok iter 50420 | train 2.947 | val 2.830 | lr 4.72e-04 | 102,460 tok/s | 54.64 GB | 6.609B tok iter 50440 | train 2.794 | val 2.850 | lr 4.72e-04 | 102,463 tok/s | 54.64 GB | 6.611B tok iter 50460 | train 2.752 | val 2.913 | lr 4.72e-04 | 102,467 tok/s | 54.64 GB | 6.614B tok iter 50480 | train 2.714 | val 2.877 | lr 4.72e-04 | 102,470 tok/s | 54.64 GB | 6.617B tok checkpoint @ iter 50490 -> checkpoints/500m_latest.pt iter 50500 | train 2.723 | val 2.903 | lr 4.72e-04 | 102,449 tok/s | 54.64 GB | 6.619B tok checkpoint @ iter 50500 -> checkpoints/500m_latest.pt iter 50520 | train 2.936 | val 2.869 | lr 4.72e-04 | 102,410 tok/s | 54.64 GB | 6.622B tok iter 50540 | train 2.874 | val 2.852 | lr 4.71e-04 | 102,413 tok/s | 54.64 GB | 6.625B tok iter 50560 | train 2.815 | val 2.891 | lr 4.71e-04 | 102,417 tok/s | 54.64 GB | 6.627B tok iter 50580 | train 2.870 | val 2.876 | lr 4.71e-04 | 102,417 tok/s | 54.64 GB | 6.630B tok iter 50600 | train 2.754 | val 2.839 | lr 4.71e-04 | 102,420 tok/s | 54.64 GB | 6.632B tok iter 50620 | train 2.910 | val 2.871 | lr 4.71e-04 | 102,424 tok/s | 54.64 GB | 6.635B tok iter 50640 | train 2.817 | val 2.864 | lr 4.71e-04 | 102,427 tok/s | 54.64 GB | 6.638B tok iter 50660 | train 2.828 | val 2.858 | lr 4.71e-04 | 102,427 tok/s | 54.64 GB | 6.640B tok iter 50680 | train 2.955 | val 2.889 | lr 4.71e-04 | 102,430 tok/s | 54.64 GB | 6.643B tok iter 50700 | train 2.940 | val 2.838 | lr 4.71e-04 | 102,434 tok/s | 54.64 GB | 6.645B tok iter 50720 | train 2.733 | val 2.827 | lr 4.71e-04 | 102,437 tok/s | 54.64 GB | 6.648B tok iter 50740 | train 2.891 | val 2.840 | lr 4.71e-04 | 102,437 tok/s | 54.64 GB | 6.651B tok iter 50760 | train 2.920 | val 2.843 | lr 4.70e-04 | 102,441 tok/s | 54.64 GB | 6.653B tok iter 50780 | train 3.000 | val 2.816 | lr 4.70e-04 | 102,444 tok/s | 54.64 GB | 6.656B tok iter 50800 | train 2.911 | val 2.849 | lr 4.70e-04 | 102,447 tok/s | 54.64 GB | 6.659B tok iter 50820 | train 2.689 | val 2.907 | lr 4.70e-04 | 102,447 tok/s | 54.64 GB | 6.661B tok iter 50840 | train 2.887 | val 2.872 | lr 4.70e-04 | 102,450 tok/s | 54.64 GB | 6.664B tok iter 50860 | train 2.854 | val 2.800 | lr 4.70e-04 | 102,454 tok/s | 54.64 GB | 6.666B tok iter 50880 | train 2.903 | val 2.914 | lr 4.70e-04 | 102,457 tok/s | 54.64 GB | 6.669B tok iter 50900 | train 2.763 | val 2.911 | lr 4.70e-04 | 102,457 tok/s | 54.64 GB | 6.672B tok iter 50920 | train 2.824 | val 2.908 | lr 4.70e-04 | 102,460 tok/s | 54.64 GB | 6.674B tok iter 50940 | train 2.897 | val 2.831 | lr 4.70e-04 | 102,463 tok/s | 54.64 GB | 6.677B tok iter 50960 | train 2.770 | val 2.855 | lr 4.69e-04 | 102,467 tok/s | 54.64 GB | 6.680B tok iter 50980 | train 2.843 | val 2.823 | lr 4.69e-04 | 102,467 tok/s | 54.64 GB | 6.682B tok checkpoint @ iter 50990 -> checkpoints/500m_latest.pt iter 51000 | train 2.858 | val 2.801 | lr 4.69e-04 | 102,450 tok/s | 54.64 GB | 6.685B tok checkpoint @ iter 51000 -> checkpoints/500m_latest.pt iter 51020 | train 2.844 | val 2.827 | lr 4.69e-04 | 102,411 tok/s | 54.64 GB | 6.687B tok iter 51040 | train 2.860 | val 2.870 | lr 4.69e-04 | 102,414 tok/s | 54.64 GB | 6.690B tok iter 51060 | train 2.829 | val 2.814 | lr 4.69e-04 | 102,415 tok/s | 54.64 GB | 6.693B tok iter 51080 | train 2.687 | val 2.868 | lr 4.69e-04 | 102,418 tok/s | 54.64 GB | 6.695B tok iter 51100 | train 2.732 | val 2.846 | lr 4.69e-04 | 102,421 tok/s | 54.64 GB | 6.698B tok iter 51120 | train 2.941 | val 2.870 | lr 4.69e-04 | 102,425 tok/s | 54.64 GB | 6.701B tok iter 51140 | train 2.779 | val 2.855 | lr 4.69e-04 | 102,425 tok/s | 54.64 GB | 6.703B tok iter 51160 | train 2.845 | val 2.883 | lr 4.68e-04 | 102,428 tok/s | 54.64 GB | 6.706B tok iter 51180 | train 2.836 | val 2.893 | lr 4.68e-04 | 102,431 tok/s | 54.64 GB | 6.708B tok iter 51200 | train 2.815 | val 2.845 | lr 4.68e-04 | 102,435 tok/s | 54.64 GB | 6.711B tok iter 51220 | train 2.963 | val 2.799 | lr 4.68e-04 | 102,435 tok/s | 54.64 GB | 6.714B tok iter 51240 | train 2.857 | val 2.897 | lr 4.68e-04 | 102,438 tok/s | 54.64 GB | 6.716B tok iter 51260 | train 2.756 | val 2.867 | lr 4.68e-04 | 102,441 tok/s | 54.64 GB | 6.719B tok iter 51280 | train 2.927 | val 2.881 | lr 4.68e-04 | 102,445 tok/s | 54.64 GB | 6.722B tok iter 51300 | train 2.811 | val 2.858 | lr 4.68e-04 | 102,445 tok/s | 54.64 GB | 6.724B tok iter 51320 | train 3.060 | val 2.879 | lr 4.68e-04 | 102,448 tok/s | 54.64 GB | 6.727B tok iter 51340 | train 2.842 | val 2.854 | lr 4.68e-04 | 102,451 tok/s | 54.64 GB | 6.729B tok iter 51360 | train 2.907 | val 2.895 | lr 4.68e-04 | 102,454 tok/s | 54.64 GB | 6.732B tok iter 51380 | train 2.790 | val 2.888 | lr 4.67e-04 | 102,455 tok/s | 54.64 GB | 6.735B tok iter 51400 | train 2.807 | val 2.820 | lr 4.67e-04 | 102,458 tok/s | 54.64 GB | 6.737B tok iter 51420 | train 2.840 | val 2.834 | lr 4.67e-04 | 102,461 tok/s | 54.64 GB | 6.740B tok iter 51440 | train 2.835 | val 2.854 | lr 4.67e-04 | 102,464 tok/s | 54.64 GB | 6.742B tok iter 51460 | train 2.883 | val 2.864 | lr 4.67e-04 | 102,465 tok/s | 54.64 GB | 6.745B tok iter 51480 | train 2.914 | val 2.855 | lr 4.67e-04 | 102,468 tok/s | 54.64 GB | 6.748B tok checkpoint @ iter 51490 -> checkpoints/500m_latest.pt iter 51500 | train 2.972 | val 2.839 | lr 4.67e-04 | 102,451 tok/s | 54.64 GB | 6.750B tok checkpoint @ iter 51500 -> checkpoints/500m_latest.pt iter 51520 | train 2.784 | val 2.861 | lr 4.67e-04 | 102,414 tok/s | 54.64 GB | 6.753B tok iter 51540 | train 2.922 | val 2.849 | lr 4.67e-04 | 102,414 tok/s | 54.64 GB | 6.756B tok iter 51560 | train 2.799 | val 2.901 | lr 4.67e-04 | 102,417 tok/s | 54.64 GB | 6.758B tok iter 51580 | train 2.944 | val 2.873 | lr 4.66e-04 | 102,421 tok/s | 54.64 GB | 6.761B tok iter 51600 | train 2.885 | val 2.793 | lr 4.66e-04 | 102,424 tok/s | 54.64 GB | 6.763B tok iter 51620 | train 2.842 | val 2.836 | lr 4.66e-04 | 102,424 tok/s | 54.64 GB | 6.766B tok iter 51640 | train 2.854 | val 2.845 | lr 4.66e-04 | 102,427 tok/s | 54.64 GB | 6.769B tok iter 51660 | train 2.801 | val 2.828 | lr 4.66e-04 | 102,430 tok/s | 54.64 GB | 6.771B tok iter 51680 | train 2.751 | val 2.865 | lr 4.66e-04 | 102,433 tok/s | 54.64 GB | 6.774B tok iter 51700 | train 2.827 | val 2.894 | lr 4.66e-04 | 102,434 tok/s | 54.64 GB | 6.777B tok iter 51720 | train 2.831 | val 2.859 | lr 4.66e-04 | 102,437 tok/s | 54.64 GB | 6.779B tok iter 51740 | train 2.831 | val 2.787 | lr 4.66e-04 | 102,440 tok/s | 54.64 GB | 6.782B tok iter 51760 | train 2.788 | val 2.888 | lr 4.66e-04 | 102,443 tok/s | 54.64 GB | 6.784B tok iter 51780 | train 2.845 | val 2.803 | lr 4.65e-04 | 102,444 tok/s | 54.64 GB | 6.787B tok iter 51800 | train 2.945 | val 2.881 | lr 4.65e-04 | 102,447 tok/s | 54.64 GB | 6.790B tok iter 51820 | train 2.733 | val 2.903 | lr 4.65e-04 | 102,450 tok/s | 54.64 GB | 6.792B tok iter 51840 | train 2.794 | val 2.878 | lr 4.65e-04 | 102,453 tok/s | 54.64 GB | 6.795B tok iter 51860 | train 2.624 | val 2.869 | lr 4.65e-04 | 102,454 tok/s | 54.64 GB | 6.798B tok iter 51880 | train 2.867 | val 2.816 | lr 4.65e-04 | 102,457 tok/s | 54.64 GB | 6.800B tok iter 51900 | train 2.832 | val 2.912 | lr 4.65e-04 | 102,460 tok/s | 54.64 GB | 6.803B tok iter 51920 | train 2.846 | val 2.878 | lr 4.65e-04 | 102,463 tok/s | 54.64 GB | 6.805B tok iter 51940 | train 2.812 | val 2.852 | lr 4.65e-04 | 102,464 tok/s | 54.64 GB | 6.808B tok iter 51960 | train 2.929 | val 2.862 | lr 4.65e-04 | 102,467 tok/s | 54.64 GB | 6.811B tok iter 51980 | train 2.845 | val 2.880 | lr 4.65e-04 | 102,470 tok/s | 54.64 GB | 6.813B tok checkpoint @ iter 51990 -> checkpoints/500m_latest.pt iter 52000 | train 3.014 | val 2.860 | lr 4.64e-04 | 102,452 tok/s | 54.64 GB | 6.816B tok checkpoint @ iter 52000 -> checkpoints/500m_latest.pt iter 52020 | train 2.938 | val 2.881 | lr 4.64e-04 | 102,412 tok/s | 54.64 GB | 6.818B tok iter 52040 | train 2.805 | val 2.891 | lr 4.64e-04 | 102,415 tok/s | 54.64 GB | 6.821B tok iter 52060 | train 2.772 | val 2.845 | lr 4.64e-04 | 102,418 tok/s | 54.64 GB | 6.824B tok iter 52080 | train 2.743 | val 2.845 | lr 4.64e-04 | 102,421 tok/s | 54.64 GB | 6.826B tok iter 52100 | train 2.843 | val 2.856 | lr 4.64e-04 | 102,422 tok/s | 54.64 GB | 6.829B tok iter 52120 | train 2.745 | val 2.868 | lr 4.64e-04 | 102,425 tok/s | 54.64 GB | 6.832B tok iter 52140 | train 2.734 | val 2.838 | lr 4.64e-04 | 102,428 tok/s | 54.64 GB | 6.834B tok iter 52160 | train 2.810 | val 2.816 | lr 4.64e-04 | 102,431 tok/s | 54.64 GB | 6.837B tok iter 52180 | train 2.802 | val 2.833 | lr 4.64e-04 | 102,431 tok/s | 54.64 GB | 6.839B tok iter 52200 | train 2.900 | val 2.819 | lr 4.63e-04 | 102,434 tok/s | 54.64 GB | 6.842B tok iter 52220 | train 2.791 | val 2.857 | lr 4.63e-04 | 102,438 tok/s | 54.64 GB | 6.845B tok iter 52240 | train 2.842 | val 2.881 | lr 4.63e-04 | 102,441 tok/s | 54.64 GB | 6.847B tok iter 52260 | train 2.874 | val 2.879 | lr 4.63e-04 | 102,441 tok/s | 54.64 GB | 6.850B tok iter 52280 | train 2.814 | val 2.853 | lr 4.63e-04 | 102,444 tok/s | 54.64 GB | 6.853B tok iter 52300 | train 2.829 | val 2.868 | lr 4.63e-04 | 102,447 tok/s | 54.64 GB | 6.855B tok iter 52320 | train 2.932 | val 2.838 | lr 4.63e-04 | 102,450 tok/s | 54.64 GB | 6.858B tok iter 52340 | train 2.928 | val 2.824 | lr 4.63e-04 | 102,451 tok/s | 54.64 GB | 6.860B tok iter 52360 | train 2.843 | val 2.825 | lr 4.63e-04 | 102,454 tok/s | 54.64 GB | 6.863B tok iter 52380 | train 2.842 | val 2.820 | lr 4.63e-04 | 102,457 tok/s | 54.64 GB | 6.866B tok iter 52400 | train 2.862 | val 2.862 | lr 4.62e-04 | 102,460 tok/s | 54.64 GB | 6.868B tok iter 52420 | train 2.758 | val 2.852 | lr 4.62e-04 | 102,460 tok/s | 54.64 GB | 6.871B tok iter 52440 | train 2.893 | val 2.815 | lr 4.62e-04 | 102,464 tok/s | 54.64 GB | 6.874B tok iter 52460 | train 2.825 | val 2.838 | lr 4.62e-04 | 102,467 tok/s | 54.64 GB | 6.876B tok iter 52480 | train 2.746 | val 2.827 | lr 4.62e-04 | 102,470 tok/s | 54.64 GB | 6.879B tok checkpoint @ iter 52490 -> checkpoints/500m_latest.pt iter 52500 | train 2.987 | val 2.845 | lr 4.62e-04 | 102,450 tok/s | 54.64 GB | 6.881B tok checkpoint @ iter 52500 -> checkpoints/500m_latest.pt iter 52520 | train 2.962 | val 2.825 | lr 4.62e-04 | 102,411 tok/s | 54.64 GB | 6.884B tok iter 52540 | train 2.789 | val 2.896 | lr 4.62e-04 | 102,415 tok/s | 54.64 GB | 6.887B tok iter 52560 | train 2.802 | val 2.804 | lr 4.62e-04 | 102,418 tok/s | 54.64 GB | 6.889B tok iter 52580 | train 2.839 | val 2.889 | lr 4.62e-04 | 102,418 tok/s | 54.64 GB | 6.892B tok iter 52600 | train 2.757 | val 2.880 | lr 4.61e-04 | 102,421 tok/s | 54.64 GB | 6.895B tok iter 52620 | train 2.830 | val 2.894 | lr 4.61e-04 | 102,424 tok/s | 54.64 GB | 6.897B tok iter 52640 | train 2.752 | val 2.800 | lr 4.61e-04 | 102,427 tok/s | 54.64 GB | 6.900B tok iter 52660 | train 2.852 | val 2.868 | lr 4.61e-04 | 102,428 tok/s | 54.64 GB | 6.902B tok iter 52680 | train 2.930 | val 2.827 | lr 4.61e-04 | 102,431 tok/s | 54.64 GB | 6.905B tok iter 52700 | train 2.831 | val 2.901 | lr 4.61e-04 | 102,434 tok/s | 54.64 GB | 6.908B tok iter 52720 | train 2.862 | val 2.844 | lr 4.61e-04 | 102,437 tok/s | 54.64 GB | 6.910B tok iter 52740 | train 2.793 | val 2.858 | lr 4.61e-04 | 102,437 tok/s | 54.64 GB | 6.913B tok iter 52760 | train 2.780 | val 2.846 | lr 4.61e-04 | 102,440 tok/s | 54.64 GB | 6.915B tok iter 52780 | train 2.796 | val 2.845 | lr 4.61e-04 | 102,444 tok/s | 54.64 GB | 6.918B tok iter 52800 | train 2.794 | val 2.861 | lr 4.61e-04 | 102,447 tok/s | 54.64 GB | 6.921B tok iter 52820 | train 2.932 | val 2.859 | lr 4.60e-04 | 102,447 tok/s | 54.64 GB | 6.923B tok iter 52840 | train 2.844 | val 2.887 | lr 4.60e-04 | 102,450 tok/s | 54.64 GB | 6.926B tok iter 52860 | train 2.582 | val 2.840 | lr 4.60e-04 | 102,453 tok/s | 54.64 GB | 6.929B tok iter 52880 | train 2.873 | val 2.836 | lr 4.60e-04 | 102,456 tok/s | 54.64 GB | 6.931B tok iter 52900 | train 2.867 | val 2.861 | lr 4.60e-04 | 102,457 tok/s | 54.64 GB | 6.934B tok iter 52920 | train 2.907 | val 2.862 | lr 4.60e-04 | 102,460 tok/s | 54.64 GB | 6.936B tok iter 52940 | train 2.854 | val 2.869 | lr 4.60e-04 | 102,463 tok/s | 54.64 GB | 6.939B tok iter 52960 | train 2.723 | val 2.839 | lr 4.60e-04 | 102,466 tok/s | 54.64 GB | 6.942B tok iter 52980 | train 2.885 | val 2.851 | lr 4.60e-04 | 102,466 tok/s | 54.64 GB | 6.944B tok checkpoint @ iter 52990 -> checkpoints/500m_latest.pt iter 53000 | train 2.820 | val 2.886 | lr 4.60e-04 | 102,449 tok/s | 54.64 GB | 6.947B tok checkpoint @ iter 53000 -> checkpoints/500m_latest.pt iter 53020 | train 2.921 | val 2.891 | lr 4.59e-04 | 102,412 tok/s | 54.64 GB | 6.950B tok iter 53040 | train 2.938 | val 2.840 | lr 4.59e-04 | 102,415 tok/s | 54.64 GB | 6.952B tok iter 53060 | train 2.827 | val 2.879 | lr 4.59e-04 | 102,416 tok/s | 54.64 GB | 6.955B tok iter 53080 | train 2.982 | val 2.864 | lr 4.59e-04 | 102,419 tok/s | 54.64 GB | 6.957B tok iter 53100 | train 2.835 | val 2.822 | lr 4.59e-04 | 102,422 tok/s | 54.64 GB | 6.960B tok iter 53120 | train 2.916 | val 2.818 | lr 4.59e-04 | 102,425 tok/s | 54.64 GB | 6.963B tok iter 53140 | train 2.816 | val 2.858 | lr 4.59e-04 | 102,425 tok/s | 54.64 GB | 6.965B tok iter 53160 | train 2.782 | val 2.835 | lr 4.59e-04 | 102,428 tok/s | 54.64 GB | 6.968B tok iter 53180 | train 2.922 | val 2.840 | lr 4.59e-04 | 102,431 tok/s | 54.64 GB | 6.971B tok iter 53200 | train 2.908 | val 2.821 | lr 4.59e-04 | 102,434 tok/s | 54.64 GB | 6.973B tok iter 53220 | train 2.882 | val 2.887 | lr 4.58e-04 | 102,435 tok/s | 54.64 GB | 6.976B tok iter 53240 | train 2.987 | val 2.915 | lr 4.58e-04 | 102,438 tok/s | 54.64 GB | 6.978B tok iter 53260 | train 2.813 | val 2.896 | lr 4.58e-04 | 102,441 tok/s | 54.64 GB | 6.981B tok iter 53280 | train 2.807 | val 2.823 | lr 4.58e-04 | 102,444 tok/s | 54.64 GB | 6.984B tok iter 53300 | train 2.970 | val 2.809 | lr 4.58e-04 | 102,444 tok/s | 54.64 GB | 6.986B tok iter 53320 | train 2.834 | val 2.806 | lr 4.58e-04 | 102,447 tok/s | 54.64 GB | 6.989B tok iter 53340 | train 2.850 | val 2.855 | lr 4.58e-04 | 102,450 tok/s | 54.64 GB | 6.992B tok iter 53360 | train 2.838 | val 2.864 | lr 4.58e-04 | 102,453 tok/s | 54.64 GB | 6.994B tok iter 53380 | train 2.972 | val 2.861 | lr 4.58e-04 | 102,453 tok/s | 54.64 GB | 6.997B tok iter 53400 | train 2.917 | val 2.879 | lr 4.58e-04 | 102,456 tok/s | 54.64 GB | 6.999B tok iter 53420 | train 2.882 | val 2.844 | lr 4.57e-04 | 102,459 tok/s | 54.64 GB | 7.002B tok iter 53440 | train 2.887 | val 2.862 | lr 4.57e-04 | 102,462 tok/s | 54.64 GB | 7.005B tok iter 53460 | train 2.962 | val 2.865 | lr 4.57e-04 | 102,463 tok/s | 54.64 GB | 7.007B tok iter 53480 | train 2.834 | val 2.808 | lr 4.57e-04 | 102,466 tok/s | 54.64 GB | 7.010B tok checkpoint @ iter 53490 -> checkpoints/500m_latest.pt iter 53500 | train 2.721 | val 2.867 | lr 4.57e-04 | 102,449 tok/s | 54.64 GB | 7.012B tok checkpoint @ iter 53500 -> checkpoints/500m_latest.pt iter 53520 | train 2.961 | val 2.905 | lr 4.57e-04 | 102,412 tok/s | 54.64 GB | 7.015B tok iter 53540 | train 2.842 | val 2.836 | lr 4.57e-04 | 102,413 tok/s | 54.64 GB | 7.018B tok iter 53560 | train 2.854 | val 2.850 | lr 4.57e-04 | 102,416 tok/s | 54.64 GB | 7.020B tok iter 53580 | train 2.726 | val 2.821 | lr 4.57e-04 | 102,419 tok/s | 54.64 GB | 7.023B tok iter 53600 | train 2.861 | val 2.871 | lr 4.57e-04 | 102,422 tok/s | 54.64 GB | 7.026B tok iter 53620 | train 2.830 | val 2.803 | lr 4.56e-04 | 102,422 tok/s | 54.64 GB | 7.028B tok iter 53640 | train 2.800 | val 2.830 | lr 4.56e-04 | 102,425 tok/s | 54.64 GB | 7.031B tok iter 53660 | train 3.036 | val 2.813 | lr 4.56e-04 | 102,428 tok/s | 54.64 GB | 7.033B tok iter 53680 | train 2.652 | val 2.815 | lr 4.56e-04 | 102,431 tok/s | 54.64 GB | 7.036B tok iter 53700 | train 2.872 | val 2.855 | lr 4.56e-04 | 102,432 tok/s | 54.64 GB | 7.039B tok iter 53720 | train 2.848 | val 2.803 | lr 4.56e-04 | 102,435 tok/s | 54.64 GB | 7.041B tok iter 53740 | train 2.834 | val 2.824 | lr 4.56e-04 | 102,438 tok/s | 54.64 GB | 7.044B tok iter 53760 | train 2.891 | val 2.887 | lr 4.56e-04 | 102,441 tok/s | 54.64 GB | 7.047B tok iter 53780 | train 2.752 | val 2.833 | lr 4.56e-04 | 102,441 tok/s | 54.64 GB | 7.049B tok iter 53800 | train 2.845 | val 2.870 | lr 4.56e-04 | 102,444 tok/s | 54.64 GB | 7.052B tok iter 53820 | train 2.751 | val 2.824 | lr 4.55e-04 | 102,447 tok/s | 54.64 GB | 7.054B tok iter 53840 | train 2.853 | val 2.829 | lr 4.55e-04 | 102,450 tok/s | 54.64 GB | 7.057B tok iter 53860 | train 2.888 | val 2.845 | lr 4.55e-04 | 102,451 tok/s | 54.64 GB | 7.060B tok iter 53880 | train 2.755 | val 2.852 | lr 4.55e-04 | 102,454 tok/s | 54.64 GB | 7.062B tok iter 53900 | train 2.921 | val 2.858 | lr 4.55e-04 | 102,457 tok/s | 54.64 GB | 7.065B tok iter 53920 | train 2.858 | val 2.857 | lr 4.55e-04 | 102,460 tok/s | 54.64 GB | 7.068B tok iter 53940 | train 2.841 | val 2.879 | lr 4.55e-04 | 102,460 tok/s | 54.64 GB | 7.070B tok iter 53960 | train 2.822 | val 2.833 | lr 4.55e-04 | 102,463 tok/s | 54.64 GB | 7.073B tok iter 53980 | train 2.958 | val 2.896 | lr 4.55e-04 | 102,466 tok/s | 54.64 GB | 7.075B tok checkpoint @ iter 53990 -> checkpoints/500m_latest.pt iter 54000 | train 2.794 | val 2.889 | lr 4.55e-04 | 102,450 tok/s | 54.64 GB | 7.078B tok checkpoint @ iter 54000 -> checkpoints/500m_latest.pt iter 54020 | train 2.814 | val 2.876 | lr 4.54e-04 | 102,412 tok/s | 54.64 GB | 7.081B tok iter 54040 | train 2.845 | val 2.844 | lr 4.54e-04 | 102,415 tok/s | 54.64 GB | 7.083B tok iter 54060 | train 2.761 | val 2.879 | lr 4.54e-04 | 102,418 tok/s | 54.64 GB | 7.086B tok iter 54080 | train 2.689 | val 2.866 | lr 4.54e-04 | 102,421 tok/s | 54.64 GB | 7.089B tok iter 54100 | train 2.908 | val 2.916 | lr 4.54e-04 | 102,422 tok/s | 54.64 GB | 7.091B tok iter 54120 | train 2.884 | val 2.878 | lr 4.54e-04 | 102,425 tok/s | 54.64 GB | 7.094B tok iter 54140 | train 2.919 | val 2.842 | lr 4.54e-04 | 102,428 tok/s | 54.64 GB | 7.096B tok iter 54160 | train 2.835 | val 2.872 | lr 4.54e-04 | 102,431 tok/s | 54.64 GB | 7.099B tok iter 54180 | train 2.913 | val 2.889 | lr 4.54e-04 | 102,431 tok/s | 54.64 GB | 7.102B tok iter 54200 | train 2.719 | val 2.887 | lr 4.54e-04 | 102,434 tok/s | 54.64 GB | 7.104B tok iter 54220 | train 2.992 | val 2.880 | lr 4.53e-04 | 102,437 tok/s | 54.64 GB | 7.107B tok iter 54240 | train 2.776 | val 2.839 | lr 4.53e-04 | 102,440 tok/s | 54.64 GB | 7.109B tok iter 54260 | train 2.808 | val 2.913 | lr 4.53e-04 | 102,440 tok/s | 54.64 GB | 7.112B tok iter 54280 | train 2.881 | val 2.837 | lr 4.53e-04 | 102,443 tok/s | 54.64 GB | 7.115B tok iter 54300 | train 2.888 | val 2.878 | lr 4.53e-04 | 102,446 tok/s | 54.64 GB | 7.117B tok iter 54320 | train 2.821 | val 2.852 | lr 4.53e-04 | 102,449 tok/s | 54.64 GB | 7.120B tok iter 54340 | train 2.832 | val 2.870 | lr 4.53e-04 | 102,450 tok/s | 54.64 GB | 7.123B tok iter 54360 | train 2.825 | val 2.842 | lr 4.53e-04 | 102,453 tok/s | 54.64 GB | 7.125B tok iter 54380 | train 2.805 | val 2.789 | lr 4.53e-04 | 102,456 tok/s | 54.64 GB | 7.128B tok iter 54400 | train 2.835 | val 2.821 | lr 4.53e-04 | 102,459 tok/s | 54.64 GB | 7.130B tok iter 54420 | train 2.849 | val 2.881 | lr 4.52e-04 | 102,459 tok/s | 54.64 GB | 7.133B tok iter 54440 | train 2.916 | val 2.917 | lr 4.52e-04 | 102,462 tok/s | 54.64 GB | 7.136B tok iter 54460 | train 2.948 | val 2.864 | lr 4.52e-04 | 102,465 tok/s | 54.64 GB | 7.138B tok iter 54480 | train 2.792 | val 2.811 | lr 4.52e-04 | 102,468 tok/s | 54.64 GB | 7.141B tok checkpoint @ iter 54491 -> checkpoints/500m_latest.pt iter 54500 | train 2.818 | val 2.861 | lr 4.52e-04 | 102,449 tok/s | 54.64 GB | 7.144B tok checkpoint @ iter 54500 -> checkpoints/500m_latest.pt iter 54520 | train 2.915 | val 2.854 | lr 4.52e-04 | 102,415 tok/s | 54.64 GB | 7.146B tok iter 54540 | train 2.741 | val 2.886 | lr 4.52e-04 | 102,418 tok/s | 54.64 GB | 7.149B tok iter 54560 | train 2.803 | val 2.833 | lr 4.52e-04 | 102,421 tok/s | 54.64 GB | 7.151B tok iter 54580 | train 2.769 | val 2.863 | lr 4.52e-04 | 102,422 tok/s | 54.64 GB | 7.154B tok iter 54600 | train 2.802 | val 2.846 | lr 4.52e-04 | 102,425 tok/s | 54.64 GB | 7.157B tok iter 54620 | train 2.784 | val 2.852 | lr 4.51e-04 | 102,427 tok/s | 54.64 GB | 7.159B tok iter 54640 | train 2.897 | val 2.860 | lr 4.51e-04 | 102,430 tok/s | 54.64 GB | 7.162B tok iter 54660 | train 2.846 | val 2.851 | lr 4.51e-04 | 102,431 tok/s | 54.64 GB | 7.165B tok iter 54680 | train 2.836 | val 2.867 | lr 4.51e-04 | 102,434 tok/s | 54.64 GB | 7.167B tok iter 54700 | train 2.984 | val 2.872 | lr 4.51e-04 | 102,437 tok/s | 54.64 GB | 7.170B tok iter 54720 | train 2.820 | val 2.861 | lr 4.51e-04 | 102,440 tok/s | 54.64 GB | 7.172B tok iter 54740 | train 2.768 | val 2.856 | lr 4.51e-04 | 102,440 tok/s | 54.64 GB | 7.175B tok iter 54760 | train 2.861 | val 2.845 | lr 4.51e-04 | 102,443 tok/s | 54.64 GB | 7.178B tok iter 54780 | train 2.884 | val 2.849 | lr 4.51e-04 | 102,446 tok/s | 54.64 GB | 7.180B tok iter 54800 | train 2.995 | val 2.858 | lr 4.51e-04 | 102,449 tok/s | 54.64 GB | 7.183B tok iter 54820 | train 2.830 | val 2.888 | lr 4.50e-04 | 102,450 tok/s | 54.64 GB | 7.185B tok iter 54840 | train 2.955 | val 2.811 | lr 4.50e-04 | 102,453 tok/s | 54.64 GB | 7.188B tok iter 54860 | train 2.987 | val 2.883 | lr 4.50e-04 | 102,456 tok/s | 54.64 GB | 7.191B tok iter 54880 | train 2.893 | val 2.859 | lr 4.50e-04 | 102,459 tok/s | 54.64 GB | 7.193B tok iter 54900 | train 2.721 | val 2.801 | lr 4.50e-04 | 102,459 tok/s | 54.64 GB | 7.196B tok iter 54920 | train 2.779 | val 2.838 | lr 4.50e-04 | 102,462 tok/s | 54.64 GB | 7.199B tok iter 54940 | train 2.801 | val 2.912 | lr 4.50e-04 | 102,465 tok/s | 54.64 GB | 7.201B tok iter 54960 | train 2.879 | val 2.858 | lr 4.50e-04 | 102,468 tok/s | 54.64 GB | 7.204B tok iter 54980 | train 2.859 | val 2.843 | lr 4.50e-04 | 102,468 tok/s | 54.64 GB | 7.206B tok checkpoint @ iter 54991 -> checkpoints/500m_latest.pt iter 55000 | train 2.867 | val 2.865 | lr 4.50e-04 | 102,452 tok/s | 54.64 GB | 7.209B tok checkpoint @ iter 55000 -> checkpoints/500m_latest.pt iter 55020 | train 2.873 | val 2.840 | lr 4.49e-04 | 102,417 tok/s | 54.64 GB | 7.212B tok iter 55040 | train 2.874 | val 2.860 | lr 4.49e-04 | 102,419 tok/s | 54.64 GB | 7.214B tok iter 55060 | train 2.897 | val 2.847 | lr 4.49e-04 | 102,420 tok/s | 54.64 GB | 7.217B tok iter 55080 | train 2.721 | val 2.841 | lr 4.49e-04 | 102,423 tok/s | 54.64 GB | 7.220B tok iter 55100 | train 2.910 | val 2.828 | lr 4.49e-04 | 102,426 tok/s | 54.64 GB | 7.222B tok iter 55120 | train 2.873 | val 2.837 | lr 4.49e-04 | 102,429 tok/s | 54.64 GB | 7.225B tok iter 55140 | train 2.837 | val 2.843 | lr 4.49e-04 | 102,429 tok/s | 54.64 GB | 7.227B tok iter 55160 | train 2.806 | val 2.831 | lr 4.49e-04 | 102,432 tok/s | 54.64 GB | 7.230B tok iter 55180 | train 2.846 | val 2.811 | lr 4.49e-04 | 102,435 tok/s | 54.64 GB | 7.233B tok iter 55200 | train 2.806 | val 2.862 | lr 4.49e-04 | 102,438 tok/s | 54.64 GB | 7.235B tok iter 55220 | train 2.989 | val 2.837 | lr 4.48e-04 | 102,438 tok/s | 54.64 GB | 7.238B tok iter 55240 | train 2.874 | val 2.851 | lr 4.48e-04 | 102,441 tok/s | 54.64 GB | 7.241B tok iter 55260 | train 2.819 | val 2.885 | lr 4.48e-04 | 102,444 tok/s | 54.64 GB | 7.243B tok iter 55280 | train 2.967 | val 2.843 | lr 4.48e-04 | 102,447 tok/s | 54.64 GB | 7.246B tok iter 55300 | train 2.829 | val 2.858 | lr 4.48e-04 | 102,448 tok/s | 54.64 GB | 7.248B tok iter 55320 | train 2.943 | val 2.909 | lr 4.48e-04 | 102,451 tok/s | 54.64 GB | 7.251B tok iter 55340 | train 2.806 | val 2.862 | lr 4.48e-04 | 102,454 tok/s | 54.64 GB | 7.254B tok iter 55360 | train 2.929 | val 2.825 | lr 4.48e-04 | 102,457 tok/s | 54.64 GB | 7.256B tok iter 55380 | train 2.884 | val 2.867 | lr 4.48e-04 | 102,457 tok/s | 54.64 GB | 7.259B tok iter 55400 | train 2.930 | val 2.840 | lr 4.48e-04 | 102,460 tok/s | 54.64 GB | 7.262B tok iter 55420 | train 2.896 | val 2.867 | lr 4.47e-04 | 102,463 tok/s | 54.64 GB | 7.264B tok iter 55440 | train 2.858 | val 2.845 | lr 4.47e-04 | 102,466 tok/s | 54.64 GB | 7.267B tok iter 55460 | train 2.800 | val 2.874 | lr 4.47e-04 | 102,466 tok/s | 54.64 GB | 7.269B tok iter 55480 | train 2.716 | val 2.864 | lr 4.47e-04 | 102,469 tok/s | 54.64 GB | 7.272B tok checkpoint @ iter 55490 -> checkpoints/500m_latest.pt iter 55500 | train 2.773 | val 2.839 | lr 4.47e-04 | 102,454 tok/s | 54.64 GB | 7.275B tok checkpoint @ iter 55500 -> checkpoints/500m_latest.pt iter 55520 | train 2.686 | val 2.789 | lr 4.47e-04 | 102,420 tok/s | 54.64 GB | 7.277B tok iter 55540 | train 2.777 | val 2.843 | lr 4.47e-04 | 102,421 tok/s | 54.64 GB | 7.280B tok iter 55560 | train 2.887 | val 2.825 | lr 4.47e-04 | 102,424 tok/s | 54.64 GB | 7.282B tok iter 55580 | train 2.933 | val 2.863 | lr 4.47e-04 | 102,427 tok/s | 54.64 GB | 7.285B tok iter 55600 | train 2.857 | val 2.888 | lr 4.47e-04 | 102,429 tok/s | 54.64 GB | 7.288B tok iter 55620 | train 2.862 | val 2.838 | lr 4.46e-04 | 102,430 tok/s | 54.64 GB | 7.290B tok iter 55640 | train 2.953 | val 2.855 | lr 4.46e-04 | 102,433 tok/s | 54.64 GB | 7.293B tok iter 55660 | train 2.702 | val 2.841 | lr 4.46e-04 | 102,436 tok/s | 54.64 GB | 7.296B tok iter 55680 | train 2.835 | val 2.847 | lr 4.46e-04 | 102,439 tok/s | 54.64 GB | 7.298B tok iter 55700 | train 2.831 | val 2.839 | lr 4.46e-04 | 102,439 tok/s | 54.64 GB | 7.301B tok iter 55720 | train 2.911 | val 2.866 | lr 4.46e-04 | 102,442 tok/s | 54.64 GB | 7.303B tok iter 55740 | train 2.890 | val 2.856 | lr 4.46e-04 | 102,445 tok/s | 54.64 GB | 7.306B tok iter 55760 | train 2.950 | val 2.800 | lr 4.46e-04 | 102,448 tok/s | 54.64 GB | 7.309B tok iter 55780 | train 2.813 | val 2.837 | lr 4.46e-04 | 102,448 tok/s | 54.64 GB | 7.311B tok iter 55800 | train 2.753 | val 2.804 | lr 4.46e-04 | 102,451 tok/s | 54.64 GB | 7.314B tok iter 55820 | train 2.750 | val 2.819 | lr 4.45e-04 | 102,454 tok/s | 54.64 GB | 7.317B tok iter 55840 | train 2.773 | val 2.837 | lr 4.45e-04 | 102,457 tok/s | 54.64 GB | 7.319B tok iter 55860 | train 2.687 | val 2.839 | lr 4.45e-04 | 102,457 tok/s | 54.64 GB | 7.322B tok iter 55880 | train 2.894 | val 2.848 | lr 4.45e-04 | 102,460 tok/s | 54.64 GB | 7.324B tok iter 55900 | train 2.769 | val 2.835 | lr 4.45e-04 | 102,463 tok/s | 54.64 GB | 7.327B tok iter 55920 | train 2.794 | val 2.858 | lr 4.45e-04 | 102,466 tok/s | 54.64 GB | 7.330B tok iter 55940 | train 2.888 | val 2.874 | lr 4.45e-04 | 102,466 tok/s | 54.64 GB | 7.332B tok iter 55960 | train 2.839 | val 2.800 | lr 4.45e-04 | 102,469 tok/s | 54.64 GB | 7.335B tok iter 55980 | train 2.862 | val 2.828 | lr 4.45e-04 | 102,472 tok/s | 54.64 GB | 7.338B tok checkpoint @ iter 55991 -> checkpoints/500m_latest.pt iter 56000 | train 2.837 | val 2.861 | lr 4.44e-04 | 102,456 tok/s | 54.64 GB | 7.340B tok checkpoint @ iter 56000 -> checkpoints/500m_latest.pt iter 56020 | train 2.700 | val 2.788 | lr 4.44e-04 | 102,420 tok/s | 54.64 GB | 7.343B tok iter 56040 | train 2.837 | val 2.867 | lr 4.44e-04 | 102,423 tok/s | 54.64 GB | 7.345B tok iter 56060 | train 2.853 | val 2.841 | lr 4.44e-04 | 102,426 tok/s | 54.64 GB | 7.348B tok iter 56080 | train 3.018 | val 2.821 | lr 4.44e-04 | 102,428 tok/s | 54.64 GB | 7.351B tok iter 56100 | train 2.881 | val 2.888 | lr 4.44e-04 | 102,429 tok/s | 54.64 GB | 7.353B tok iter 56120 | train 2.806 | val 2.791 | lr 4.44e-04 | 102,432 tok/s | 54.64 GB | 7.356B tok iter 56140 | train 2.833 | val 2.888 | lr 4.44e-04 | 102,435 tok/s | 54.64 GB | 7.359B tok iter 56160 | train 2.790 | val 2.856 | lr 4.44e-04 | 102,438 tok/s | 54.64 GB | 7.361B tok iter 56180 | train 2.847 | val 2.831 | lr 4.44e-04 | 102,438 tok/s | 54.64 GB | 7.364B tok iter 56200 | train 2.736 | val 2.808 | lr 4.43e-04 | 102,441 tok/s | 54.64 GB | 7.366B tok iter 56220 | train 2.819 | val 2.866 | lr 4.43e-04 | 102,444 tok/s | 54.64 GB | 7.369B tok iter 56240 | train 2.867 | val 2.840 | lr 4.43e-04 | 102,447 tok/s | 54.64 GB | 7.372B tok iter 56260 | train 2.892 | val 2.831 | lr 4.43e-04 | 102,447 tok/s | 54.64 GB | 7.374B tok iter 56280 | train 2.806 | val 2.840 | lr 4.43e-04 | 102,450 tok/s | 54.64 GB | 7.377B tok iter 56300 | train 2.961 | val 2.848 | lr 4.43e-04 | 102,453 tok/s | 54.64 GB | 7.379B tok iter 56320 | train 2.915 | val 2.789 | lr 4.43e-04 | 102,456 tok/s | 54.64 GB | 7.382B tok iter 56340 | train 2.920 | val 2.866 | lr 4.43e-04 | 102,456 tok/s | 54.64 GB | 7.385B tok iter 56360 | train 2.865 | val 2.858 | lr 4.43e-04 | 102,459 tok/s | 54.64 GB | 7.387B tok iter 56380 | train 3.091 | val 2.850 | lr 4.43e-04 | 102,462 tok/s | 54.64 GB | 7.390B tok iter 56400 | train 2.811 | val 2.862 | lr 4.42e-04 | 102,465 tok/s | 54.64 GB | 7.393B tok iter 56420 | train 2.790 | val 2.837 | lr 4.42e-04 | 102,466 tok/s | 54.64 GB | 7.395B tok iter 56440 | train 2.788 | val 2.830 | lr 4.42e-04 | 102,468 tok/s | 54.64 GB | 7.398B tok iter 56460 | train 2.899 | val 2.855 | lr 4.42e-04 | 102,471 tok/s | 54.64 GB | 7.400B tok iter 56480 | train 2.792 | val 2.872 | lr 4.42e-04 | 102,474 tok/s | 54.64 GB | 7.403B tok checkpoint @ iter 56491 -> checkpoints/500m_latest.pt iter 56500 | train 2.945 | val 2.868 | lr 4.42e-04 | 102,456 tok/s | 54.64 GB | 7.406B tok checkpoint @ iter 56500 -> checkpoints/500m_latest.pt iter 56520 | train 2.887 | val 2.850 | lr 4.42e-04 | 102,423 tok/s | 54.64 GB | 7.408B tok iter 56540 | train 2.901 | val 2.836 | lr 4.42e-04 | 102,426 tok/s | 54.64 GB | 7.411B tok iter 56560 | train 2.827 | val 2.856 | lr 4.42e-04 | 102,429 tok/s | 54.64 GB | 7.414B tok iter 56580 | train 2.830 | val 2.841 | lr 4.42e-04 | 102,429 tok/s | 54.64 GB | 7.416B tok iter 56600 | train 2.766 | val 2.847 | lr 4.41e-04 | 102,432 tok/s | 54.64 GB | 7.419B tok iter 56620 | train 2.831 | val 2.856 | lr 4.41e-04 | 102,435 tok/s | 54.64 GB | 7.421B tok iter 56640 | train 2.816 | val 2.841 | lr 4.41e-04 | 102,438 tok/s | 54.64 GB | 7.424B tok iter 56660 | train 2.790 | val 2.819 | lr 4.41e-04 | 102,438 tok/s | 54.64 GB | 7.427B tok iter 56680 | train 2.888 | val 2.855 | lr 4.41e-04 | 102,441 tok/s | 54.64 GB | 7.429B tok iter 56700 | train 2.838 | val 2.863 | lr 4.41e-04 | 102,444 tok/s | 54.64 GB | 7.432B tok iter 56720 | train 2.754 | val 2.843 | lr 4.41e-04 | 102,447 tok/s | 54.64 GB | 7.435B tok iter 56740 | train 2.816 | val 2.821 | lr 4.41e-04 | 102,447 tok/s | 54.64 GB | 7.437B tok iter 56760 | train 3.048 | val 2.810 | lr 4.41e-04 | 102,450 tok/s | 54.64 GB | 7.440B tok iter 56780 | train 2.866 | val 2.809 | lr 4.41e-04 | 102,453 tok/s | 54.64 GB | 7.442B tok iter 56800 | train 2.864 | val 2.818 | lr 4.40e-04 | 102,456 tok/s | 54.64 GB | 7.445B tok iter 56820 | train 2.633 | val 2.814 | lr 4.40e-04 | 102,456 tok/s | 54.64 GB | 7.448B tok iter 56840 | train 2.806 | val 2.817 | lr 4.40e-04 | 102,459 tok/s | 54.64 GB | 7.450B tok iter 56860 | train 2.894 | val 2.842 | lr 4.40e-04 | 102,462 tok/s | 54.64 GB | 7.453B tok iter 56880 | train 2.827 | val 2.834 | lr 4.40e-04 | 102,465 tok/s | 54.64 GB | 7.456B tok iter 56900 | train 2.851 | val 2.898 | lr 4.40e-04 | 102,465 tok/s | 54.64 GB | 7.458B tok iter 56920 | train 2.942 | val 2.839 | lr 4.40e-04 | 102,468 tok/s | 54.64 GB | 7.461B tok iter 56940 | train 2.741 | val 2.838 | lr 4.40e-04 | 102,471 tok/s | 54.64 GB | 7.463B tok iter 56960 | train 2.891 | val 2.843 | lr 4.40e-04 | 102,474 tok/s | 54.64 GB | 7.466B tok iter 56980 | train 2.899 | val 2.810 | lr 4.39e-04 | 102,474 tok/s | 54.64 GB | 7.469B tok checkpoint @ iter 56991 -> checkpoints/500m_latest.pt iter 57000 | train 2.873 | val 2.785 | lr 4.39e-04 | 102,460 tok/s | 54.64 GB | 7.471B tok checkpoint @ iter 57000 -> checkpoints/500m_latest.pt iter 57020 | train 2.822 | val 2.872 | lr 4.39e-04 | 102,427 tok/s | 54.64 GB | 7.474B tok iter 57040 | train 2.876 | val 2.828 | lr 4.39e-04 | 102,430 tok/s | 54.64 GB | 7.476B tok iter 57060 | train 2.742 | val 2.820 | lr 4.39e-04 | 102,431 tok/s | 54.64 GB | 7.479B tok iter 57080 | train 2.736 | val 2.798 | lr 4.39e-04 | 102,433 tok/s | 54.64 GB | 7.482B tok iter 57100 | train 2.909 | val 2.841 | lr 4.39e-04 | 102,436 tok/s | 54.64 GB | 7.484B tok iter 57120 | train 2.868 | val 2.817 | lr 4.39e-04 | 102,439 tok/s | 54.64 GB | 7.487B tok iter 57140 | train 2.829 | val 2.853 | lr 4.39e-04 | 102,440 tok/s | 54.64 GB | 7.490B tok iter 57160 | train 2.767 | val 2.849 | lr 4.39e-04 | 102,442 tok/s | 54.64 GB | 7.492B tok iter 57180 | train 2.795 | val 2.905 | lr 4.38e-04 | 102,445 tok/s | 54.64 GB | 7.495B tok iter 57200 | train 3.009 | val 2.867 | lr 4.38e-04 | 102,448 tok/s | 54.64 GB | 7.497B tok iter 57220 | train 2.814 | val 2.864 | lr 4.38e-04 | 102,449 tok/s | 54.64 GB | 7.500B tok iter 57240 | train 2.790 | val 2.819 | lr 4.38e-04 | 102,451 tok/s | 54.64 GB | 7.503B tok iter 57260 | train 2.756 | val 2.867 | lr 4.38e-04 | 102,454 tok/s | 54.64 GB | 7.505B tok iter 57280 | train 2.725 | val 2.805 | lr 4.38e-04 | 102,457 tok/s | 54.64 GB | 7.508B tok iter 57300 | train 2.801 | val 2.844 | lr 4.38e-04 | 102,458 tok/s | 54.64 GB | 7.511B tok iter 57320 | train 3.043 | val 2.903 | lr 4.38e-04 | 102,460 tok/s | 54.64 GB | 7.513B tok iter 57340 | train 2.931 | val 2.829 | lr 4.38e-04 | 102,463 tok/s | 54.64 GB | 7.516B tok iter 57360 | train 2.856 | val 2.890 | lr 4.38e-04 | 102,466 tok/s | 54.64 GB | 7.518B tok iter 57380 | train 3.008 | val 2.890 | lr 4.37e-04 | 102,466 tok/s | 54.64 GB | 7.521B tok iter 57400 | train 2.758 | val 2.859 | lr 4.37e-04 | 102,469 tok/s | 54.64 GB | 7.524B tok iter 57420 | train 2.802 | val 2.860 | lr 4.37e-04 | 102,472 tok/s | 54.64 GB | 7.526B tok iter 57440 | train 2.840 | val 2.866 | lr 4.37e-04 | 102,475 tok/s | 54.64 GB | 7.529B tok iter 57460 | train 3.005 | val 2.831 | lr 4.37e-04 | 102,475 tok/s | 54.64 GB | 7.532B tok iter 57480 | train 2.770 | val 2.855 | lr 4.37e-04 | 102,478 tok/s | 54.64 GB | 7.534B tok checkpoint @ iter 57491 -> checkpoints/500m_latest.pt iter 57500 | train 2.848 | val 2.836 | lr 4.37e-04 | 102,463 tok/s | 54.64 GB | 7.537B tok checkpoint @ iter 57500 -> checkpoints/500m_latest.pt iter 57520 | train 2.701 | val 2.821 | lr 4.37e-04 | 102,431 tok/s | 54.64 GB | 7.539B tok iter 57540 | train 2.759 | val 2.889 | lr 4.37e-04 | 102,431 tok/s | 54.64 GB | 7.542B tok iter 57560 | train 2.877 | val 2.863 | lr 4.37e-04 | 102,434 tok/s | 54.64 GB | 7.545B tok iter 57580 | train 2.949 | val 2.868 | lr 4.36e-04 | 102,437 tok/s | 54.64 GB | 7.547B tok iter 57600 | train 2.974 | val 2.814 | lr 4.36e-04 | 102,439 tok/s | 54.64 GB | 7.550B tok iter 57620 | train 2.903 | val 2.828 | lr 4.36e-04 | 102,440 tok/s | 54.64 GB | 7.552B tok iter 57640 | train 2.832 | val 2.840 | lr 4.36e-04 | 102,443 tok/s | 54.64 GB | 7.555B tok iter 57660 | train 2.853 | val 2.858 | lr 4.36e-04 | 102,445 tok/s | 54.64 GB | 7.558B tok iter 57680 | train 2.783 | val 2.806 | lr 4.36e-04 | 102,448 tok/s | 54.64 GB | 7.560B tok iter 57700 | train 2.815 | val 2.856 | lr 4.36e-04 | 102,449 tok/s | 54.64 GB | 7.563B tok iter 57720 | train 2.917 | val 2.884 | lr 4.36e-04 | 102,451 tok/s | 54.64 GB | 7.566B tok iter 57740 | train 2.756 | val 2.889 | lr 4.36e-04 | 102,454 tok/s | 54.64 GB | 7.568B tok iter 57760 | train 2.823 | val 2.849 | lr 4.35e-04 | 102,457 tok/s | 54.64 GB | 7.571B tok iter 57780 | train 2.765 | val 2.881 | lr 4.35e-04 | 102,457 tok/s | 54.64 GB | 7.573B tok iter 57800 | train 2.961 | val 2.818 | lr 4.35e-04 | 102,460 tok/s | 54.64 GB | 7.576B tok iter 57820 | train 2.849 | val 2.869 | lr 4.35e-04 | 102,463 tok/s | 54.64 GB | 7.579B tok iter 57840 | train 2.800 | val 2.848 | lr 4.35e-04 | 102,466 tok/s | 54.64 GB | 7.581B tok iter 57860 | train 2.827 | val 2.816 | lr 4.35e-04 | 102,466 tok/s | 54.64 GB | 7.584B tok iter 57880 | train 2.807 | val 2.869 | lr 4.35e-04 | 102,469 tok/s | 54.64 GB | 7.587B tok iter 57900 | train 2.759 | val 2.811 | lr 4.35e-04 | 102,472 tok/s | 54.64 GB | 7.589B tok iter 57920 | train 2.804 | val 2.841 | lr 4.35e-04 | 102,475 tok/s | 54.64 GB | 7.592B tok iter 57940 | train 2.797 | val 2.872 | lr 4.35e-04 | 102,475 tok/s | 54.64 GB | 7.594B tok iter 57960 | train 2.861 | val 2.823 | lr 4.34e-04 | 102,478 tok/s | 54.64 GB | 7.597B tok iter 57980 | train 2.739 | val 2.881 | lr 4.34e-04 | 102,481 tok/s | 54.64 GB | 7.600B tok checkpoint @ iter 57990 -> checkpoints/500m_latest.pt iter 58000 | train 2.811 | val 2.814 | lr 4.34e-04 | 102,465 tok/s | 54.64 GB | 7.602B tok checkpoint @ iter 58000 -> checkpoints/500m_latest.pt iter 58020 | train 2.875 | val 2.803 | lr 4.34e-04 | 102,431 tok/s | 54.64 GB | 7.605B tok iter 58040 | train 2.878 | val 2.825 | lr 4.34e-04 | 102,433 tok/s | 54.64 GB | 7.608B tok iter 58060 | train 2.810 | val 2.828 | lr 4.34e-04 | 102,436 tok/s | 54.64 GB | 7.610B tok iter 58080 | train 2.833 | val 2.783 | lr 4.34e-04 | 102,439 tok/s | 54.64 GB | 7.613B tok iter 58100 | train 2.796 | val 2.825 | lr 4.34e-04 | 102,440 tok/s | 54.64 GB | 7.615B tok iter 58120 | train 2.854 | val 2.830 | lr 4.34e-04 | 102,442 tok/s | 54.64 GB | 7.618B tok iter 58140 | train 2.845 | val 2.853 | lr 4.34e-04 | 102,445 tok/s | 54.64 GB | 7.621B tok iter 58160 | train 2.895 | val 2.778 | lr 4.33e-04 | 102,448 tok/s | 54.64 GB | 7.623B tok iter 58180 | train 2.761 | val 2.761 | lr 4.33e-04 | 102,448 tok/s | 54.64 GB | 7.626B tok iter 58200 | train 2.704 | val 2.840 | lr 4.33e-04 | 102,451 tok/s | 54.64 GB | 7.629B tok iter 58220 | train 2.947 | val 2.849 | lr 4.33e-04 | 102,454 tok/s | 54.64 GB | 7.631B tok iter 58240 | train 2.840 | val 2.810 | lr 4.33e-04 | 102,457 tok/s | 54.64 GB | 7.634B tok iter 58260 | train 2.824 | val 2.839 | lr 4.33e-04 | 102,457 tok/s | 54.64 GB | 7.636B tok iter 58280 | train 2.866 | val 2.838 | lr 4.33e-04 | 102,460 tok/s | 54.64 GB | 7.639B tok iter 58300 | train 2.996 | val 2.881 | lr 4.33e-04 | 102,463 tok/s | 54.64 GB | 7.642B tok iter 58320 | train 2.828 | val 2.908 | lr 4.33e-04 | 102,465 tok/s | 54.64 GB | 7.644B tok iter 58340 | train 2.726 | val 2.853 | lr 4.32e-04 | 102,466 tok/s | 54.64 GB | 7.647B tok iter 58360 | train 2.782 | val 2.822 | lr 4.32e-04 | 102,469 tok/s | 54.64 GB | 7.649B tok iter 58380 | train 2.749 | val 2.827 | lr 4.32e-04 | 102,471 tok/s | 54.64 GB | 7.652B tok iter 58400 | train 2.851 | val 2.839 | lr 4.32e-04 | 102,474 tok/s | 54.64 GB | 7.655B tok iter 58420 | train 2.817 | val 2.870 | lr 4.32e-04 | 102,474 tok/s | 54.64 GB | 7.657B tok iter 58440 | train 2.862 | val 2.865 | lr 4.32e-04 | 102,477 tok/s | 54.64 GB | 7.660B tok iter 58460 | train 2.834 | val 2.836 | lr 4.32e-04 | 102,480 tok/s | 54.64 GB | 7.663B tok iter 58480 | train 2.884 | val 2.856 | lr 4.32e-04 | 102,483 tok/s | 54.64 GB | 7.665B tok checkpoint @ iter 58491 -> checkpoints/500m_latest.pt iter 58500 | train 2.923 | val 2.794 | lr 4.32e-04 | 102,466 tok/s | 54.64 GB | 7.668B tok checkpoint @ iter 58500 -> checkpoints/500m_latest.pt iter 58520 | train 2.852 | val 2.840 | lr 4.32e-04 | 102,433 tok/s | 54.64 GB | 7.670B tok iter 58540 | train 2.764 | val 2.824 | lr 4.31e-04 | 102,436 tok/s | 54.64 GB | 7.673B tok iter 58560 | train 2.932 | val 2.847 | lr 4.31e-04 | 102,439 tok/s | 54.64 GB | 7.676B tok iter 58580 | train 2.885 | val 2.822 | lr 4.31e-04 | 102,439 tok/s | 54.64 GB | 7.678B tok iter 58600 | train 2.881 | val 2.857 | lr 4.31e-04 | 102,442 tok/s | 54.64 GB | 7.681B tok iter 58620 | train 2.917 | val 2.808 | lr 4.31e-04 | 102,445 tok/s | 54.64 GB | 7.684B tok iter 58640 | train 2.591 | val 2.852 | lr 4.31e-04 | 102,446 tok/s | 54.64 GB | 7.686B tok iter 58660 | train 2.872 | val 2.852 | lr 4.31e-04 | 102,447 tok/s | 54.64 GB | 7.689B tok iter 58680 | train 2.879 | val 2.838 | lr 4.31e-04 | 102,450 tok/s | 54.64 GB | 7.691B tok iter 58700 | train 2.747 | val 2.839 | lr 4.31e-04 | 102,452 tok/s | 54.64 GB | 7.694B tok iter 58720 | train 2.814 | val 2.842 | lr 4.30e-04 | 102,455 tok/s | 54.64 GB | 7.697B tok iter 58740 | train 2.829 | val 2.831 | lr 4.30e-04 | 102,455 tok/s | 54.64 GB | 7.699B tok iter 58760 | train 2.786 | val 2.825 | lr 4.30e-04 | 102,458 tok/s | 54.64 GB | 7.702B tok iter 58780 | train 2.783 | val 2.838 | lr 4.30e-04 | 102,461 tok/s | 54.64 GB | 7.705B tok iter 58800 | train 2.867 | val 2.825 | lr 4.30e-04 | 102,464 tok/s | 54.64 GB | 7.707B tok iter 58820 | train 2.858 | val 2.850 | lr 4.30e-04 | 102,464 tok/s | 54.64 GB | 7.710B tok iter 58840 | train 2.941 | val 2.807 | lr 4.30e-04 | 102,467 tok/s | 54.64 GB | 7.712B tok iter 58860 | train 2.917 | val 2.881 | lr 4.30e-04 | 102,470 tok/s | 54.64 GB | 7.715B tok iter 58880 | train 2.852 | val 2.814 | lr 4.30e-04 | 102,472 tok/s | 54.64 GB | 7.718B tok iter 58900 | train 2.800 | val 2.851 | lr 4.30e-04 | 102,473 tok/s | 54.64 GB | 7.720B tok iter 58920 | train 2.824 | val 2.852 | lr 4.29e-04 | 102,476 tok/s | 54.64 GB | 7.723B tok iter 58940 | train 2.730 | val 2.826 | lr 4.29e-04 | 102,478 tok/s | 54.64 GB | 7.726B tok iter 58960 | train 2.811 | val 2.814 | lr 4.29e-04 | 102,481 tok/s | 54.64 GB | 7.728B tok iter 58980 | train 2.931 | val 2.845 | lr 4.29e-04 | 102,482 tok/s | 54.64 GB | 7.731B tok checkpoint @ iter 58990 -> checkpoints/500m_latest.pt iter 59000 | train 2.853 | val 2.838 | lr 4.29e-04 | 102,467 tok/s | 54.64 GB | 7.733B tok checkpoint @ iter 59000 -> checkpoints/500m_latest.pt iter 59020 | train 2.823 | val 2.817 | lr 4.29e-04 | 102,435 tok/s | 54.64 GB | 7.736B tok iter 59040 | train 2.812 | val 2.805 | lr 4.29e-04 | 102,437 tok/s | 54.64 GB | 7.739B tok iter 59060 | train 2.778 | val 2.818 | lr 4.29e-04 | 102,438 tok/s | 54.64 GB | 7.741B tok iter 59080 | train 2.832 | val 2.843 | lr 4.29e-04 | 102,441 tok/s | 54.64 GB | 7.744B tok iter 59100 | train 2.795 | val 2.859 | lr 4.29e-04 | 102,443 tok/s | 54.64 GB | 7.746B tok iter 59120 | train 2.944 | val 2.878 | lr 4.28e-04 | 102,446 tok/s | 54.64 GB | 7.749B tok iter 59140 | train 2.717 | val 2.871 | lr 4.28e-04 | 102,447 tok/s | 54.64 GB | 7.752B tok iter 59160 | train 2.916 | val 2.851 | lr 4.28e-04 | 102,449 tok/s | 54.64 GB | 7.754B tok iter 59180 | train 2.661 | val 2.800 | lr 4.28e-04 | 102,452 tok/s | 54.64 GB | 7.757B tok iter 59200 | train 2.879 | val 2.812 | lr 4.28e-04 | 102,455 tok/s | 54.64 GB | 7.760B tok iter 59220 | train 2.857 | val 2.785 | lr 4.28e-04 | 102,455 tok/s | 54.64 GB | 7.762B tok iter 59240 | train 2.824 | val 2.815 | lr 4.28e-04 | 102,458 tok/s | 54.64 GB | 7.765B tok iter 59260 | train 2.859 | val 2.829 | lr 4.28e-04 | 102,461 tok/s | 54.64 GB | 7.767B tok iter 59280 | train 2.880 | val 2.836 | lr 4.28e-04 | 102,463 tok/s | 54.64 GB | 7.770B tok iter 59300 | train 3.012 | val 2.805 | lr 4.27e-04 | 102,464 tok/s | 54.64 GB | 7.773B tok iter 59320 | train 2.903 | val 2.855 | lr 4.27e-04 | 102,466 tok/s | 54.64 GB | 7.775B tok iter 59340 | train 2.963 | val 2.842 | lr 4.27e-04 | 102,469 tok/s | 54.64 GB | 7.778B tok iter 59360 | train 2.730 | val 2.864 | lr 4.27e-04 | 102,472 tok/s | 54.64 GB | 7.781B tok iter 59380 | train 2.863 | val 2.866 | lr 4.27e-04 | 102,472 tok/s | 54.64 GB | 7.783B tok iter 59400 | train 2.721 | val 2.855 | lr 4.27e-04 | 102,475 tok/s | 54.64 GB | 7.786B tok iter 59420 | train 2.779 | val 2.848 | lr 4.27e-04 | 102,478 tok/s | 54.64 GB | 7.788B tok iter 59440 | train 2.837 | val 2.860 | lr 4.27e-04 | 102,480 tok/s | 54.64 GB | 7.791B tok iter 59460 | train 2.718 | val 2.847 | lr 4.27e-04 | 102,481 tok/s | 54.64 GB | 7.794B tok iter 59480 | train 2.816 | val 2.841 | lr 4.27e-04 | 102,484 tok/s | 54.64 GB | 7.796B tok checkpoint @ iter 59491 -> checkpoints/500m_latest.pt iter 59500 | train 2.973 | val 2.821 | lr 4.26e-04 | 102,469 tok/s | 54.64 GB | 7.799B tok checkpoint @ iter 59500 -> checkpoints/500m_latest.pt iter 59520 | train 2.863 | val 2.839 | lr 4.26e-04 | 102,437 tok/s | 54.64 GB | 7.802B tok iter 59540 | train 2.921 | val 2.844 | lr 4.26e-04 | 102,438 tok/s | 54.64 GB | 7.804B tok iter 59560 | train 2.765 | val 2.829 | lr 4.26e-04 | 102,440 tok/s | 54.64 GB | 7.807B tok iter 59580 | train 2.817 | val 2.802 | lr 4.26e-04 | 102,443 tok/s | 54.64 GB | 7.809B tok iter 59600 | train 2.632 | val 2.816 | lr 4.26e-04 | 102,446 tok/s | 54.64 GB | 7.812B tok iter 59620 | train 2.775 | val 2.842 | lr 4.26e-04 | 102,446 tok/s | 54.64 GB | 7.815B tok iter 59640 | train 2.876 | val 2.798 | lr 4.26e-04 | 102,449 tok/s | 54.64 GB | 7.817B tok iter 59660 | train 2.866 | val 2.807 | lr 4.26e-04 | 102,452 tok/s | 54.64 GB | 7.820B tok iter 59680 | train 2.914 | val 2.865 | lr 4.25e-04 | 102,454 tok/s | 54.64 GB | 7.823B tok iter 59700 | train 2.701 | val 2.845 | lr 4.25e-04 | 102,455 tok/s | 54.64 GB | 7.825B tok iter 59720 | train 2.878 | val 2.849 | lr 4.25e-04 | 102,458 tok/s | 54.64 GB | 7.828B tok iter 59740 | train 2.773 | val 2.822 | lr 4.25e-04 | 102,460 tok/s | 54.64 GB | 7.830B tok iter 59760 | train 2.880 | val 2.858 | lr 4.25e-04 | 102,463 tok/s | 54.64 GB | 7.833B tok iter 59780 | train 2.780 | val 2.861 | lr 4.25e-04 | 102,463 tok/s | 54.64 GB | 7.836B tok iter 59800 | train 2.879 | val 2.825 | lr 4.25e-04 | 102,466 tok/s | 54.64 GB | 7.838B tok iter 59820 | train 2.957 | val 2.785 | lr 4.25e-04 | 102,469 tok/s | 54.64 GB | 7.841B tok iter 59840 | train 2.952 | val 2.862 | lr 4.25e-04 | 102,471 tok/s | 54.64 GB | 7.843B tok iter 59860 | train 2.797 | val 2.838 | lr 4.25e-04 | 102,472 tok/s | 54.64 GB | 7.846B tok iter 59880 | train 2.806 | val 2.845 | lr 4.24e-04 | 102,474 tok/s | 54.64 GB | 7.849B tok iter 59900 | train 2.853 | val 2.818 | lr 4.24e-04 | 102,477 tok/s | 54.64 GB | 7.851B tok iter 59920 | train 2.880 | val 2.784 | lr 4.24e-04 | 102,480 tok/s | 54.64 GB | 7.854B tok iter 59940 | train 2.897 | val 2.900 | lr 4.24e-04 | 102,480 tok/s | 54.64 GB | 7.857B tok iter 59960 | train 2.869 | val 2.833 | lr 4.24e-04 | 102,483 tok/s | 54.64 GB | 7.859B tok iter 59980 | train 2.828 | val 2.827 | lr 4.24e-04 | 102,485 tok/s | 54.64 GB | 7.862B tok checkpoint @ iter 59990 -> checkpoints/500m_latest.pt iter 60000 | train 2.736 | val 2.862 | lr 4.24e-04 | 102,471 tok/s | 54.64 GB | 7.864B tok checkpoint @ iter 60000 -> checkpoints/500m_latest.pt iter 60020 | train 2.731 | val 2.840 | lr 4.24e-04 | 102,437 tok/s | 54.64 GB | 7.867B tok iter 60040 | train 2.687 | val 2.829 | lr 4.24e-04 | 102,440 tok/s | 54.64 GB | 7.870B tok iter 60060 | train 2.794 | val 2.813 | lr 4.23e-04 | 102,443 tok/s | 54.64 GB | 7.872B tok iter 60080 | train 2.831 | val 2.872 | lr 4.23e-04 | 102,445 tok/s | 54.64 GB | 7.875B tok iter 60100 | train 2.755 | val 2.854 | lr 4.23e-04 | 102,446 tok/s | 54.64 GB | 7.878B tok iter 60120 | train 2.864 | val 2.829 | lr 4.23e-04 | 102,448 tok/s | 54.64 GB | 7.880B tok iter 60140 | train 2.646 | val 2.847 | lr 4.23e-04 | 102,451 tok/s | 54.64 GB | 7.883B tok iter 60160 | train 2.626 | val 2.830 | lr 4.23e-04 | 102,454 tok/s | 54.64 GB | 7.885B tok iter 60180 | train 2.864 | val 2.835 | lr 4.23e-04 | 102,454 tok/s | 54.64 GB | 7.888B tok iter 60200 | train 2.882 | val 2.799 | lr 4.23e-04 | 102,457 tok/s | 54.64 GB | 7.891B tok iter 60220 | train 2.884 | val 2.823 | lr 4.23e-04 | 102,460 tok/s | 54.64 GB | 7.893B tok iter 60240 | train 2.891 | val 2.826 | lr 4.23e-04 | 102,462 tok/s | 54.64 GB | 7.896B tok iter 60260 | train 2.877 | val 2.857 | lr 4.22e-04 | 102,463 tok/s | 54.64 GB | 7.899B tok iter 60280 | train 2.886 | val 2.834 | lr 4.22e-04 | 102,465 tok/s | 54.64 GB | 7.901B tok iter 60300 | train 2.955 | val 2.792 | lr 4.22e-04 | 102,468 tok/s | 54.64 GB | 7.904B tok iter 60320 | train 2.754 | val 2.852 | lr 4.22e-04 | 102,471 tok/s | 54.64 GB | 7.906B tok iter 60340 | train 2.972 | val 2.871 | lr 4.22e-04 | 102,471 tok/s | 54.64 GB | 7.909B tok iter 60360 | train 2.877 | val 2.811 | lr 4.22e-04 | 102,474 tok/s | 54.64 GB | 7.912B tok iter 60380 | train 2.737 | val 2.786 | lr 4.22e-04 | 102,477 tok/s | 54.64 GB | 7.914B tok iter 60400 | train 2.912 | val 2.822 | lr 4.22e-04 | 102,479 tok/s | 54.64 GB | 7.917B tok iter 60420 | train 2.899 | val 2.876 | lr 4.22e-04 | 102,480 tok/s | 54.64 GB | 7.920B tok iter 60440 | train 2.824 | val 2.874 | lr 4.21e-04 | 102,482 tok/s | 54.64 GB | 7.922B tok iter 60460 | train 2.950 | val 2.868 | lr 4.21e-04 | 102,485 tok/s | 54.64 GB | 7.925B tok iter 60480 | train 2.763 | val 2.833 | lr 4.21e-04 | 102,488 tok/s | 54.64 GB | 7.927B tok checkpoint @ iter 60491 -> checkpoints/500m_latest.pt iter 60500 | train 2.931 | val 2.833 | lr 4.21e-04 | 102,471 tok/s | 54.64 GB | 7.930B tok checkpoint @ iter 60500 -> checkpoints/500m_latest.pt iter 60520 | train 2.988 | val 2.782 | lr 4.21e-04 | 102,440 tok/s | 54.64 GB | 7.933B tok iter 60540 | train 2.844 | val 2.808 | lr 4.21e-04 | 102,443 tok/s | 54.64 GB | 7.935B tok iter 60560 | train 2.839 | val 2.896 | lr 4.21e-04 | 102,445 tok/s | 54.64 GB | 7.938B tok iter 60580 | train 2.827 | val 2.808 | lr 4.21e-04 | 102,446 tok/s | 54.64 GB | 7.940B tok iter 60600 | train 2.848 | val 2.862 | lr 4.21e-04 | 102,448 tok/s | 54.64 GB | 7.943B tok iter 60620 | train 2.850 | val 2.790 | lr 4.21e-04 | 102,451 tok/s | 54.64 GB | 7.946B tok iter 60640 | train 2.849 | val 2.836 | lr 4.20e-04 | 102,454 tok/s | 54.64 GB | 7.948B tok iter 60660 | train 2.883 | val 2.828 | lr 4.20e-04 | 102,454 tok/s | 54.64 GB | 7.951B tok iter 60680 | train 2.752 | val 2.817 | lr 4.20e-04 | 102,457 tok/s | 54.64 GB | 7.954B tok iter 60700 | train 2.833 | val 2.855 | lr 4.20e-04 | 102,460 tok/s | 54.64 GB | 7.956B tok iter 60720 | train 2.731 | val 2.868 | lr 4.20e-04 | 102,462 tok/s | 54.64 GB | 7.959B tok iter 60740 | train 2.833 | val 2.816 | lr 4.20e-04 | 102,463 tok/s | 54.64 GB | 7.961B tok iter 60760 | train 2.886 | val 2.877 | lr 4.20e-04 | 102,465 tok/s | 54.64 GB | 7.964B tok iter 60780 | train 2.800 | val 2.840 | lr 4.20e-04 | 102,468 tok/s | 54.64 GB | 7.967B tok iter 60800 | train 2.887 | val 2.816 | lr 4.20e-04 | 102,471 tok/s | 54.64 GB | 7.969B tok iter 60820 | train 2.780 | val 2.818 | lr 4.19e-04 | 102,471 tok/s | 54.64 GB | 7.972B tok iter 60840 | train 2.874 | val 2.859 | lr 4.19e-04 | 102,474 tok/s | 54.64 GB | 7.975B tok iter 60860 | train 2.818 | val 2.798 | lr 4.19e-04 | 102,476 tok/s | 54.64 GB | 7.977B tok iter 60880 | train 3.011 | val 2.871 | lr 4.19e-04 | 102,479 tok/s | 54.64 GB | 7.980B tok iter 60900 | train 2.836 | val 2.851 | lr 4.19e-04 | 102,479 tok/s | 54.64 GB | 7.982B tok iter 60920 | train 2.747 | val 2.862 | lr 4.19e-04 | 102,482 tok/s | 54.64 GB | 7.985B tok iter 60940 | train 2.831 | val 2.842 | lr 4.19e-04 | 102,485 tok/s | 54.64 GB | 7.988B tok iter 60960 | train 2.866 | val 2.803 | lr 4.19e-04 | 102,487 tok/s | 54.64 GB | 7.990B tok iter 60980 | train 2.735 | val 2.835 | lr 4.19e-04 | 102,487 tok/s | 54.64 GB | 7.993B tok checkpoint @ iter 60991 -> checkpoints/500m_latest.pt iter 61000 | train 2.860 | val 2.837 | lr 4.18e-04 | 102,473 tok/s | 54.64 GB | 7.996B tok checkpoint @ iter 61000 -> checkpoints/500m_latest.pt iter 61020 | train 2.884 | val 2.840 | lr 4.18e-04 | 102,442 tok/s | 54.64 GB | 7.998B tok iter 61040 | train 2.840 | val 2.854 | lr 4.18e-04 | 102,445 tok/s | 54.64 GB | 8.001B tok iter 61060 | train 2.672 | val 2.823 | lr 4.18e-04 | 102,445 tok/s | 54.64 GB | 8.003B tok iter 61080 | train 2.872 | val 2.829 | lr 4.18e-04 | 102,448 tok/s | 54.64 GB | 8.006B tok iter 61100 | train 2.925 | val 2.849 | lr 4.18e-04 | 102,451 tok/s | 54.64 GB | 8.009B tok iter 61120 | train 2.876 | val 2.871 | lr 4.18e-04 | 102,453 tok/s | 54.64 GB | 8.011B tok iter 61140 | train 2.883 | val 2.826 | lr 4.18e-04 | 102,454 tok/s | 54.64 GB | 8.014B tok iter 61160 | train 2.859 | val 2.857 | lr 4.18e-04 | 102,456 tok/s | 54.64 GB | 8.016B tok iter 61180 | train 2.778 | val 2.860 | lr 4.18e-04 | 102,459 tok/s | 54.64 GB | 8.019B tok iter 61200 | train 2.890 | val 2.810 | lr 4.17e-04 | 102,462 tok/s | 54.64 GB | 8.022B tok iter 61220 | train 2.857 | val 2.807 | lr 4.17e-04 | 102,462 tok/s | 54.64 GB | 8.024B tok iter 61240 | train 2.911 | val 2.836 | lr 4.17e-04 | 102,465 tok/s | 54.64 GB | 8.027B tok iter 61260 | train 2.811 | val 2.885 | lr 4.17e-04 | 102,467 tok/s | 54.64 GB | 8.030B tok iter 61280 | train 2.795 | val 2.821 | lr 4.17e-04 | 102,470 tok/s | 54.64 GB | 8.032B tok iter 61300 | train 2.793 | val 2.828 | lr 4.17e-04 | 102,470 tok/s | 54.64 GB | 8.035B tok iter 61320 | train 2.774 | val 2.855 | lr 4.17e-04 | 102,473 tok/s | 54.64 GB | 8.037B tok iter 61340 | train 2.764 | val 2.798 | lr 4.17e-04 | 102,475 tok/s | 54.64 GB | 8.040B tok iter 61360 | train 2.728 | val 2.866 | lr 4.17e-04 | 102,478 tok/s | 54.64 GB | 8.043B tok iter 61380 | train 2.754 | val 2.861 | lr 4.16e-04 | 102,479 tok/s | 54.64 GB | 8.045B tok iter 61400 | train 2.789 | val 2.813 | lr 4.16e-04 | 102,481 tok/s | 54.64 GB | 8.048B tok iter 61420 | train 2.727 | val 2.832 | lr 4.16e-04 | 102,484 tok/s | 54.64 GB | 8.051B tok iter 61440 | train 2.921 | val 2.855 | lr 4.16e-04 | 102,486 tok/s | 54.64 GB | 8.053B tok iter 61460 | train 2.852 | val 2.799 | lr 4.16e-04 | 102,487 tok/s | 54.64 GB | 8.056B tok iter 61480 | train 2.812 | val 2.805 | lr 4.16e-04 | 102,489 tok/s | 54.64 GB | 8.058B tok checkpoint @ iter 61491 -> checkpoints/500m_latest.pt iter 61500 | train 2.938 | val 2.779 | lr 4.16e-04 | 102,476 tok/s | 54.64 GB | 8.061B tok checkpoint @ iter 61500 -> checkpoints/500m_latest.pt iter 61520 | train 2.737 | val 2.823 | lr 4.16e-04 | 102,445 tok/s | 54.64 GB | 8.064B tok iter 61540 | train 2.850 | val 2.841 | lr 4.16e-04 | 102,446 tok/s | 54.64 GB | 8.066B tok iter 61560 | train 2.903 | val 2.838 | lr 4.16e-04 | 102,448 tok/s | 54.64 GB | 8.069B tok iter 61580 | train 2.874 | val 2.850 | lr 4.15e-04 | 102,451 tok/s | 54.64 GB | 8.072B tok iter 61600 | train 2.988 | val 2.818 | lr 4.15e-04 | 102,454 tok/s | 54.64 GB | 8.074B tok iter 61620 | train 2.865 | val 2.826 | lr 4.15e-04 | 102,454 tok/s | 54.64 GB | 8.077B tok iter 61640 | train 2.777 | val 2.860 | lr 4.15e-04 | 102,457 tok/s | 54.64 GB | 8.079B tok iter 61660 | train 2.828 | val 2.829 | lr 4.15e-04 | 102,459 tok/s | 54.64 GB | 8.082B tok iter 61680 | train 2.889 | val 2.852 | lr 4.15e-04 | 102,462 tok/s | 54.64 GB | 8.085B tok iter 61700 | train 2.830 | val 2.821 | lr 4.15e-04 | 102,462 tok/s | 54.64 GB | 8.087B tok iter 61720 | train 2.711 | val 2.831 | lr 4.15e-04 | 102,465 tok/s | 54.64 GB | 8.090B tok iter 61740 | train 2.781 | val 2.779 | lr 4.15e-04 | 102,468 tok/s | 54.64 GB | 8.093B tok iter 61760 | train 2.838 | val 2.876 | lr 4.14e-04 | 102,470 tok/s | 54.64 GB | 8.095B tok iter 61780 | train 2.835 | val 2.854 | lr 4.14e-04 | 102,471 tok/s | 54.64 GB | 8.098B tok iter 61800 | train 2.933 | val 2.785 | lr 4.14e-04 | 102,473 tok/s | 54.64 GB | 8.100B tok iter 61820 | train 2.873 | val 2.814 | lr 4.14e-04 | 102,476 tok/s | 54.64 GB | 8.103B tok iter 61840 | train 2.856 | val 2.789 | lr 4.14e-04 | 102,478 tok/s | 54.64 GB | 8.106B tok iter 61860 | train 2.772 | val 2.844 | lr 4.14e-04 | 102,479 tok/s | 54.64 GB | 8.108B tok iter 61880 | train 2.860 | val 2.812 | lr 4.14e-04 | 102,481 tok/s | 54.64 GB | 8.111B tok iter 61900 | train 2.881 | val 2.853 | lr 4.14e-04 | 102,484 tok/s | 54.64 GB | 8.113B tok iter 61920 | train 2.896 | val 2.800 | lr 4.14e-04 | 102,486 tok/s | 54.64 GB | 8.116B tok iter 61940 | train 2.743 | val 2.793 | lr 4.13e-04 | 102,487 tok/s | 54.64 GB | 8.119B tok iter 61960 | train 2.838 | val 2.784 | lr 4.13e-04 | 102,489 tok/s | 54.64 GB | 8.121B tok iter 61980 | train 2.947 | val 2.813 | lr 4.13e-04 | 102,492 tok/s | 54.64 GB | 8.124B tok checkpoint @ iter 61990 -> checkpoints/500m_latest.pt iter 62000 | train 2.867 | val 2.832 | lr 4.13e-04 | 102,478 tok/s | 54.64 GB | 8.127B tok checkpoint @ iter 62000 -> checkpoints/500m_latest.pt iter 62020 | train 2.890 | val 2.801 | lr 4.13e-04 | 102,446 tok/s | 54.64 GB | 8.129B tok iter 62040 | train 2.801 | val 2.835 | lr 4.13e-04 | 102,449 tok/s | 54.64 GB | 8.132B tok iter 62060 | train 2.773 | val 2.850 | lr 4.13e-04 | 102,451 tok/s | 54.64 GB | 8.134B tok iter 62080 | train 2.839 | val 2.844 | lr 4.13e-04 | 102,454 tok/s | 54.64 GB | 8.137B tok iter 62100 | train 2.869 | val 2.875 | lr 4.13e-04 | 102,454 tok/s | 54.64 GB | 8.140B tok iter 62120 | train 2.852 | val 2.815 | lr 4.13e-04 | 102,457 tok/s | 54.64 GB | 8.142B tok iter 62140 | train 2.888 | val 2.818 | lr 4.12e-04 | 102,460 tok/s | 54.64 GB | 8.145B tok iter 62160 | train 2.879 | val 2.829 | lr 4.12e-04 | 102,462 tok/s | 54.64 GB | 8.148B tok iter 62180 | train 2.966 | val 2.867 | lr 4.12e-04 | 102,462 tok/s | 54.64 GB | 8.150B tok iter 62200 | train 2.946 | val 2.820 | lr 4.12e-04 | 102,465 tok/s | 54.64 GB | 8.153B tok iter 62220 | train 2.955 | val 2.859 | lr 4.12e-04 | 102,468 tok/s | 54.64 GB | 8.155B tok iter 62240 | train 2.779 | val 2.822 | lr 4.12e-04 | 102,470 tok/s | 54.64 GB | 8.158B tok iter 62260 | train 2.817 | val 2.836 | lr 4.12e-04 | 102,471 tok/s | 54.64 GB | 8.161B tok iter 62280 | train 2.724 | val 2.802 | lr 4.12e-04 | 102,473 tok/s | 54.64 GB | 8.163B tok iter 62300 | train 2.790 | val 2.818 | lr 4.12e-04 | 102,476 tok/s | 54.64 GB | 8.166B tok iter 62320 | train 2.798 | val 2.823 | lr 4.11e-04 | 102,478 tok/s | 54.64 GB | 8.169B tok iter 62340 | train 2.623 | val 2.826 | lr 4.11e-04 | 102,479 tok/s | 54.64 GB | 8.171B tok iter 62360 | train 2.921 | val 2.819 | lr 4.11e-04 | 102,481 tok/s | 54.64 GB | 8.174B tok iter 62380 | train 2.798 | val 2.850 | lr 4.11e-04 | 102,484 tok/s | 54.64 GB | 8.176B tok iter 62400 | train 2.890 | val 2.852 | lr 4.11e-04 | 102,486 tok/s | 54.64 GB | 8.179B tok iter 62420 | train 2.699 | val 2.871 | lr 4.11e-04 | 102,487 tok/s | 54.64 GB | 8.182B tok iter 62440 | train 2.871 | val 2.809 | lr 4.11e-04 | 102,489 tok/s | 54.64 GB | 8.184B tok iter 62460 | train 2.881 | val 2.816 | lr 4.11e-04 | 102,492 tok/s | 54.64 GB | 8.187B tok iter 62480 | train 2.797 | val 2.805 | lr 4.11e-04 | 102,494 tok/s | 54.64 GB | 8.190B tok checkpoint @ iter 62491 -> checkpoints/500m_latest.pt iter 62500 | train 2.890 | val 2.832 | lr 4.10e-04 | 102,478 tok/s | 54.64 GB | 8.192B tok checkpoint @ iter 62500 -> checkpoints/500m_latest.pt iter 62520 | train 2.869 | val 2.824 | lr 4.10e-04 | 102,448 tok/s | 54.64 GB | 8.195B tok iter 62540 | train 2.715 | val 2.810 | lr 4.10e-04 | 102,451 tok/s | 54.64 GB | 8.197B tok iter 62560 | train 2.828 | val 2.810 | lr 4.10e-04 | 102,453 tok/s | 54.64 GB | 8.200B tok iter 62580 | train 2.838 | val 2.861 | lr 4.10e-04 | 102,454 tok/s | 54.64 GB | 8.203B tok iter 62600 | train 2.700 | val 2.786 | lr 4.10e-04 | 102,456 tok/s | 54.64 GB | 8.205B tok iter 62620 | train 2.920 | val 2.901 | lr 4.10e-04 | 102,459 tok/s | 54.64 GB | 8.208B tok iter 62640 | train 2.851 | val 2.803 | lr 4.10e-04 | 102,461 tok/s | 54.64 GB | 8.210B tok iter 62660 | train 2.767 | val 2.838 | lr 4.10e-04 | 102,462 tok/s | 54.64 GB | 8.213B tok iter 62680 | train 2.834 | val 2.851 | lr 4.10e-04 | 102,464 tok/s | 54.64 GB | 8.216B tok iter 62700 | train 2.864 | val 2.830 | lr 4.09e-04 | 102,467 tok/s | 54.64 GB | 8.218B tok iter 62720 | train 2.735 | val 2.821 | lr 4.09e-04 | 102,469 tok/s | 54.64 GB | 8.221B tok iter 62740 | train 2.760 | val 2.813 | lr 4.09e-04 | 102,470 tok/s | 54.64 GB | 8.224B tok iter 62760 | train 2.687 | val 2.863 | lr 4.09e-04 | 102,472 tok/s | 54.64 GB | 8.226B tok iter 62780 | train 2.787 | val 2.784 | lr 4.09e-04 | 102,475 tok/s | 54.64 GB | 8.229B tok iter 62800 | train 2.792 | val 2.840 | lr 4.09e-04 | 102,477 tok/s | 54.64 GB | 8.231B tok iter 62820 | train 2.819 | val 2.885 | lr 4.09e-04 | 102,478 tok/s | 54.64 GB | 8.234B tok iter 62840 | train 2.764 | val 2.846 | lr 4.09e-04 | 102,480 tok/s | 54.64 GB | 8.237B tok iter 62860 | train 2.720 | val 2.802 | lr 4.09e-04 | 102,483 tok/s | 54.64 GB | 8.239B tok iter 62880 | train 2.855 | val 2.836 | lr 4.08e-04 | 102,485 tok/s | 54.64 GB | 8.242B tok iter 62900 | train 2.853 | val 2.803 | lr 4.08e-04 | 102,486 tok/s | 54.64 GB | 8.245B tok iter 62920 | train 2.725 | val 2.834 | lr 4.08e-04 | 102,488 tok/s | 54.64 GB | 8.247B tok iter 62940 | train 2.814 | val 2.827 | lr 4.08e-04 | 102,491 tok/s | 54.64 GB | 8.250B tok iter 62960 | train 2.709 | val 2.769 | lr 4.08e-04 | 102,493 tok/s | 54.64 GB | 8.252B tok iter 62980 | train 2.768 | val 2.814 | lr 4.08e-04 | 102,494 tok/s | 54.64 GB | 8.255B tok checkpoint @ iter 62990 -> checkpoints/500m_latest.pt iter 63000 | train 2.865 | val 2.856 | lr 4.08e-04 | 102,480 tok/s | 54.64 GB | 8.258B tok checkpoint @ iter 63000 -> checkpoints/500m_latest.pt iter 63020 | train 2.857 | val 2.787 | lr 4.08e-04 | 102,450 tok/s | 54.64 GB | 8.260B tok iter 63040 | train 2.870 | val 2.833 | lr 4.08e-04 | 102,453 tok/s | 54.64 GB | 8.263B tok iter 63060 | train 2.765 | val 2.827 | lr 4.07e-04 | 102,453 tok/s | 54.64 GB | 8.266B tok iter 63080 | train 2.851 | val 2.854 | lr 4.07e-04 | 102,456 tok/s | 54.64 GB | 8.268B tok iter 63100 | train 2.901 | val 2.818 | lr 4.07e-04 | 102,458 tok/s | 54.64 GB | 8.271B tok iter 63120 | train 2.825 | val 2.822 | lr 4.07e-04 | 102,461 tok/s | 54.64 GB | 8.273B tok iter 63140 | train 2.800 | val 2.853 | lr 4.07e-04 | 102,461 tok/s | 54.64 GB | 8.276B tok iter 63160 | train 2.962 | val 2.842 | lr 4.07e-04 | 102,464 tok/s | 54.64 GB | 8.279B tok iter 63180 | train 2.877 | val 2.801 | lr 4.07e-04 | 102,466 tok/s | 54.64 GB | 8.281B tok iter 63200 | train 2.696 | val 2.867 | lr 4.07e-04 | 102,469 tok/s | 54.64 GB | 8.284B tok iter 63220 | train 2.833 | val 2.784 | lr 4.07e-04 | 102,469 tok/s | 54.64 GB | 8.287B tok iter 63240 | train 2.831 | val 2.852 | lr 4.07e-04 | 102,472 tok/s | 54.64 GB | 8.289B tok iter 63260 | train 2.932 | val 2.855 | lr 4.06e-04 | 102,474 tok/s | 54.64 GB | 8.292B tok iter 63280 | train 2.825 | val 2.855 | lr 4.06e-04 | 102,477 tok/s | 54.64 GB | 8.294B tok iter 63300 | train 2.789 | val 2.829 | lr 4.06e-04 | 102,477 tok/s | 54.64 GB | 8.297B tok iter 63320 | train 2.841 | val 2.810 | lr 4.06e-04 | 102,480 tok/s | 54.64 GB | 8.300B tok iter 63340 | train 2.923 | val 2.788 | lr 4.06e-04 | 102,482 tok/s | 54.64 GB | 8.302B tok iter 63360 | train 2.671 | val 2.797 | lr 4.06e-04 | 102,485 tok/s | 54.64 GB | 8.305B tok iter 63380 | train 2.803 | val 2.839 | lr 4.06e-04 | 102,485 tok/s | 54.64 GB | 8.307B tok iter 63400 | train 2.719 | val 2.823 | lr 4.06e-04 | 102,488 tok/s | 54.64 GB | 8.310B tok iter 63420 | train 2.763 | val 2.791 | lr 4.06e-04 | 102,490 tok/s | 54.64 GB | 8.313B tok iter 63440 | train 2.976 | val 2.832 | lr 4.05e-04 | 102,493 tok/s | 54.64 GB | 8.315B tok iter 63460 | train 2.785 | val 2.824 | lr 4.05e-04 | 102,493 tok/s | 54.64 GB | 8.318B tok iter 63480 | train 2.742 | val 2.807 | lr 4.05e-04 | 102,496 tok/s | 54.64 GB | 8.321B tok checkpoint @ iter 63490 -> checkpoints/500m_latest.pt iter 63500 | train 2.799 | val 2.762 | lr 4.05e-04 | 102,482 tok/s | 54.64 GB | 8.323B tok checkpoint @ iter 63500 -> checkpoints/500m_latest.pt iter 63520 | train 2.899 | val 2.807 | lr 4.05e-04 | 102,452 tok/s | 54.64 GB | 8.326B tok iter 63540 | train 2.873 | val 2.822 | lr 4.05e-04 | 102,452 tok/s | 54.64 GB | 8.328B tok iter 63560 | train 2.890 | val 2.865 | lr 4.05e-04 | 102,455 tok/s | 54.64 GB | 8.331B tok iter 63580 | train 2.749 | val 2.851 | lr 4.05e-04 | 102,457 tok/s | 54.64 GB | 8.334B tok iter 63600 | train 2.712 | val 2.805 | lr 4.05e-04 | 102,460 tok/s | 54.64 GB | 8.336B tok iter 63620 | train 2.907 | val 2.861 | lr 4.04e-04 | 102,460 tok/s | 54.64 GB | 8.339B tok iter 63640 | train 2.728 | val 2.843 | lr 4.04e-04 | 102,463 tok/s | 54.64 GB | 8.342B tok iter 63660 | train 2.840 | val 2.762 | lr 4.04e-04 | 102,465 tok/s | 54.64 GB | 8.344B tok iter 63680 | train 2.796 | val 2.868 | lr 4.04e-04 | 102,468 tok/s | 54.64 GB | 8.347B tok iter 63700 | train 2.757 | val 2.813 | lr 4.04e-04 | 102,468 tok/s | 54.64 GB | 8.349B tok iter 63720 | train 2.874 | val 2.808 | lr 4.04e-04 | 102,471 tok/s | 54.64 GB | 8.352B tok iter 63740 | train 2.839 | val 2.800 | lr 4.04e-04 | 102,473 tok/s | 54.64 GB | 8.355B tok iter 63760 | train 2.811 | val 2.843 | lr 4.04e-04 | 102,476 tok/s | 54.64 GB | 8.357B tok iter 63780 | train 2.939 | val 2.841 | lr 4.04e-04 | 102,476 tok/s | 54.64 GB | 8.360B tok iter 63800 | train 2.646 | val 2.873 | lr 4.03e-04 | 102,479 tok/s | 54.64 GB | 8.363B tok iter 63820 | train 2.911 | val 2.779 | lr 4.03e-04 | 102,481 tok/s | 54.64 GB | 8.365B tok iter 63840 | train 2.791 | val 2.852 | lr 4.03e-04 | 102,484 tok/s | 54.64 GB | 8.368B tok iter 63860 | train 2.971 | val 2.891 | lr 4.03e-04 | 102,484 tok/s | 54.64 GB | 8.370B tok iter 63880 | train 2.672 | val 2.808 | lr 4.03e-04 | 102,487 tok/s | 54.64 GB | 8.373B tok iter 63900 | train 2.767 | val 2.823 | lr 4.03e-04 | 102,489 tok/s | 54.64 GB | 8.376B tok iter 63920 | train 2.929 | val 2.829 | lr 4.03e-04 | 102,492 tok/s | 54.64 GB | 8.378B tok iter 63940 | train 2.811 | val 2.798 | lr 4.03e-04 | 102,492 tok/s | 54.64 GB | 8.381B tok iter 63960 | train 2.751 | val 2.868 | lr 4.03e-04 | 102,495 tok/s | 54.64 GB | 8.383B tok iter 63980 | train 2.854 | val 2.849 | lr 4.03e-04 | 102,497 tok/s | 54.64 GB | 8.386B tok checkpoint @ iter 63991 -> checkpoints/500m_latest.pt iter 64000 | train 2.813 | val 2.829 | lr 4.02e-04 | 102,485 tok/s | 54.64 GB | 8.389B tok checkpoint @ iter 64000 -> checkpoints/500m_latest.pt iter 64020 | train 2.761 | val 2.853 | lr 4.02e-04 | 102,456 tok/s | 54.64 GB | 8.391B tok iter 64040 | train 2.749 | val 2.839 | lr 4.02e-04 | 102,458 tok/s | 54.64 GB | 8.394B tok iter 64060 | train 2.739 | val 2.858 | lr 4.02e-04 | 102,461 tok/s | 54.64 GB | 8.397B tok iter 64080 | train 2.963 | val 2.814 | lr 4.02e-04 | 102,463 tok/s | 54.64 GB | 8.399B tok iter 64100 | train 2.758 | val 2.803 | lr 4.02e-04 | 102,464 tok/s | 54.64 GB | 8.402B tok iter 64120 | train 3.086 | val 2.835 | lr 4.02e-04 | 102,466 tok/s | 54.64 GB | 8.404B tok iter 64140 | train 2.854 | val 2.836 | lr 4.02e-04 | 102,469 tok/s | 54.64 GB | 8.407B tok iter 64160 | train 2.843 | val 2.802 | lr 4.02e-04 | 102,471 tok/s | 54.64 GB | 8.410B tok iter 64180 | train 2.865 | val 2.817 | lr 4.01e-04 | 102,472 tok/s | 54.64 GB | 8.412B tok iter 64200 | train 2.846 | val 2.778 | lr 4.01e-04 | 102,474 tok/s | 54.64 GB | 8.415B tok iter 64220 | train 2.757 | val 2.882 | lr 4.01e-04 | 102,477 tok/s | 54.64 GB | 8.418B tok iter 64240 | train 2.681 | val 2.818 | lr 4.01e-04 | 102,479 tok/s | 54.64 GB | 8.420B tok iter 64260 | train 2.697 | val 2.816 | lr 4.01e-04 | 102,479 tok/s | 54.64 GB | 8.423B tok iter 64280 | train 2.857 | val 2.821 | lr 4.01e-04 | 102,482 tok/s | 54.64 GB | 8.425B tok iter 64300 | train 2.635 | val 2.827 | lr 4.01e-04 | 102,484 tok/s | 54.64 GB | 8.428B tok iter 64320 | train 2.753 | val 2.791 | lr 4.01e-04 | 102,487 tok/s | 54.64 GB | 8.431B tok iter 64340 | train 2.819 | val 2.819 | lr 4.01e-04 | 102,487 tok/s | 54.64 GB | 8.433B tok iter 64360 | train 2.956 | val 2.814 | lr 4.00e-04 | 102,490 tok/s | 54.64 GB | 8.436B tok iter 64380 | train 2.781 | val 2.836 | lr 4.00e-04 | 102,492 tok/s | 54.64 GB | 8.439B tok iter 64400 | train 2.782 | val 2.831 | lr 4.00e-04 | 102,495 tok/s | 54.64 GB | 8.441B tok iter 64420 | train 2.803 | val 2.812 | lr 4.00e-04 | 102,495 tok/s | 54.64 GB | 8.444B tok iter 64440 | train 2.840 | val 2.796 | lr 4.00e-04 | 102,497 tok/s | 54.64 GB | 8.446B tok iter 64460 | train 2.804 | val 2.849 | lr 4.00e-04 | 102,500 tok/s | 54.64 GB | 8.449B tok iter 64480 | train 2.839 | val 2.744 | lr 4.00e-04 | 102,502 tok/s | 54.64 GB | 8.452B tok checkpoint @ iter 64491 -> checkpoints/500m_latest.pt iter 64500 | train 2.763 | val 2.822 | lr 4.00e-04 | 102,484 tok/s | 54.64 GB | 8.454B tok checkpoint @ iter 64500 -> checkpoints/500m_latest.pt iter 64520 | train 2.838 | val 2.822 | lr 4.00e-04 | 102,452 tok/s | 54.64 GB | 8.457B tok iter 64540 | train 2.757 | val 2.803 | lr 3.99e-04 | 102,454 tok/s | 54.64 GB | 8.460B tok iter 64560 | train 2.797 | val 2.807 | lr 3.99e-04 | 102,457 tok/s | 54.64 GB | 8.462B tok iter 64580 | train 2.843 | val 2.837 | lr 3.99e-04 | 102,457 tok/s | 54.64 GB | 8.465B tok iter 64600 | train 2.909 | val 2.839 | lr 3.99e-04 | 102,460 tok/s | 54.64 GB | 8.467B tok iter 64620 | train 2.837 | val 2.832 | lr 3.99e-04 | 102,462 tok/s | 54.64 GB | 8.470B tok iter 64640 | train 2.744 | val 2.850 | lr 3.99e-04 | 102,465 tok/s | 54.64 GB | 8.473B tok iter 64660 | train 2.799 | val 2.782 | lr 3.99e-04 | 102,465 tok/s | 54.64 GB | 8.475B tok iter 64680 | train 2.730 | val 2.850 | lr 3.99e-04 | 102,468 tok/s | 54.64 GB | 8.478B tok iter 64700 | train 2.834 | val 2.804 | lr 3.99e-04 | 102,470 tok/s | 54.64 GB | 8.480B tok iter 64720 | train 2.909 | val 2.859 | lr 3.99e-04 | 102,473 tok/s | 54.64 GB | 8.483B tok iter 64740 | train 2.925 | val 2.839 | lr 3.98e-04 | 102,473 tok/s | 54.64 GB | 8.486B tok iter 64760 | train 2.905 | val 2.803 | lr 3.98e-04 | 102,475 tok/s | 54.64 GB | 8.488B tok iter 64780 | train 2.710 | val 2.842 | lr 3.98e-04 | 102,478 tok/s | 54.64 GB | 8.491B tok iter 64800 | train 2.743 | val 2.838 | lr 3.98e-04 | 102,480 tok/s | 54.64 GB | 8.494B tok iter 64820 | train 2.798 | val 2.816 | lr 3.98e-04 | 102,481 tok/s | 54.64 GB | 8.496B tok iter 64840 | train 2.819 | val 2.807 | lr 3.98e-04 | 102,483 tok/s | 54.64 GB | 8.499B tok iter 64860 | train 2.843 | val 2.828 | lr 3.98e-04 | 102,486 tok/s | 54.64 GB | 8.501B tok iter 64880 | train 2.946 | val 2.811 | lr 3.98e-04 | 102,488 tok/s | 54.64 GB | 8.504B tok iter 64900 | train 2.788 | val 2.822 | lr 3.98e-04 | 102,488 tok/s | 54.64 GB | 8.507B tok iter 64920 | train 2.811 | val 2.841 | lr 3.97e-04 | 102,491 tok/s | 54.64 GB | 8.509B tok iter 64940 | train 2.836 | val 2.832 | lr 3.97e-04 | 102,493 tok/s | 54.64 GB | 8.512B tok iter 64960 | train 2.795 | val 2.834 | lr 3.97e-04 | 102,496 tok/s | 54.64 GB | 8.515B tok iter 64980 | train 2.629 | val 2.856 | lr 3.97e-04 | 102,496 tok/s | 54.64 GB | 8.517B tok checkpoint @ iter 64989 -> checkpoints/500m_latest.pt iter 65000 | train 2.679 | val 2.754 | lr 3.97e-04 | 102,483 tok/s | 54.64 GB | 8.520B tok checkpoint @ iter 65000 -> checkpoints/500m_latest.pt iter 65020 | train 2.742 | val 2.795 | lr 3.97e-04 | 102,457 tok/s | 54.64 GB | 8.522B tok iter 65040 | train 2.783 | val 2.837 | lr 3.97e-04 | 102,459 tok/s | 54.64 GB | 8.525B tok iter 65060 | train 2.659 | val 2.826 | lr 3.97e-04 | 102,460 tok/s | 54.64 GB | 8.528B tok iter 65080 | train 2.771 | val 2.836 | lr 3.97e-04 | 102,462 tok/s | 54.64 GB | 8.530B tok iter 65100 | train 2.798 | val 2.763 | lr 3.96e-04 | 102,465 tok/s | 54.64 GB | 8.533B tok iter 65120 | train 2.857 | val 2.853 | lr 3.96e-04 | 102,467 tok/s | 54.64 GB | 8.536B tok iter 65140 | train 2.844 | val 2.848 | lr 3.96e-04 | 102,467 tok/s | 54.64 GB | 8.538B tok iter 65160 | train 2.854 | val 2.834 | lr 3.96e-04 | 102,470 tok/s | 54.64 GB | 8.541B tok iter 65180 | train 2.867 | val 2.792 | lr 3.96e-04 | 102,472 tok/s | 54.64 GB | 8.543B tok iter 65200 | train 2.860 | val 2.778 | lr 3.96e-04 | 102,475 tok/s | 54.64 GB | 8.546B tok iter 65220 | train 2.914 | val 2.860 | lr 3.96e-04 | 102,475 tok/s | 54.64 GB | 8.549B tok iter 65240 | train 2.774 | val 2.844 | lr 3.96e-04 | 102,477 tok/s | 54.64 GB | 8.551B tok iter 65260 | train 2.856 | val 2.829 | lr 3.96e-04 | 102,480 tok/s | 54.64 GB | 8.554B tok iter 65280 | train 2.740 | val 2.826 | lr 3.95e-04 | 102,482 tok/s | 54.64 GB | 8.557B tok iter 65300 | train 2.695 | val 2.795 | lr 3.95e-04 | 102,483 tok/s | 54.64 GB | 8.559B tok iter 65320 | train 2.893 | val 2.825 | lr 3.95e-04 | 102,485 tok/s | 54.64 GB | 8.562B tok iter 65340 | train 2.849 | val 2.883 | lr 3.95e-04 | 102,488 tok/s | 54.64 GB | 8.564B tok iter 65360 | train 2.780 | val 2.787 | lr 3.95e-04 | 102,490 tok/s | 54.64 GB | 8.567B tok iter 65380 | train 2.869 | val 2.871 | lr 3.95e-04 | 102,491 tok/s | 54.64 GB | 8.570B tok iter 65400 | train 2.932 | val 2.832 | lr 3.95e-04 | 102,493 tok/s | 54.64 GB | 8.572B tok iter 65420 | train 2.811 | val 2.842 | lr 3.95e-04 | 102,495 tok/s | 54.64 GB | 8.575B tok iter 65440 | train 2.839 | val 2.848 | lr 3.95e-04 | 102,498 tok/s | 54.64 GB | 8.577B tok iter 65460 | train 2.736 | val 2.790 | lr 3.94e-04 | 102,498 tok/s | 54.64 GB | 8.580B tok iter 65480 | train 2.944 | val 2.864 | lr 3.94e-04 | 102,501 tok/s | 54.64 GB | 8.583B tok checkpoint @ iter 65491 -> checkpoints/500m_latest.pt iter 65500 | train 2.841 | val 2.824 | lr 3.94e-04 | 102,487 tok/s | 54.64 GB | 8.585B tok checkpoint @ iter 65500 -> checkpoints/500m_latest.pt iter 65520 | train 2.817 | val 2.784 | lr 3.94e-04 | 102,456 tok/s | 54.64 GB | 8.588B tok iter 65540 | train 2.817 | val 2.822 | lr 3.94e-04 | 102,457 tok/s | 54.64 GB | 8.591B tok iter 65560 | train 2.835 | val 2.827 | lr 3.94e-04 | 102,459 tok/s | 54.64 GB | 8.593B tok iter 65580 | train 2.747 | val 2.813 | lr 3.94e-04 | 102,462 tok/s | 54.64 GB | 8.596B tok iter 65600 | train 2.828 | val 2.810 | lr 3.94e-04 | 102,464 tok/s | 54.64 GB | 8.598B tok iter 65620 | train 2.711 | val 2.799 | lr 3.94e-04 | 102,464 tok/s | 54.64 GB | 8.601B tok iter 65640 | train 2.840 | val 2.863 | lr 3.94e-04 | 102,467 tok/s | 54.64 GB | 8.604B tok iter 65660 | train 2.759 | val 2.775 | lr 3.93e-04 | 102,469 tok/s | 54.64 GB | 8.606B tok iter 65680 | train 3.099 | val 2.862 | lr 3.93e-04 | 102,472 tok/s | 54.64 GB | 8.609B tok iter 65700 | train 3.011 | val 2.797 | lr 3.93e-04 | 102,472 tok/s | 54.64 GB | 8.612B tok iter 65720 | train 2.711 | val 2.824 | lr 3.93e-04 | 102,475 tok/s | 54.64 GB | 8.614B tok iter 65740 | train 2.908 | val 2.839 | lr 3.93e-04 | 102,477 tok/s | 54.64 GB | 8.617B tok iter 65760 | train 2.781 | val 2.820 | lr 3.93e-04 | 102,480 tok/s | 54.64 GB | 8.619B tok iter 65780 | train 2.834 | val 2.828 | lr 3.93e-04 | 102,480 tok/s | 54.64 GB | 8.622B tok iter 65800 | train 2.773 | val 2.839 | lr 3.93e-04 | 102,482 tok/s | 54.64 GB | 8.625B tok iter 65820 | train 2.963 | val 2.841 | lr 3.93e-04 | 102,485 tok/s | 54.64 GB | 8.627B tok iter 65840 | train 2.749 | val 2.811 | lr 3.92e-04 | 102,487 tok/s | 54.64 GB | 8.630B tok iter 65860 | train 2.674 | val 2.792 | lr 3.92e-04 | 102,488 tok/s | 54.64 GB | 8.633B tok iter 65880 | train 2.929 | val 2.850 | lr 3.92e-04 | 102,490 tok/s | 54.64 GB | 8.635B tok iter 65900 | train 2.810 | val 2.820 | lr 3.92e-04 | 102,493 tok/s | 54.64 GB | 8.638B tok iter 65920 | train 2.764 | val 2.821 | lr 3.92e-04 | 102,495 tok/s | 54.64 GB | 8.640B tok iter 65940 | train 3.030 | val 2.810 | lr 3.92e-04 | 102,496 tok/s | 54.64 GB | 8.643B tok iter 65960 | train 2.857 | val 2.801 | lr 3.92e-04 | 102,498 tok/s | 54.64 GB | 8.646B tok iter 65980 | train 2.787 | val 2.872 | lr 3.92e-04 | 102,500 tok/s | 54.64 GB | 8.648B tok checkpoint @ iter 65990 -> checkpoints/500m_latest.pt iter 66000 | train 2.767 | val 2.865 | lr 3.92e-04 | 102,490 tok/s | 54.64 GB | 8.651B tok checkpoint @ iter 66000 -> checkpoints/500m_latest.pt iter 66020 | train 2.805 | val 2.857 | lr 3.91e-04 | 102,458 tok/s | 54.64 GB | 8.654B tok iter 66040 | train 2.848 | val 2.861 | lr 3.91e-04 | 102,460 tok/s | 54.64 GB | 8.656B tok iter 66060 | train 2.826 | val 2.824 | lr 3.91e-04 | 102,463 tok/s | 54.64 GB | 8.659B tok iter 66080 | train 2.783 | val 2.823 | lr 3.91e-04 | 102,465 tok/s | 54.64 GB | 8.661B tok iter 66100 | train 2.880 | val 2.821 | lr 3.91e-04 | 102,466 tok/s | 54.64 GB | 8.664B tok iter 66120 | train 2.736 | val 2.781 | lr 3.91e-04 | 102,468 tok/s | 54.64 GB | 8.667B tok iter 66140 | train 2.961 | val 2.844 | lr 3.91e-04 | 102,470 tok/s | 54.64 GB | 8.669B tok iter 66160 | train 2.960 | val 2.795 | lr 3.91e-04 | 102,473 tok/s | 54.64 GB | 8.672B tok iter 66180 | train 2.855 | val 2.799 | lr 3.91e-04 | 102,473 tok/s | 54.64 GB | 8.674B tok iter 66200 | train 2.842 | val 2.872 | lr 3.90e-04 | 102,476 tok/s | 54.64 GB | 8.677B tok iter 66220 | train 2.949 | val 2.788 | lr 3.90e-04 | 102,478 tok/s | 54.64 GB | 8.680B tok iter 66240 | train 2.837 | val 2.841 | lr 3.90e-04 | 102,481 tok/s | 54.64 GB | 8.682B tok iter 66260 | train 2.845 | val 2.831 | lr 3.90e-04 | 102,481 tok/s | 54.64 GB | 8.685B tok iter 66280 | train 2.718 | val 2.824 | lr 3.90e-04 | 102,483 tok/s | 54.64 GB | 8.688B tok iter 66300 | train 2.803 | val 2.848 | lr 3.90e-04 | 102,486 tok/s | 54.64 GB | 8.690B tok iter 66320 | train 2.888 | val 2.820 | lr 3.90e-04 | 102,488 tok/s | 54.64 GB | 8.693B tok iter 66340 | train 2.808 | val 2.825 | lr 3.90e-04 | 102,489 tok/s | 54.64 GB | 8.695B tok iter 66360 | train 2.881 | val 2.835 | lr 3.90e-04 | 102,491 tok/s | 54.64 GB | 8.698B tok iter 66380 | train 2.865 | val 2.822 | lr 3.89e-04 | 102,494 tok/s | 54.64 GB | 8.701B tok iter 66400 | train 2.819 | val 2.879 | lr 3.89e-04 | 102,496 tok/s | 54.64 GB | 8.703B tok iter 66420 | train 2.810 | val 2.770 | lr 3.89e-04 | 102,496 tok/s | 54.64 GB | 8.706B tok iter 66440 | train 2.860 | val 2.814 | lr 3.89e-04 | 102,499 tok/s | 54.64 GB | 8.709B tok iter 66460 | train 2.937 | val 2.834 | lr 3.89e-04 | 102,501 tok/s | 54.64 GB | 8.711B tok iter 66480 | train 2.884 | val 2.821 | lr 3.89e-04 | 102,504 tok/s | 54.64 GB | 8.714B tok checkpoint @ iter 66490 -> checkpoints/500m_latest.pt iter 66500 | train 2.915 | val 2.835 | lr 3.89e-04 | 102,488 tok/s | 54.64 GB | 8.716B tok checkpoint @ iter 66500 -> checkpoints/500m_latest.pt iter 66520 | train 2.775 | val 2.827 | lr 3.89e-04 | 102,457 tok/s | 54.64 GB | 8.719B tok iter 66540 | train 2.879 | val 2.777 | lr 3.89e-04 | 102,460 tok/s | 54.64 GB | 8.722B tok iter 66560 | train 2.818 | val 2.833 | lr 3.88e-04 | 102,462 tok/s | 54.64 GB | 8.724B tok iter 66580 | train 2.866 | val 2.879 | lr 3.88e-04 | 102,463 tok/s | 54.64 GB | 8.727B tok iter 66600 | train 2.768 | val 2.793 | lr 3.88e-04 | 102,465 tok/s | 54.64 GB | 8.730B tok iter 66620 | train 2.743 | val 2.790 | lr 3.88e-04 | 102,467 tok/s | 54.64 GB | 8.732B tok iter 66640 | train 2.793 | val 2.841 | lr 3.88e-04 | 102,470 tok/s | 54.64 GB | 8.735B tok iter 66660 | train 2.857 | val 2.858 | lr 3.88e-04 | 102,470 tok/s | 54.64 GB | 8.737B tok iter 66680 | train 2.904 | val 2.791 | lr 3.88e-04 | 102,473 tok/s | 54.64 GB | 8.740B tok iter 66700 | train 2.985 | val 2.839 | lr 3.88e-04 | 102,475 tok/s | 54.64 GB | 8.743B tok iter 66720 | train 2.862 | val 2.832 | lr 3.88e-04 | 102,478 tok/s | 54.64 GB | 8.745B tok iter 66740 | train 2.754 | val 2.813 | lr 3.87e-04 | 102,478 tok/s | 54.64 GB | 8.748B tok iter 66760 | train 2.861 | val 2.836 | lr 3.87e-04 | 102,480 tok/s | 54.64 GB | 8.750B tok iter 66780 | train 2.750 | val 2.788 | lr 3.87e-04 | 102,483 tok/s | 54.64 GB | 8.753B tok iter 66800 | train 2.786 | val 2.777 | lr 3.87e-04 | 102,485 tok/s | 54.64 GB | 8.756B tok iter 66820 | train 2.865 | val 2.819 | lr 3.87e-04 | 102,485 tok/s | 54.64 GB | 8.758B tok iter 66840 | train 2.814 | val 2.857 | lr 3.87e-04 | 102,488 tok/s | 54.64 GB | 8.761B tok iter 66860 | train 2.840 | val 2.851 | lr 3.87e-04 | 102,490 tok/s | 54.64 GB | 8.764B tok iter 66880 | train 2.839 | val 2.815 | lr 3.87e-04 | 102,492 tok/s | 54.64 GB | 8.766B tok iter 66900 | train 2.880 | val 2.818 | lr 3.87e-04 | 102,493 tok/s | 54.64 GB | 8.769B tok iter 66920 | train 2.838 | val 2.767 | lr 3.86e-04 | 102,495 tok/s | 54.64 GB | 8.771B tok iter 66940 | train 2.779 | val 2.831 | lr 3.86e-04 | 102,498 tok/s | 54.64 GB | 8.774B tok iter 66960 | train 2.759 | val 2.903 | lr 3.86e-04 | 102,500 tok/s | 54.64 GB | 8.777B tok iter 66980 | train 2.808 | val 2.789 | lr 3.86e-04 | 102,500 tok/s | 54.64 GB | 8.779B tok checkpoint @ iter 66989 -> checkpoints/500m_latest.pt iter 67000 | train 2.916 | val 2.821 | lr 3.86e-04 | 102,487 tok/s | 54.64 GB | 8.782B tok checkpoint @ iter 67000 -> checkpoints/500m_latest.pt iter 67020 | train 2.833 | val 2.809 | lr 3.86e-04 | 102,456 tok/s | 54.64 GB | 8.785B tok iter 67040 | train 2.841 | val 2.804 | lr 3.86e-04 | 102,458 tok/s | 54.64 GB | 8.787B tok iter 67060 | train 2.809 | val 2.842 | lr 3.86e-04 | 102,458 tok/s | 54.64 GB | 8.790B tok iter 67080 | train 2.770 | val 2.829 | lr 3.86e-04 | 102,461 tok/s | 54.64 GB | 8.792B tok iter 67100 | train 2.802 | val 2.840 | lr 3.86e-04 | 102,463 tok/s | 54.64 GB | 8.795B tok iter 67120 | train 2.968 | val 2.811 | lr 3.85e-04 | 102,466 tok/s | 54.64 GB | 8.798B tok iter 67140 | train 2.816 | val 2.829 | lr 3.85e-04 | 102,466 tok/s | 54.64 GB | 8.800B tok iter 67160 | train 2.868 | val 2.828 | lr 3.85e-04 | 102,468 tok/s | 54.64 GB | 8.803B tok iter 67180 | train 2.791 | val 2.867 | lr 3.85e-04 | 102,471 tok/s | 54.64 GB | 8.806B tok iter 67200 | train 2.864 | val 2.802 | lr 3.85e-04 | 102,473 tok/s | 54.64 GB | 8.808B tok iter 67220 | train 2.703 | val 2.817 | lr 3.85e-04 | 102,474 tok/s | 54.64 GB | 8.811B tok iter 67240 | train 2.758 | val 2.772 | lr 3.85e-04 | 102,476 tok/s | 54.64 GB | 8.813B tok iter 67260 | train 2.900 | val 2.753 | lr 3.85e-04 | 102,478 tok/s | 54.64 GB | 8.816B tok iter 67280 | train 2.916 | val 2.814 | lr 3.85e-04 | 102,481 tok/s | 54.64 GB | 8.819B tok iter 67300 | train 2.748 | val 2.829 | lr 3.84e-04 | 102,481 tok/s | 54.64 GB | 8.821B tok iter 67320 | train 2.722 | val 2.821 | lr 3.84e-04 | 102,484 tok/s | 54.64 GB | 8.824B tok iter 67340 | train 2.957 | val 2.805 | lr 3.84e-04 | 102,486 tok/s | 54.64 GB | 8.827B tok iter 67360 | train 2.763 | val 2.816 | lr 3.84e-04 | 102,488 tok/s | 54.64 GB | 8.829B tok iter 67380 | train 2.757 | val 2.790 | lr 3.84e-04 | 102,489 tok/s | 54.64 GB | 8.832B tok iter 67400 | train 2.725 | val 2.817 | lr 3.84e-04 | 102,491 tok/s | 54.64 GB | 8.834B tok iter 67420 | train 2.878 | val 2.840 | lr 3.84e-04 | 102,493 tok/s | 54.64 GB | 8.837B tok iter 67440 | train 2.793 | val 2.785 | lr 3.84e-04 | 102,496 tok/s | 54.64 GB | 8.840B tok iter 67460 | train 2.806 | val 2.792 | lr 3.84e-04 | 102,496 tok/s | 54.64 GB | 8.842B tok iter 67480 | train 2.724 | val 2.822 | lr 3.83e-04 | 102,499 tok/s | 54.64 GB | 8.845B tok checkpoint @ iter 67490 -> checkpoints/500m_latest.pt iter 67500 | train 2.794 | val 2.820 | lr 3.83e-04 | 102,485 tok/s | 54.64 GB | 8.847B tok checkpoint @ iter 67500 -> checkpoints/500m_latest.pt iter 67520 | train 2.772 | val 2.778 | lr 3.83e-04 | 102,456 tok/s | 54.64 GB | 8.850B tok iter 67540 | train 2.773 | val 2.839 | lr 3.83e-04 | 102,456 tok/s | 54.64 GB | 8.853B tok iter 67560 | train 3.011 | val 2.823 | lr 3.83e-04 | 102,459 tok/s | 54.64 GB | 8.855B tok iter 67580 | train 2.832 | val 2.801 | lr 3.83e-04 | 102,461 tok/s | 54.64 GB | 8.858B tok iter 67600 | train 2.675 | val 2.824 | lr 3.83e-04 | 102,463 tok/s | 54.64 GB | 8.861B tok iter 67620 | train 2.884 | val 2.826 | lr 3.83e-04 | 102,464 tok/s | 54.64 GB | 8.863B tok iter 67640 | train 2.776 | val 2.866 | lr 3.83e-04 | 102,466 tok/s | 54.64 GB | 8.866B tok iter 67660 | train 2.728 | val 2.804 | lr 3.82e-04 | 102,469 tok/s | 54.64 GB | 8.868B tok iter 67680 | train 2.749 | val 2.794 | lr 3.82e-04 | 102,471 tok/s | 54.64 GB | 8.871B tok iter 67700 | train 2.839 | val 2.843 | lr 3.82e-04 | 102,471 tok/s | 54.64 GB | 8.874B tok iter 67720 | train 2.979 | val 2.821 | lr 3.82e-04 | 102,474 tok/s | 54.64 GB | 8.876B tok iter 67740 | train 2.782 | val 2.789 | lr 3.82e-04 | 102,476 tok/s | 54.64 GB | 8.879B tok iter 67760 | train 2.868 | val 2.788 | lr 3.82e-04 | 102,478 tok/s | 54.64 GB | 8.882B tok iter 67780 | train 2.857 | val 2.827 | lr 3.82e-04 | 102,479 tok/s | 54.64 GB | 8.884B tok iter 67800 | train 2.864 | val 2.826 | lr 3.82e-04 | 102,481 tok/s | 54.64 GB | 8.887B tok iter 67820 | train 2.801 | val 2.869 | lr 3.82e-04 | 102,483 tok/s | 54.64 GB | 8.889B tok iter 67840 | train 2.786 | val 2.781 | lr 3.81e-04 | 102,486 tok/s | 54.64 GB | 8.892B tok iter 67860 | train 2.700 | val 2.776 | lr 3.81e-04 | 102,486 tok/s | 54.64 GB | 8.895B tok iter 67880 | train 2.667 | val 2.750 | lr 3.81e-04 | 102,489 tok/s | 54.64 GB | 8.897B tok iter 67900 | train 2.898 | val 2.869 | lr 3.81e-04 | 102,491 tok/s | 54.64 GB | 8.900B tok iter 67920 | train 2.937 | val 2.831 | lr 3.81e-04 | 102,493 tok/s | 54.64 GB | 8.903B tok iter 67940 | train 2.819 | val 2.801 | lr 3.81e-04 | 102,494 tok/s | 54.64 GB | 8.905B tok iter 67960 | train 2.849 | val 2.812 | lr 3.81e-04 | 102,496 tok/s | 54.64 GB | 8.908B tok iter 67980 | train 2.837 | val 2.843 | lr 3.81e-04 | 102,498 tok/s | 54.64 GB | 8.910B tok checkpoint @ iter 67990 -> checkpoints/500m_latest.pt iter 68000 | train 2.792 | val 2.842 | lr 3.81e-04 | 102,485 tok/s | 54.64 GB | 8.913B tok checkpoint @ iter 68000 -> checkpoints/500m_latest.pt iter 68020 | train 2.689 | val 2.761 | lr 3.80e-04 | 102,456 tok/s | 54.64 GB | 8.916B tok iter 68040 | train 2.833 | val 2.813 | lr 3.80e-04 | 102,458 tok/s | 54.64 GB | 8.918B tok iter 68060 | train 2.878 | val 2.853 | lr 3.80e-04 | 102,460 tok/s | 54.64 GB | 8.921B tok iter 68080 | train 2.802 | val 2.826 | lr 3.80e-04 | 102,463 tok/s | 54.64 GB | 8.924B tok iter 68100 | train 2.823 | val 2.844 | lr 3.80e-04 | 102,463 tok/s | 54.64 GB | 8.926B tok iter 68120 | train 2.830 | val 2.808 | lr 3.80e-04 | 102,465 tok/s | 54.64 GB | 8.929B tok iter 68140 | train 2.842 | val 2.833 | lr 3.80e-04 | 102,468 tok/s | 54.64 GB | 8.931B tok iter 68160 | train 2.909 | val 2.850 | lr 3.80e-04 | 102,470 tok/s | 54.64 GB | 8.934B tok iter 68180 | train 2.788 | val 2.809 | lr 3.80e-04 | 102,470 tok/s | 54.64 GB | 8.937B tok iter 68200 | train 2.839 | val 2.786 | lr 3.79e-04 | 102,473 tok/s | 54.64 GB | 8.939B tok iter 68220 | train 2.876 | val 2.818 | lr 3.79e-04 | 102,475 tok/s | 54.64 GB | 8.942B tok iter 68240 | train 2.744 | val 2.837 | lr 3.79e-04 | 102,478 tok/s | 54.64 GB | 8.944B tok iter 68260 | train 2.773 | val 2.771 | lr 3.79e-04 | 102,478 tok/s | 54.64 GB | 8.947B tok iter 68280 | train 2.832 | val 2.830 | lr 3.79e-04 | 102,480 tok/s | 54.64 GB | 8.950B tok iter 68300 | train 2.847 | val 2.771 | lr 3.79e-04 | 102,483 tok/s | 54.64 GB | 8.952B tok iter 68320 | train 2.922 | val 2.854 | lr 3.79e-04 | 102,485 tok/s | 54.64 GB | 8.955B tok iter 68340 | train 2.753 | val 2.842 | lr 3.79e-04 | 102,485 tok/s | 54.64 GB | 8.958B tok iter 68360 | train 2.733 | val 2.818 | lr 3.79e-04 | 102,488 tok/s | 54.64 GB | 8.960B tok iter 68380 | train 2.918 | val 2.790 | lr 3.78e-04 | 102,490 tok/s | 54.64 GB | 8.963B tok iter 68400 | train 2.741 | val 2.808 | lr 3.78e-04 | 102,492 tok/s | 54.64 GB | 8.965B tok iter 68420 | train 2.811 | val 2.799 | lr 3.78e-04 | 102,493 tok/s | 54.64 GB | 8.968B tok iter 68440 | train 2.867 | val 2.834 | lr 3.78e-04 | 102,495 tok/s | 54.64 GB | 8.971B tok iter 68460 | train 2.757 | val 2.792 | lr 3.78e-04 | 102,497 tok/s | 54.64 GB | 8.973B tok iter 68480 | train 2.728 | val 2.790 | lr 3.78e-04 | 102,500 tok/s | 54.64 GB | 8.976B tok checkpoint @ iter 68491 -> checkpoints/500m_latest.pt iter 68500 | train 2.813 | val 2.816 | lr 3.78e-04 | 102,485 tok/s | 54.64 GB | 8.979B tok checkpoint @ iter 68500 -> checkpoints/500m_latest.pt iter 68520 | train 2.778 | val 2.858 | lr 3.78e-04 | 102,458 tok/s | 54.64 GB | 8.981B tok iter 68540 | train 2.788 | val 2.819 | lr 3.78e-04 | 102,460 tok/s | 54.64 GB | 8.984B tok iter 68560 | train 2.882 | val 2.848 | lr 3.77e-04 | 102,462 tok/s | 54.64 GB | 8.986B tok iter 68580 | train 2.795 | val 2.828 | lr 3.77e-04 | 102,463 tok/s | 54.64 GB | 8.989B tok iter 68600 | train 2.750 | val 2.799 | lr 3.77e-04 | 102,465 tok/s | 54.64 GB | 8.992B tok iter 68620 | train 2.713 | val 2.824 | lr 3.77e-04 | 102,467 tok/s | 54.64 GB | 8.994B tok iter 68640 | train 2.930 | val 2.833 | lr 3.77e-04 | 102,470 tok/s | 54.64 GB | 8.997B tok iter 68660 | train 2.655 | val 2.842 | lr 3.77e-04 | 102,470 tok/s | 54.64 GB | 9.000B tok iter 68680 | train 2.822 | val 2.825 | lr 3.77e-04 | 102,473 tok/s | 54.64 GB | 9.002B tok iter 68700 | train 2.915 | val 2.824 | lr 3.77e-04 | 102,475 tok/s | 54.64 GB | 9.005B tok iter 68720 | train 2.595 | val 2.838 | lr 3.77e-04 | 102,477 tok/s | 54.64 GB | 9.007B tok iter 68740 | train 2.743 | val 2.829 | lr 3.76e-04 | 102,478 tok/s | 54.64 GB | 9.010B tok iter 68760 | train 2.750 | val 2.796 | lr 3.76e-04 | 102,480 tok/s | 54.64 GB | 9.013B tok iter 68780 | train 2.804 | val 2.857 | lr 3.76e-04 | 102,482 tok/s | 54.64 GB | 9.015B tok iter 68800 | train 2.762 | val 2.812 | lr 3.76e-04 | 102,485 tok/s | 54.64 GB | 9.018B tok iter 68820 | train 2.830 | val 2.801 | lr 3.76e-04 | 102,485 tok/s | 54.64 GB | 9.021B tok iter 68840 | train 2.762 | val 2.838 | lr 3.76e-04 | 102,487 tok/s | 54.64 GB | 9.023B tok iter 68860 | train 2.846 | val 2.835 | lr 3.76e-04 | 102,490 tok/s | 54.64 GB | 9.026B tok iter 68880 | train 2.808 | val 2.783 | lr 3.76e-04 | 102,492 tok/s | 54.64 GB | 9.028B tok iter 68900 | train 2.745 | val 2.825 | lr 3.76e-04 | 102,493 tok/s | 54.64 GB | 9.031B tok iter 68920 | train 2.775 | val 2.820 | lr 3.75e-04 | 102,495 tok/s | 54.64 GB | 9.034B tok iter 68940 | train 2.831 | val 2.805 | lr 3.75e-04 | 102,497 tok/s | 54.64 GB | 9.036B tok iter 68960 | train 2.822 | val 2.815 | lr 3.75e-04 | 102,500 tok/s | 54.64 GB | 9.039B tok iter 68980 | train 2.825 | val 2.790 | lr 3.75e-04 | 102,500 tok/s | 54.64 GB | 9.041B tok checkpoint @ iter 68991 -> checkpoints/500m_latest.pt iter 69000 | train 2.785 | val 2.803 | lr 3.75e-04 | 102,487 tok/s | 54.64 GB | 9.044B tok checkpoint @ iter 69000 -> checkpoints/500m_latest.pt iter 69020 | train 2.684 | val 2.818 | lr 3.75e-04 | 102,458 tok/s | 54.64 GB | 9.047B tok iter 69040 | train 2.912 | val 2.814 | lr 3.75e-04 | 102,461 tok/s | 54.64 GB | 9.049B tok iter 69060 | train 2.912 | val 2.811 | lr 3.75e-04 | 102,461 tok/s | 54.64 GB | 9.052B tok iter 69080 | train 2.750 | val 2.842 | lr 3.75e-04 | 102,464 tok/s | 54.64 GB | 9.055B tok iter 69100 | train 2.744 | val 2.804 | lr 3.74e-04 | 102,466 tok/s | 54.64 GB | 9.057B tok iter 69120 | train 2.852 | val 2.833 | lr 3.74e-04 | 102,468 tok/s | 54.64 GB | 9.060B tok iter 69140 | train 2.777 | val 2.812 | lr 3.74e-04 | 102,469 tok/s | 54.64 GB | 9.062B tok iter 69160 | train 2.797 | val 2.852 | lr 3.74e-04 | 102,471 tok/s | 54.64 GB | 9.065B tok iter 69180 | train 2.784 | val 2.819 | lr 3.74e-04 | 102,474 tok/s | 54.64 GB | 9.068B tok iter 69200 | train 2.899 | val 2.838 | lr 3.74e-04 | 102,476 tok/s | 54.64 GB | 9.070B tok iter 69220 | train 2.755 | val 2.809 | lr 3.74e-04 | 102,476 tok/s | 54.64 GB | 9.073B tok iter 69240 | train 2.922 | val 2.865 | lr 3.74e-04 | 102,479 tok/s | 54.64 GB | 9.076B tok iter 69260 | train 2.874 | val 2.838 | lr 3.74e-04 | 102,481 tok/s | 54.64 GB | 9.078B tok iter 69280 | train 2.878 | val 2.839 | lr 3.73e-04 | 102,483 tok/s | 54.64 GB | 9.081B tok iter 69300 | train 2.812 | val 2.832 | lr 3.73e-04 | 102,484 tok/s | 54.64 GB | 9.083B tok iter 69320 | train 2.774 | val 2.767 | lr 3.73e-04 | 102,486 tok/s | 54.64 GB | 9.086B tok iter 69340 | train 2.671 | val 2.799 | lr 3.73e-04 | 102,489 tok/s | 54.64 GB | 9.089B tok iter 69360 | train 2.710 | val 2.797 | lr 3.73e-04 | 102,491 tok/s | 54.64 GB | 9.091B tok iter 69380 | train 2.790 | val 2.812 | lr 3.73e-04 | 102,491 tok/s | 54.64 GB | 9.094B tok iter 69400 | train 2.877 | val 2.784 | lr 3.73e-04 | 102,493 tok/s | 54.64 GB | 9.097B tok iter 69420 | train 2.735 | val 2.813 | lr 3.73e-04 | 102,496 tok/s | 54.64 GB | 9.099B tok iter 69440 | train 2.973 | val 2.785 | lr 3.73e-04 | 102,498 tok/s | 54.64 GB | 9.102B tok iter 69460 | train 2.674 | val 2.818 | lr 3.72e-04 | 102,498 tok/s | 54.64 GB | 9.104B tok iter 69480 | train 2.744 | val 2.876 | lr 3.72e-04 | 102,501 tok/s | 54.64 GB | 9.107B tok checkpoint @ iter 69491 -> checkpoints/500m_latest.pt iter 69500 | train 2.885 | val 2.760 | lr 3.72e-04 | 102,488 tok/s | 54.64 GB | 9.110B tok checkpoint @ iter 69500 -> checkpoints/500m_latest.pt iter 69520 | train 2.918 | val 2.784 | lr 3.72e-04 | 102,459 tok/s | 54.64 GB | 9.112B tok iter 69540 | train 2.772 | val 2.814 | lr 3.72e-04 | 102,459 tok/s | 54.64 GB | 9.115B tok iter 69560 | train 2.696 | val 2.797 | lr 3.72e-04 | 102,462 tok/s | 54.64 GB | 9.117B tok iter 69580 | train 2.914 | val 2.823 | lr 3.72e-04 | 102,464 tok/s | 54.64 GB | 9.120B tok iter 69600 | train 2.750 | val 2.807 | lr 3.72e-04 | 102,466 tok/s | 54.64 GB | 9.123B tok iter 69620 | train 2.847 | val 2.823 | lr 3.72e-04 | 102,467 tok/s | 54.64 GB | 9.125B tok iter 69640 | train 2.927 | val 2.797 | lr 3.71e-04 | 102,469 tok/s | 54.64 GB | 9.128B tok iter 69660 | train 2.738 | val 2.853 | lr 3.71e-04 | 102,471 tok/s | 54.64 GB | 9.131B tok iter 69680 | train 2.801 | val 2.809 | lr 3.71e-04 | 102,474 tok/s | 54.64 GB | 9.133B tok iter 69700 | train 2.847 | val 2.796 | lr 3.71e-04 | 102,474 tok/s | 54.64 GB | 9.136B tok iter 69720 | train 2.748 | val 2.818 | lr 3.71e-04 | 102,476 tok/s | 54.64 GB | 9.138B tok iter 69740 | train 2.904 | val 2.744 | lr 3.71e-04 | 102,479 tok/s | 54.64 GB | 9.141B tok iter 69760 | train 2.799 | val 2.783 | lr 3.71e-04 | 102,481 tok/s | 54.64 GB | 9.144B tok iter 69780 | train 2.829 | val 2.864 | lr 3.71e-04 | 102,481 tok/s | 54.64 GB | 9.146B tok iter 69800 | train 2.776 | val 2.794 | lr 3.71e-04 | 102,484 tok/s | 54.64 GB | 9.149B tok iter 69820 | train 2.771 | val 2.805 | lr 3.70e-04 | 102,486 tok/s | 54.64 GB | 9.152B tok iter 69840 | train 2.679 | val 2.810 | lr 3.70e-04 | 102,488 tok/s | 54.64 GB | 9.154B tok iter 69860 | train 2.648 | val 2.830 | lr 3.70e-04 | 102,489 tok/s | 54.64 GB | 9.157B tok iter 69880 | train 2.824 | val 2.792 | lr 3.70e-04 | 102,491 tok/s | 54.64 GB | 9.159B tok iter 69900 | train 2.917 | val 2.775 | lr 3.70e-04 | 102,493 tok/s | 54.64 GB | 9.162B tok iter 69920 | train 2.804 | val 2.797 | lr 3.70e-04 | 102,496 tok/s | 54.64 GB | 9.165B tok iter 69940 | train 2.881 | val 2.830 | lr 3.70e-04 | 102,496 tok/s | 54.64 GB | 9.167B tok iter 69960 | train 2.941 | val 2.792 | lr 3.70e-04 | 102,498 tok/s | 54.64 GB | 9.170B tok iter 69980 | train 2.673 | val 2.829 | lr 3.70e-04 | 102,501 tok/s | 54.64 GB | 9.173B tok checkpoint @ iter 69990 -> checkpoints/500m_latest.pt iter 70000 | train 2.799 | val 2.809 | lr 3.69e-04 | 102,488 tok/s | 54.64 GB | 9.175B tok checkpoint @ iter 70000 -> checkpoints/500m_latest.pt iter 70020 | train 2.796 | val 2.819 | lr 3.69e-04 | 102,458 tok/s | 54.64 GB | 9.178B tok iter 70040 | train 2.932 | val 2.793 | lr 3.69e-04 | 102,461 tok/s | 54.64 GB | 9.180B tok iter 70060 | train 2.910 | val 2.812 | lr 3.69e-04 | 102,463 tok/s | 54.64 GB | 9.183B tok iter 70080 | train 2.770 | val 2.765 | lr 3.69e-04 | 102,465 tok/s | 54.64 GB | 9.186B tok iter 70100 | train 2.918 | val 2.799 | lr 3.69e-04 | 102,466 tok/s | 54.64 GB | 9.188B tok iter 70120 | train 2.826 | val 2.849 | lr 3.69e-04 | 102,468 tok/s | 54.64 GB | 9.191B tok iter 70140 | train 2.704 | val 2.806 | lr 3.69e-04 | 102,470 tok/s | 54.64 GB | 9.194B tok iter 70160 | train 2.832 | val 2.760 | lr 3.69e-04 | 102,473 tok/s | 54.64 GB | 9.196B tok iter 70180 | train 2.796 | val 2.823 | lr 3.68e-04 | 102,473 tok/s | 54.64 GB | 9.199B tok iter 70200 | train 2.886 | val 2.795 | lr 3.68e-04 | 102,475 tok/s | 54.64 GB | 9.201B tok iter 70220 | train 2.804 | val 2.780 | lr 3.68e-04 | 102,478 tok/s | 54.64 GB | 9.204B tok iter 70240 | train 2.924 | val 2.836 | lr 3.68e-04 | 102,480 tok/s | 54.64 GB | 9.207B tok iter 70260 | train 2.722 | val 2.740 | lr 3.68e-04 | 102,480 tok/s | 54.64 GB | 9.209B tok iter 70280 | train 2.875 | val 2.802 | lr 3.68e-04 | 102,482 tok/s | 54.64 GB | 9.212B tok iter 70300 | train 2.861 | val 2.807 | lr 3.68e-04 | 102,485 tok/s | 54.64 GB | 9.214B tok iter 70320 | train 2.718 | val 2.825 | lr 3.68e-04 | 102,487 tok/s | 54.64 GB | 9.217B tok iter 70340 | train 2.862 | val 2.836 | lr 3.68e-04 | 102,487 tok/s | 54.64 GB | 9.220B tok iter 70360 | train 2.905 | val 2.761 | lr 3.67e-04 | 102,490 tok/s | 54.64 GB | 9.222B tok iter 70380 | train 2.899 | val 2.811 | lr 3.67e-04 | 102,492 tok/s | 54.64 GB | 9.225B tok iter 70400 | train 2.891 | val 2.814 | lr 3.67e-04 | 102,494 tok/s | 54.64 GB | 9.228B tok iter 70420 | train 2.846 | val 2.815 | lr 3.67e-04 | 102,495 tok/s | 54.64 GB | 9.230B tok iter 70440 | train 2.691 | val 2.878 | lr 3.67e-04 | 102,497 tok/s | 54.64 GB | 9.233B tok iter 70460 | train 2.849 | val 2.836 | lr 3.67e-04 | 102,499 tok/s | 54.64 GB | 9.235B tok iter 70480 | train 2.778 | val 2.858 | lr 3.67e-04 | 102,502 tok/s | 54.64 GB | 9.238B tok checkpoint @ iter 70490 -> checkpoints/500m_latest.pt iter 70500 | train 2.886 | val 2.825 | lr 3.67e-04 | 102,488 tok/s | 54.64 GB | 9.241B tok checkpoint @ iter 70500 -> checkpoints/500m_latest.pt iter 70520 | train 2.755 | val 2.817 | lr 3.67e-04 | 102,460 tok/s | 54.64 GB | 9.243B tok iter 70540 | train 2.725 | val 2.849 | lr 3.66e-04 | 102,463 tok/s | 54.64 GB | 9.246B tok iter 70560 | train 2.914 | val 2.754 | lr 3.66e-04 | 102,465 tok/s | 54.64 GB | 9.249B tok iter 70580 | train 2.879 | val 2.783 | lr 3.66e-04 | 102,465 tok/s | 54.64 GB | 9.251B tok iter 70600 | train 2.758 | val 2.808 | lr 3.66e-04 | 102,468 tok/s | 54.64 GB | 9.254B tok iter 70620 | train 2.875 | val 2.761 | lr 3.66e-04 | 102,470 tok/s | 54.64 GB | 9.256B tok iter 70640 | train 2.837 | val 2.806 | lr 3.66e-04 | 102,472 tok/s | 54.64 GB | 9.259B tok iter 70660 | train 2.833 | val 2.876 | lr 3.66e-04 | 102,473 tok/s | 54.64 GB | 9.262B tok iter 70680 | train 2.843 | val 2.848 | lr 3.66e-04 | 102,475 tok/s | 54.64 GB | 9.264B tok iter 70700 | train 2.905 | val 2.786 | lr 3.66e-04 | 102,477 tok/s | 54.64 GB | 9.267B tok iter 70720 | train 2.835 | val 2.788 | lr 3.65e-04 | 102,480 tok/s | 54.64 GB | 9.270B tok iter 70740 | train 2.666 | val 2.785 | lr 3.65e-04 | 102,480 tok/s | 54.64 GB | 9.272B tok iter 70760 | train 2.710 | val 2.795 | lr 3.65e-04 | 102,482 tok/s | 54.64 GB | 9.275B tok iter 70780 | train 2.730 | val 2.750 | lr 3.65e-04 | 102,485 tok/s | 54.64 GB | 9.277B tok iter 70800 | train 2.741 | val 2.752 | lr 3.65e-04 | 102,487 tok/s | 54.64 GB | 9.280B tok iter 70820 | train 2.700 | val 2.835 | lr 3.65e-04 | 102,487 tok/s | 54.64 GB | 9.283B tok iter 70840 | train 2.749 | val 2.790 | lr 3.65e-04 | 102,490 tok/s | 54.64 GB | 9.285B tok iter 70860 | train 2.915 | val 2.814 | lr 3.65e-04 | 102,492 tok/s | 54.64 GB | 9.288B tok iter 70880 | train 2.726 | val 2.877 | lr 3.65e-04 | 102,494 tok/s | 54.64 GB | 9.291B tok iter 70900 | train 2.829 | val 2.834 | lr 3.64e-04 | 102,494 tok/s | 54.64 GB | 9.293B tok iter 70920 | train 2.772 | val 2.862 | lr 3.64e-04 | 102,497 tok/s | 54.64 GB | 9.296B tok iter 70940 | train 2.827 | val 2.787 | lr 3.64e-04 | 102,499 tok/s | 54.64 GB | 9.298B tok iter 70960 | train 2.805 | val 2.844 | lr 3.64e-04 | 102,501 tok/s | 54.64 GB | 9.301B tok iter 70980 | train 2.693 | val 2.792 | lr 3.64e-04 | 102,502 tok/s | 54.64 GB | 9.304B tok checkpoint @ iter 70991 -> checkpoints/500m_latest.pt iter 71000 | train 2.629 | val 2.765 | lr 3.64e-04 | 102,490 tok/s | 54.64 GB | 9.306B tok checkpoint @ iter 71000 -> checkpoints/500m_latest.pt iter 71020 | train 2.741 | val 2.779 | lr 3.64e-04 | 102,462 tok/s | 54.64 GB | 9.309B tok iter 71040 | train 2.853 | val 2.809 | lr 3.64e-04 | 102,464 tok/s | 54.64 GB | 9.311B tok iter 71060 | train 2.891 | val 2.851 | lr 3.64e-04 | 102,464 tok/s | 54.64 GB | 9.314B tok iter 71080 | train 2.720 | val 2.762 | lr 3.63e-04 | 102,466 tok/s | 54.64 GB | 9.317B tok iter 71100 | train 2.675 | val 2.823 | lr 3.63e-04 | 102,469 tok/s | 54.64 GB | 9.319B tok iter 71120 | train 2.964 | val 2.850 | lr 3.63e-04 | 102,471 tok/s | 54.64 GB | 9.322B tok iter 71140 | train 2.726 | val 2.776 | lr 3.63e-04 | 102,471 tok/s | 54.64 GB | 9.325B tok iter 71160 | train 2.858 | val 2.866 | lr 3.63e-04 | 102,474 tok/s | 54.64 GB | 9.327B tok iter 71180 | train 2.645 | val 2.807 | lr 3.63e-04 | 102,476 tok/s | 54.64 GB | 9.330B tok iter 71200 | train 2.672 | val 2.808 | lr 3.63e-04 | 102,478 tok/s | 54.64 GB | 9.332B tok iter 71220 | train 2.746 | val 2.783 | lr 3.63e-04 | 102,479 tok/s | 54.64 GB | 9.335B tok iter 71240 | train 2.733 | val 2.859 | lr 3.63e-04 | 102,481 tok/s | 54.64 GB | 9.338B tok iter 71260 | train 2.828 | val 2.769 | lr 3.62e-04 | 102,483 tok/s | 54.64 GB | 9.340B tok iter 71280 | train 2.747 | val 2.803 | lr 3.62e-04 | 102,485 tok/s | 54.64 GB | 9.343B tok iter 71300 | train 2.731 | val 2.833 | lr 3.62e-04 | 102,486 tok/s | 54.64 GB | 9.346B tok iter 71320 | train 2.655 | val 2.797 | lr 3.62e-04 | 102,488 tok/s | 54.64 GB | 9.348B tok iter 71340 | train 2.731 | val 2.860 | lr 3.62e-04 | 102,490 tok/s | 54.64 GB | 9.351B tok iter 71360 | train 2.849 | val 2.785 | lr 3.62e-04 | 102,493 tok/s | 54.64 GB | 9.353B tok iter 71380 | train 2.597 | val 2.802 | lr 3.62e-04 | 102,493 tok/s | 54.64 GB | 9.356B tok iter 71400 | train 2.869 | val 2.791 | lr 3.62e-04 | 102,495 tok/s | 54.64 GB | 9.359B tok iter 71420 | train 3.018 | val 2.813 | lr 3.62e-04 | 102,497 tok/s | 54.64 GB | 9.361B tok iter 71440 | train 2.837 | val 2.784 | lr 3.61e-04 | 102,500 tok/s | 54.64 GB | 9.364B tok iter 71460 | train 2.777 | val 2.788 | lr 3.61e-04 | 102,500 tok/s | 54.64 GB | 9.367B tok iter 71480 | train 2.826 | val 2.784 | lr 3.61e-04 | 102,502 tok/s | 54.64 GB | 9.369B tok checkpoint @ iter 71490 -> checkpoints/500m_latest.pt iter 71500 | train 2.719 | val 2.773 | lr 3.61e-04 | 102,489 tok/s | 54.64 GB | 9.372B tok checkpoint @ iter 71500 -> checkpoints/500m_latest.pt iter 71520 | train 2.663 | val 2.820 | lr 3.61e-04 | 102,461 tok/s | 54.64 GB | 9.374B tok iter 71540 | train 2.989 | val 2.781 | lr 3.61e-04 | 102,462 tok/s | 54.64 GB | 9.377B tok iter 71560 | train 2.905 | val 2.811 | lr 3.61e-04 | 102,464 tok/s | 54.64 GB | 9.380B tok iter 71580 | train 2.831 | val 2.792 | lr 3.61e-04 | 102,466 tok/s | 54.64 GB | 9.382B tok iter 71600 | train 2.873 | val 2.816 | lr 3.61e-04 | 102,469 tok/s | 54.64 GB | 9.385B tok iter 71620 | train 2.844 | val 2.837 | lr 3.60e-04 | 102,469 tok/s | 54.64 GB | 9.388B tok iter 71640 | train 2.727 | val 2.779 | lr 3.60e-04 | 102,471 tok/s | 54.64 GB | 9.390B tok iter 71660 | train 2.769 | val 2.802 | lr 3.60e-04 | 102,474 tok/s | 54.64 GB | 9.393B tok iter 71680 | train 2.966 | val 2.857 | lr 3.60e-04 | 102,476 tok/s | 54.64 GB | 9.395B tok iter 71700 | train 2.696 | val 2.853 | lr 3.60e-04 | 102,476 tok/s | 54.64 GB | 9.398B tok iter 71720 | train 2.792 | val 2.841 | lr 3.60e-04 | 102,478 tok/s | 54.64 GB | 9.401B tok iter 71740 | train 2.849 | val 2.784 | lr 3.60e-04 | 102,481 tok/s | 54.64 GB | 9.403B tok iter 71760 | train 2.774 | val 2.781 | lr 3.60e-04 | 102,483 tok/s | 54.64 GB | 9.406B tok iter 71780 | train 2.744 | val 2.840 | lr 3.60e-04 | 102,483 tok/s | 54.64 GB | 9.408B tok iter 71800 | train 2.808 | val 2.784 | lr 3.59e-04 | 102,485 tok/s | 54.64 GB | 9.411B tok iter 71820 | train 2.925 | val 2.801 | lr 3.59e-04 | 102,488 tok/s | 54.64 GB | 9.414B tok iter 71840 | train 2.774 | val 2.827 | lr 3.59e-04 | 102,490 tok/s | 54.64 GB | 9.416B tok iter 71860 | train 2.678 | val 2.798 | lr 3.59e-04 | 102,490 tok/s | 54.64 GB | 9.419B tok iter 71880 | train 2.812 | val 2.827 | lr 3.59e-04 | 102,493 tok/s | 54.64 GB | 9.422B tok iter 71900 | train 2.699 | val 2.822 | lr 3.59e-04 | 102,495 tok/s | 54.64 GB | 9.424B tok iter 71920 | train 2.774 | val 2.760 | lr 3.59e-04 | 102,497 tok/s | 54.64 GB | 9.427B tok iter 71940 | train 2.743 | val 2.803 | lr 3.59e-04 | 102,497 tok/s | 54.64 GB | 9.429B tok iter 71960 | train 2.866 | val 2.843 | lr 3.59e-04 | 102,499 tok/s | 54.64 GB | 9.432B tok iter 71980 | train 2.807 | val 2.805 | lr 3.58e-04 | 102,502 tok/s | 54.64 GB | 9.435B tok checkpoint @ iter 71990 -> checkpoints/500m_latest.pt iter 72000 | train 2.843 | val 2.787 | lr 3.58e-04 | 102,489 tok/s | 54.64 GB | 9.437B tok checkpoint @ iter 72000 -> checkpoints/500m_latest.pt iter 72020 | train 2.815 | val 2.811 | lr 3.58e-04 | 102,460 tok/s | 54.64 GB | 9.440B tok iter 72040 | train 2.832 | val 2.811 | lr 3.58e-04 | 102,462 tok/s | 54.64 GB | 9.443B tok iter 72060 | train 2.782 | val 2.805 | lr 3.58e-04 | 102,464 tok/s | 54.64 GB | 9.445B tok iter 72080 | train 2.835 | val 2.788 | lr 3.58e-04 | 102,467 tok/s | 54.64 GB | 9.448B tok iter 72100 | train 2.867 | val 2.766 | lr 3.58e-04 | 102,467 tok/s | 54.64 GB | 9.450B tok iter 72120 | train 2.651 | val 2.806 | lr 3.58e-04 | 102,469 tok/s | 54.64 GB | 9.453B tok iter 72140 | train 3.000 | val 2.792 | lr 3.58e-04 | 102,471 tok/s | 54.64 GB | 9.456B tok iter 72160 | train 2.826 | val 2.791 | lr 3.57e-04 | 102,474 tok/s | 54.64 GB | 9.458B tok iter 72180 | train 2.604 | val 2.852 | lr 3.57e-04 | 102,474 tok/s | 54.64 GB | 9.461B tok iter 72200 | train 2.830 | val 2.789 | lr 3.57e-04 | 102,476 tok/s | 54.64 GB | 9.464B tok iter 72220 | train 2.913 | val 2.805 | lr 3.57e-04 | 102,478 tok/s | 54.64 GB | 9.466B tok iter 72240 | train 2.818 | val 2.789 | lr 3.57e-04 | 102,481 tok/s | 54.64 GB | 9.469B tok iter 72260 | train 2.811 | val 2.794 | lr 3.57e-04 | 102,481 tok/s | 54.64 GB | 9.471B tok iter 72280 | train 2.716 | val 2.824 | lr 3.57e-04 | 102,483 tok/s | 54.64 GB | 9.474B tok iter 72300 | train 2.753 | val 2.786 | lr 3.57e-04 | 102,486 tok/s | 54.64 GB | 9.477B tok iter 72320 | train 2.755 | val 2.809 | lr 3.57e-04 | 102,488 tok/s | 54.64 GB | 9.479B tok iter 72340 | train 2.769 | val 2.837 | lr 3.56e-04 | 102,488 tok/s | 54.64 GB | 9.482B tok iter 72360 | train 2.714 | val 2.789 | lr 3.56e-04 | 102,490 tok/s | 54.64 GB | 9.485B tok iter 72380 | train 2.783 | val 2.767 | lr 3.56e-04 | 102,493 tok/s | 54.64 GB | 9.487B tok iter 72400 | train 2.794 | val 2.814 | lr 3.56e-04 | 102,495 tok/s | 54.64 GB | 9.490B tok iter 72420 | train 2.912 | val 2.823 | lr 3.56e-04 | 102,495 tok/s | 54.64 GB | 9.492B tok iter 72440 | train 2.743 | val 2.820 | lr 3.56e-04 | 102,497 tok/s | 54.64 GB | 9.495B tok iter 72460 | train 2.850 | val 2.773 | lr 3.56e-04 | 102,500 tok/s | 54.64 GB | 9.498B tok iter 72480 | train 2.832 | val 2.848 | lr 3.56e-04 | 102,502 tok/s | 54.64 GB | 9.500B tok checkpoint @ iter 72491 -> checkpoints/500m_latest.pt iter 72500 | train 2.783 | val 2.787 | lr 3.56e-04 | 102,487 tok/s | 54.64 GB | 9.503B tok checkpoint @ iter 72500 -> checkpoints/500m_latest.pt iter 72520 | train 2.760 | val 2.767 | lr 3.55e-04 | 102,460 tok/s | 54.64 GB | 9.505B tok iter 72540 | train 2.786 | val 2.807 | lr 3.55e-04 | 102,462 tok/s | 54.64 GB | 9.508B tok iter 72560 | train 2.781 | val 2.815 | lr 3.55e-04 | 102,464 tok/s | 54.64 GB | 9.511B tok iter 72580 | train 2.786 | val 2.746 | lr 3.55e-04 | 102,465 tok/s | 54.64 GB | 9.513B tok iter 72600 | train 2.901 | val 2.810 | lr 3.55e-04 | 102,467 tok/s | 54.64 GB | 9.516B tok iter 72620 | train 2.808 | val 2.861 | lr 3.55e-04 | 102,469 tok/s | 54.64 GB | 9.519B tok iter 72640 | train 2.851 | val 2.797 | lr 3.55e-04 | 102,472 tok/s | 54.64 GB | 9.521B tok iter 72660 | train 2.892 | val 2.810 | lr 3.55e-04 | 102,472 tok/s | 54.64 GB | 9.524B tok iter 72680 | train 2.718 | val 2.772 | lr 3.55e-04 | 102,474 tok/s | 54.64 GB | 9.526B tok iter 72700 | train 2.846 | val 2.796 | lr 3.54e-04 | 102,476 tok/s | 54.64 GB | 9.529B tok iter 72720 | train 3.024 | val 2.802 | lr 3.54e-04 | 102,478 tok/s | 54.64 GB | 9.532B tok iter 72740 | train 2.817 | val 2.817 | lr 3.54e-04 | 102,479 tok/s | 54.64 GB | 9.534B tok iter 72760 | train 2.737 | val 2.743 | lr 3.54e-04 | 102,481 tok/s | 54.64 GB | 9.537B tok iter 72780 | train 2.870 | val 2.861 | lr 3.54e-04 | 102,483 tok/s | 54.64 GB | 9.540B tok iter 72800 | train 2.907 | val 2.799 | lr 3.54e-04 | 102,485 tok/s | 54.64 GB | 9.542B tok iter 72820 | train 2.874 | val 2.799 | lr 3.54e-04 | 102,486 tok/s | 54.64 GB | 9.545B tok iter 72840 | train 2.773 | val 2.803 | lr 3.54e-04 | 102,488 tok/s | 54.64 GB | 9.547B tok iter 72860 | train 2.861 | val 2.830 | lr 3.54e-04 | 102,490 tok/s | 54.64 GB | 9.550B tok iter 72880 | train 2.795 | val 2.833 | lr 3.53e-04 | 102,492 tok/s | 54.64 GB | 9.553B tok iter 72900 | train 2.655 | val 2.803 | lr 3.53e-04 | 102,493 tok/s | 54.64 GB | 9.555B tok iter 72920 | train 2.811 | val 2.779 | lr 3.53e-04 | 102,495 tok/s | 54.64 GB | 9.558B tok iter 72940 | train 2.750 | val 2.830 | lr 3.53e-04 | 102,497 tok/s | 54.64 GB | 9.561B tok iter 72960 | train 2.845 | val 2.784 | lr 3.53e-04 | 102,499 tok/s | 54.64 GB | 9.563B tok iter 72980 | train 2.836 | val 2.789 | lr 3.53e-04 | 102,500 tok/s | 54.64 GB | 9.566B tok checkpoint @ iter 72990 -> checkpoints/500m_latest.pt iter 73000 | train 2.828 | val 2.810 | lr 3.53e-04 | 102,487 tok/s | 54.64 GB | 9.568B tok checkpoint @ iter 73000 -> checkpoints/500m_latest.pt iter 73020 | train 2.917 | val 2.829 | lr 3.53e-04 | 102,461 tok/s | 54.64 GB | 9.571B tok iter 73040 | train 2.840 | val 2.802 | lr 3.53e-04 | 102,463 tok/s | 54.64 GB | 9.574B tok iter 73060 | train 2.828 | val 2.785 | lr 3.52e-04 | 102,463 tok/s | 54.64 GB | 9.576B tok iter 73080 | train 2.757 | val 2.850 | lr 3.52e-04 | 102,466 tok/s | 54.64 GB | 9.579B tok iter 73100 | train 2.751 | val 2.787 | lr 3.52e-04 | 102,468 tok/s | 54.64 GB | 9.581B tok iter 73120 | train 2.711 | val 2.770 | lr 3.52e-04 | 102,470 tok/s | 54.64 GB | 9.584B tok iter 73140 | train 2.714 | val 2.851 | lr 3.52e-04 | 102,470 tok/s | 54.64 GB | 9.587B tok iter 73160 | train 2.828 | val 2.771 | lr 3.52e-04 | 102,473 tok/s | 54.64 GB | 9.589B tok iter 73180 | train 2.911 | val 2.821 | lr 3.52e-04 | 102,475 tok/s | 54.64 GB | 9.592B tok iter 73200 | train 2.810 | val 2.771 | lr 3.52e-04 | 102,477 tok/s | 54.64 GB | 9.595B tok iter 73220 | train 2.883 | val 2.811 | lr 3.52e-04 | 102,477 tok/s | 54.64 GB | 9.597B tok iter 73240 | train 2.924 | val 2.775 | lr 3.51e-04 | 102,480 tok/s | 54.64 GB | 9.600B tok iter 73260 | train 2.870 | val 2.771 | lr 3.51e-04 | 102,482 tok/s | 54.64 GB | 9.602B tok iter 73280 | train 2.624 | val 2.835 | lr 3.51e-04 | 102,484 tok/s | 54.64 GB | 9.605B tok iter 73300 | train 2.855 | val 2.815 | lr 3.51e-04 | 102,484 tok/s | 54.64 GB | 9.608B tok iter 73320 | train 2.808 | val 2.819 | lr 3.51e-04 | 102,486 tok/s | 54.64 GB | 9.610B tok iter 73340 | train 2.787 | val 2.823 | lr 3.51e-04 | 102,489 tok/s | 54.64 GB | 9.613B tok iter 73360 | train 2.822 | val 2.802 | lr 3.51e-04 | 102,491 tok/s | 54.64 GB | 9.616B tok iter 73380 | train 2.802 | val 2.838 | lr 3.51e-04 | 102,491 tok/s | 54.64 GB | 9.618B tok iter 73400 | train 2.759 | val 2.798 | lr 3.51e-04 | 102,493 tok/s | 54.64 GB | 9.621B tok iter 73420 | train 2.777 | val 2.798 | lr 3.50e-04 | 102,496 tok/s | 54.64 GB | 9.623B tok iter 73440 | train 2.764 | val 2.798 | lr 3.50e-04 | 102,498 tok/s | 54.64 GB | 9.626B tok iter 73460 | train 2.906 | val 2.807 | lr 3.50e-04 | 102,498 tok/s | 54.64 GB | 9.629B tok iter 73480 | train 2.877 | val 2.812 | lr 3.50e-04 | 102,500 tok/s | 54.64 GB | 9.631B tok checkpoint @ iter 73490 -> checkpoints/500m_latest.pt iter 73500 | train 2.850 | val 2.793 | lr 3.50e-04 | 102,488 tok/s | 54.64 GB | 9.634B tok checkpoint @ iter 73500 -> checkpoints/500m_latest.pt iter 73520 | train 2.782 | val 2.786 | lr 3.50e-04 | 102,462 tok/s | 54.64 GB | 9.637B tok iter 73540 | train 2.740 | val 2.800 | lr 3.50e-04 | 102,462 tok/s | 54.64 GB | 9.639B tok iter 73560 | train 2.782 | val 2.768 | lr 3.50e-04 | 102,464 tok/s | 54.64 GB | 9.642B tok iter 73580 | train 2.822 | val 2.838 | lr 3.50e-04 | 102,467 tok/s | 54.64 GB | 9.644B tok iter 73600 | train 2.682 | val 2.799 | lr 3.49e-04 | 102,469 tok/s | 54.64 GB | 9.647B tok iter 73620 | train 2.797 | val 2.794 | lr 3.49e-04 | 102,469 tok/s | 54.64 GB | 9.650B tok iter 73640 | train 2.720 | val 2.829 | lr 3.49e-04 | 102,471 tok/s | 54.64 GB | 9.652B tok iter 73660 | train 2.804 | val 2.799 | lr 3.49e-04 | 102,473 tok/s | 54.64 GB | 9.655B tok iter 73680 | train 2.889 | val 2.814 | lr 3.49e-04 | 102,476 tok/s | 54.64 GB | 9.658B tok iter 73700 | train 2.874 | val 2.839 | lr 3.49e-04 | 102,476 tok/s | 54.64 GB | 9.660B tok iter 73720 | train 2.784 | val 2.805 | lr 3.49e-04 | 102,478 tok/s | 54.64 GB | 9.663B tok iter 73740 | train 2.765 | val 2.783 | lr 3.49e-04 | 102,480 tok/s | 54.64 GB | 9.665B tok iter 73760 | train 2.754 | val 2.762 | lr 3.48e-04 | 102,483 tok/s | 54.64 GB | 9.668B tok iter 73780 | train 2.858 | val 2.794 | lr 3.48e-04 | 102,483 tok/s | 54.64 GB | 9.671B tok iter 73800 | train 2.722 | val 2.801 | lr 3.48e-04 | 102,485 tok/s | 54.64 GB | 9.673B tok iter 73820 | train 2.803 | val 2.807 | lr 3.48e-04 | 102,487 tok/s | 54.64 GB | 9.676B tok iter 73840 | train 2.904 | val 2.813 | lr 3.48e-04 | 102,489 tok/s | 54.64 GB | 9.678B tok iter 73860 | train 3.005 | val 2.829 | lr 3.48e-04 | 102,490 tok/s | 54.64 GB | 9.681B tok iter 73880 | train 2.797 | val 2.826 | lr 3.48e-04 | 102,492 tok/s | 54.64 GB | 9.684B tok iter 73900 | train 2.825 | val 2.770 | lr 3.48e-04 | 102,494 tok/s | 54.64 GB | 9.686B tok iter 73920 | train 2.782 | val 2.758 | lr 3.48e-04 | 102,496 tok/s | 54.64 GB | 9.689B tok iter 73940 | train 2.876 | val 2.838 | lr 3.47e-04 | 102,497 tok/s | 54.64 GB | 9.692B tok iter 73960 | train 2.755 | val 2.806 | lr 3.47e-04 | 102,499 tok/s | 54.64 GB | 9.694B tok iter 73980 | train 2.742 | val 2.789 | lr 3.47e-04 | 102,501 tok/s | 54.64 GB | 9.697B tok checkpoint @ iter 73990 -> checkpoints/500m_latest.pt iter 74000 | train 2.740 | val 2.827 | lr 3.47e-04 | 102,489 tok/s | 54.64 GB | 9.699B tok checkpoint @ iter 74000 -> checkpoints/500m_latest.pt iter 74020 | train 2.864 | val 2.797 | lr 3.47e-04 | 102,461 tok/s | 54.64 GB | 9.702B tok iter 74040 | train 2.450 | val 2.789 | lr 3.47e-04 | 102,463 tok/s | 54.64 GB | 9.705B tok iter 74060 | train 2.845 | val 2.774 | lr 3.47e-04 | 102,465 tok/s | 54.64 GB | 9.707B tok iter 74080 | train 2.761 | val 2.767 | lr 3.47e-04 | 102,467 tok/s | 54.64 GB | 9.710B tok iter 74100 | train 2.849 | val 2.758 | lr 3.47e-04 | 102,468 tok/s | 54.64 GB | 9.713B tok iter 74120 | train 2.765 | val 2.781 | lr 3.46e-04 | 102,470 tok/s | 54.64 GB | 9.715B tok iter 74140 | train 2.935 | val 2.811 | lr 3.46e-04 | 102,472 tok/s | 54.64 GB | 9.718B tok iter 74160 | train 2.687 | val 2.856 | lr 3.46e-04 | 102,474 tok/s | 54.64 GB | 9.720B tok iter 74180 | train 2.881 | val 2.751 | lr 3.46e-04 | 102,475 tok/s | 54.64 GB | 9.723B tok iter 74200 | train 2.797 | val 2.813 | lr 3.46e-04 | 102,477 tok/s | 54.64 GB | 9.726B tok iter 74220 | train 2.843 | val 2.871 | lr 3.46e-04 | 102,479 tok/s | 54.64 GB | 9.728B tok iter 74240 | train 2.765 | val 2.773 | lr 3.46e-04 | 102,481 tok/s | 54.64 GB | 9.731B tok iter 74260 | train 2.876 | val 2.780 | lr 3.46e-04 | 102,481 tok/s | 54.64 GB | 9.734B tok iter 74280 | train 2.738 | val 2.796 | lr 3.46e-04 | 102,484 tok/s | 54.64 GB | 9.736B tok iter 74300 | train 2.792 | val 2.836 | lr 3.45e-04 | 102,486 tok/s | 54.64 GB | 9.739B tok iter 74320 | train 2.995 | val 2.810 | lr 3.45e-04 | 102,488 tok/s | 54.64 GB | 9.741B tok iter 74340 | train 2.683 | val 2.805 | lr 3.45e-04 | 102,488 tok/s | 54.64 GB | 9.744B tok iter 74360 | train 2.834 | val 2.802 | lr 3.45e-04 | 102,491 tok/s | 54.64 GB | 9.747B tok iter 74380 | train 2.773 | val 2.805 | lr 3.45e-04 | 102,493 tok/s | 54.64 GB | 9.749B tok iter 74400 | train 2.911 | val 2.802 | lr 3.45e-04 | 102,495 tok/s | 54.64 GB | 9.752B tok iter 74420 | train 2.765 | val 2.815 | lr 3.45e-04 | 102,495 tok/s | 54.64 GB | 9.755B tok iter 74440 | train 2.786 | val 2.786 | lr 3.45e-04 | 102,497 tok/s | 54.64 GB | 9.757B tok iter 74460 | train 2.838 | val 2.834 | lr 3.45e-04 | 102,499 tok/s | 54.64 GB | 9.760B tok iter 74480 | train 2.937 | val 2.816 | lr 3.44e-04 | 102,501 tok/s | 54.64 GB | 9.762B tok checkpoint @ iter 74490 -> checkpoints/500m_latest.pt iter 74500 | train 2.785 | val 2.767 | lr 3.44e-04 | 102,488 tok/s | 54.64 GB | 9.765B tok checkpoint @ iter 74500 -> checkpoints/500m_latest.pt iter 74520 | train 2.841 | val 2.787 | lr 3.44e-04 | 102,462 tok/s | 54.64 GB | 9.768B tok iter 74540 | train 2.787 | val 2.767 | lr 3.44e-04 | 102,465 tok/s | 54.64 GB | 9.770B tok iter 74560 | train 2.860 | val 2.816 | lr 3.44e-04 | 102,467 tok/s | 54.64 GB | 9.773B tok iter 74580 | train 2.873 | val 2.807 | lr 3.44e-04 | 102,467 tok/s | 54.64 GB | 9.775B tok iter 74600 | train 2.825 | val 2.778 | lr 3.44e-04 | 102,469 tok/s | 54.64 GB | 9.778B tok iter 74620 | train 2.702 | val 2.758 | lr 3.44e-04 | 102,472 tok/s | 54.64 GB | 9.781B tok iter 74640 | train 2.769 | val 2.759 | lr 3.44e-04 | 102,474 tok/s | 54.64 GB | 9.783B tok iter 74660 | train 2.844 | val 2.800 | lr 3.43e-04 | 102,474 tok/s | 54.64 GB | 9.786B tok iter 74680 | train 2.637 | val 2.829 | lr 3.43e-04 | 102,476 tok/s | 54.64 GB | 9.789B tok iter 74700 | train 2.830 | val 2.809 | lr 3.43e-04 | 102,478 tok/s | 54.64 GB | 9.791B tok iter 74720 | train 2.893 | val 2.738 | lr 3.43e-04 | 102,481 tok/s | 54.64 GB | 9.794B tok iter 74740 | train 2.853 | val 2.844 | lr 3.43e-04 | 102,481 tok/s | 54.64 GB | 9.796B tok iter 74760 | train 2.872 | val 2.835 | lr 3.43e-04 | 102,483 tok/s | 54.64 GB | 9.799B tok iter 74780 | train 2.790 | val 2.754 | lr 3.43e-04 | 102,485 tok/s | 54.64 GB | 9.802B tok iter 74800 | train 2.852 | val 2.779 | lr 3.43e-04 | 102,487 tok/s | 54.64 GB | 9.804B tok iter 74820 | train 2.780 | val 2.805 | lr 3.43e-04 | 102,488 tok/s | 54.64 GB | 9.807B tok iter 74840 | train 2.866 | val 2.796 | lr 3.42e-04 | 102,490 tok/s | 54.64 GB | 9.810B tok iter 74860 | train 2.850 | val 2.817 | lr 3.42e-04 | 102,492 tok/s | 54.64 GB | 9.812B tok iter 74880 | train 2.762 | val 2.844 | lr 3.42e-04 | 102,494 tok/s | 54.64 GB | 9.815B tok iter 74900 | train 2.786 | val 2.808 | lr 3.42e-04 | 102,494 tok/s | 54.64 GB | 9.817B tok iter 74920 | train 2.863 | val 2.844 | lr 3.42e-04 | 102,497 tok/s | 54.64 GB | 9.820B tok iter 74940 | train 2.847 | val 2.803 | lr 3.42e-04 | 102,499 tok/s | 54.64 GB | 9.823B tok iter 74960 | train 2.717 | val 2.840 | lr 3.42e-04 | 102,501 tok/s | 54.64 GB | 9.825B tok iter 74980 | train 2.763 | val 2.733 | lr 3.42e-04 | 102,501 tok/s | 54.64 GB | 9.828B tok checkpoint @ iter 74991 -> checkpoints/500m_latest.pt iter 75000 | train 2.803 | val 2.868 | lr 3.42e-04 | 102,489 tok/s | 54.64 GB | 9.831B tok checkpoint @ iter 75000 -> checkpoints/500m_latest.pt iter 75020 | train 2.768 | val 2.774 | lr 3.41e-04 | 102,464 tok/s | 54.64 GB | 9.833B tok iter 75040 | train 2.828 | val 2.806 | lr 3.41e-04 | 102,466 tok/s | 54.64 GB | 9.836B tok iter 75060 | train 2.747 | val 2.820 | lr 3.41e-04 | 102,466 tok/s | 54.64 GB | 9.838B tok iter 75080 | train 2.806 | val 2.779 | lr 3.41e-04 | 102,468 tok/s | 54.64 GB | 9.841B tok iter 75100 | train 2.805 | val 2.804 | lr 3.41e-04 | 102,470 tok/s | 54.64 GB | 9.844B tok iter 75120 | train 2.675 | val 2.790 | lr 3.41e-04 | 102,472 tok/s | 54.64 GB | 9.846B tok iter 75140 | train 2.849 | val 2.818 | lr 3.41e-04 | 102,473 tok/s | 54.64 GB | 9.849B tok iter 75160 | train 2.880 | val 2.809 | lr 3.41e-04 | 102,475 tok/s | 54.64 GB | 9.852B tok iter 75180 | train 2.784 | val 2.798 | lr 3.41e-04 | 102,477 tok/s | 54.64 GB | 9.854B tok iter 75200 | train 2.788 | val 2.814 | lr 3.40e-04 | 102,479 tok/s | 54.64 GB | 9.857B tok iter 75220 | train 2.947 | val 2.823 | lr 3.40e-04 | 102,480 tok/s | 54.64 GB | 9.859B tok iter 75240 | train 2.796 | val 2.803 | lr 3.40e-04 | 102,482 tok/s | 54.64 GB | 9.862B tok iter 75260 | train 2.863 | val 2.768 | lr 3.40e-04 | 102,484 tok/s | 54.64 GB | 9.865B tok iter 75280 | train 2.816 | val 2.783 | lr 3.40e-04 | 102,486 tok/s | 54.64 GB | 9.867B tok iter 75300 | train 2.819 | val 2.832 | lr 3.40e-04 | 102,486 tok/s | 54.64 GB | 9.870B tok iter 75320 | train 2.807 | val 2.790 | lr 3.40e-04 | 102,488 tok/s | 54.64 GB | 9.872B tok iter 75340 | train 2.728 | val 2.783 | lr 3.40e-04 | 102,490 tok/s | 54.64 GB | 9.875B tok iter 75360 | train 2.686 | val 2.789 | lr 3.40e-04 | 102,493 tok/s | 54.64 GB | 9.878B tok iter 75380 | train 2.817 | val 2.738 | lr 3.39e-04 | 102,493 tok/s | 54.64 GB | 9.880B tok iter 75400 | train 2.829 | val 2.758 | lr 3.39e-04 | 102,495 tok/s | 54.64 GB | 9.883B tok iter 75420 | train 2.725 | val 2.760 | lr 3.39e-04 | 102,497 tok/s | 54.64 GB | 9.886B tok iter 75440 | train 2.839 | val 2.793 | lr 3.39e-04 | 102,499 tok/s | 54.64 GB | 9.888B tok iter 75460 | train 2.819 | val 2.819 | lr 3.39e-04 | 102,500 tok/s | 54.64 GB | 9.891B tok iter 75480 | train 2.816 | val 2.800 | lr 3.39e-04 | 102,502 tok/s | 54.64 GB | 9.893B tok checkpoint @ iter 75490 -> checkpoints/500m_latest.pt iter 75500 | train 2.620 | val 2.808 | lr 3.39e-04 | 102,489 tok/s | 54.64 GB | 9.896B tok checkpoint @ iter 75500 -> checkpoints/500m_latest.pt iter 75520 | train 2.863 | val 2.755 | lr 3.39e-04 | 102,463 tok/s | 54.64 GB | 9.899B tok iter 75540 | train 2.860 | val 2.784 | lr 3.39e-04 | 102,463 tok/s | 54.64 GB | 9.901B tok iter 75560 | train 2.805 | val 2.824 | lr 3.38e-04 | 102,466 tok/s | 54.64 GB | 9.904B tok iter 75580 | train 2.860 | val 2.821 | lr 3.38e-04 | 102,468 tok/s | 54.64 GB | 9.907B tok iter 75600 | train 2.564 | val 2.802 | lr 3.38e-04 | 102,470 tok/s | 54.64 GB | 9.909B tok iter 75620 | train 2.763 | val 2.798 | lr 3.38e-04 | 102,470 tok/s | 54.64 GB | 9.912B tok iter 75640 | train 2.735 | val 2.812 | lr 3.38e-04 | 102,472 tok/s | 54.64 GB | 9.914B tok iter 75660 | train 2.841 | val 2.792 | lr 3.38e-04 | 102,474 tok/s | 54.64 GB | 9.917B tok iter 75680 | train 2.802 | val 2.780 | lr 3.38e-04 | 102,477 tok/s | 54.64 GB | 9.920B tok iter 75700 | train 2.628 | val 2.808 | lr 3.38e-04 | 102,477 tok/s | 54.64 GB | 9.922B tok iter 75720 | train 2.934 | val 2.743 | lr 3.38e-04 | 102,479 tok/s | 54.64 GB | 9.925B tok iter 75740 | train 2.766 | val 2.759 | lr 3.37e-04 | 102,481 tok/s | 54.64 GB | 9.928B tok iter 75760 | train 2.890 | val 2.763 | lr 3.37e-04 | 102,483 tok/s | 54.64 GB | 9.930B tok iter 75780 | train 2.816 | val 2.756 | lr 3.37e-04 | 102,484 tok/s | 54.64 GB | 9.933B tok iter 75800 | train 2.870 | val 2.718 | lr 3.37e-04 | 102,486 tok/s | 54.64 GB | 9.935B tok iter 75820 | train 2.836 | val 2.805 | lr 3.37e-04 | 102,488 tok/s | 54.64 GB | 9.938B tok iter 75840 | train 2.734 | val 2.817 | lr 3.37e-04 | 102,490 tok/s | 54.64 GB | 9.941B tok iter 75860 | train 2.910 | val 2.797 | lr 3.37e-04 | 102,490 tok/s | 54.64 GB | 9.943B tok iter 75880 | train 2.829 | val 2.803 | lr 3.37e-04 | 102,492 tok/s | 54.64 GB | 9.946B tok iter 75900 | train 2.630 | val 2.706 | lr 3.36e-04 | 102,495 tok/s | 54.64 GB | 9.948B tok iter 75920 | train 2.803 | val 2.754 | lr 3.36e-04 | 102,497 tok/s | 54.64 GB | 9.951B tok iter 75940 | train 2.744 | val 2.802 | lr 3.36e-04 | 102,497 tok/s | 54.64 GB | 9.954B tok iter 75960 | train 2.865 | val 2.754 | lr 3.36e-04 | 102,499 tok/s | 54.64 GB | 9.956B tok iter 75980 | train 2.851 | val 2.797 | lr 3.36e-04 | 102,501 tok/s | 54.64 GB | 9.959B tok checkpoint @ iter 75990 -> checkpoints/500m_latest.pt iter 76000 | train 2.715 | val 2.798 | lr 3.36e-04 | 102,489 tok/s | 54.64 GB | 9.962B tok checkpoint @ iter 76000 -> checkpoints/500m_latest.pt iter 76020 | train 2.829 | val 2.779 | lr 3.36e-04 | 102,462 tok/s | 54.64 GB | 9.964B tok iter 76040 | train 2.845 | val 2.795 | lr 3.36e-04 | 102,464 tok/s | 54.64 GB | 9.967B tok iter 76060 | train 2.739 | val 2.798 | lr 3.36e-04 | 102,467 tok/s | 54.64 GB | 9.969B tok iter 76080 | train 2.594 | val 2.737 | lr 3.35e-04 | 102,469 tok/s | 54.64 GB | 9.972B tok iter 76100 | train 2.778 | val 2.846 | lr 3.35e-04 | 102,469 tok/s | 54.64 GB | 9.975B tok iter 76120 | train 2.774 | val 2.743 | lr 3.35e-04 | 102,471 tok/s | 54.64 GB | 9.977B tok iter 76140 | train 2.647 | val 2.821 | lr 3.35e-04 | 102,473 tok/s | 54.64 GB | 9.980B tok iter 76160 | train 2.712 | val 2.773 | lr 3.35e-04 | 102,475 tok/s | 54.64 GB | 9.983B tok iter 76180 | train 2.714 | val 2.817 | lr 3.35e-04 | 102,476 tok/s | 54.64 GB | 9.985B tok iter 76200 | train 2.751 | val 2.819 | lr 3.35e-04 | 102,478 tok/s | 54.64 GB | 9.988B tok iter 76220 | train 2.910 | val 2.776 | lr 3.35e-04 | 102,480 tok/s | 54.64 GB | 9.990B tok iter 76240 | train 2.846 | val 2.754 | lr 3.35e-04 | 102,482 tok/s | 54.64 GB | 9.993B tok iter 76260 | train 2.822 | val 2.776 | lr 3.34e-04 | 102,482 tok/s | 54.64 GB | 9.996B tok iter 76280 | train 2.726 | val 2.746 | lr 3.34e-04 | 102,484 tok/s | 54.64 GB | 9.998B tok iter 76300 | train 2.810 | val 2.788 | lr 3.34e-04 | 102,486 tok/s | 54.64 GB | 10.001B tok iter 76320 | train 2.682 | val 2.774 | lr 3.34e-04 | 102,489 tok/s | 54.64 GB | 10.004B tok iter 76340 | train 2.706 | val 2.784 | lr 3.34e-04 | 102,489 tok/s | 54.64 GB | 10.006B tok iter 76360 | train 2.660 | val 2.779 | lr 3.34e-04 | 102,491 tok/s | 54.64 GB | 10.009B tok iter 76380 | train 2.829 | val 2.816 | lr 3.34e-04 | 102,493 tok/s | 54.64 GB | 10.011B tok iter 76400 | train 2.815 | val 2.830 | lr 3.34e-04 | 102,495 tok/s | 54.64 GB | 10.014B tok iter 76420 | train 2.936 | val 2.758 | lr 3.34e-04 | 102,496 tok/s | 54.64 GB | 10.017B tok iter 76440 | train 2.841 | val 2.811 | lr 3.33e-04 | 102,498 tok/s | 54.64 GB | 10.019B tok iter 76460 | train 2.800 | val 2.795 | lr 3.33e-04 | 102,500 tok/s | 54.64 GB | 10.022B tok iter 76480 | train 2.666 | val 2.757 | lr 3.33e-04 | 102,502 tok/s | 54.64 GB | 10.025B tok checkpoint @ iter 76490 -> checkpoints/500m_latest.pt iter 76500 | train 2.687 | val 2.797 | lr 3.33e-04 | 102,488 tok/s | 54.64 GB | 10.027B tok checkpoint @ iter 76500 -> checkpoints/500m_latest.pt iter 76520 | train 2.765 | val 2.784 | lr 3.33e-04 | 102,463 tok/s | 54.64 GB | 10.030B tok iter 76540 | train 2.814 | val 2.773 | lr 3.33e-04 | 102,465 tok/s | 54.64 GB | 10.032B tok iter 76560 | train 2.823 | val 2.793 | lr 3.33e-04 | 102,467 tok/s | 54.64 GB | 10.035B tok iter 76580 | train 2.818 | val 2.753 | lr 3.33e-04 | 102,467 tok/s | 54.64 GB | 10.038B tok iter 76600 | train 2.951 | val 2.811 | lr 3.33e-04 | 102,469 tok/s | 54.64 GB | 10.040B tok iter 76620 | train 2.770 | val 2.815 | lr 3.32e-04 | 102,471 tok/s | 54.64 GB | 10.043B tok iter 76640 | train 2.715 | val 2.849 | lr 3.32e-04 | 102,474 tok/s | 54.64 GB | 10.045B tok iter 76660 | train 2.731 | val 2.795 | lr 3.32e-04 | 102,474 tok/s | 54.64 GB | 10.048B tok iter 76680 | train 2.798 | val 2.789 | lr 3.32e-04 | 102,476 tok/s | 54.64 GB | 10.051B tok iter 76700 | train 2.909 | val 2.763 | lr 3.32e-04 | 102,478 tok/s | 54.64 GB | 10.053B tok iter 76720 | train 2.658 | val 2.791 | lr 3.32e-04 | 102,480 tok/s | 54.64 GB | 10.056B tok iter 76740 | train 2.821 | val 2.758 | lr 3.32e-04 | 102,481 tok/s | 54.64 GB | 10.059B tok iter 76760 | train 2.702 | val 2.769 | lr 3.32e-04 | 102,483 tok/s | 54.64 GB | 10.061B tok iter 76780 | train 2.724 | val 2.801 | lr 3.32e-04 | 102,485 tok/s | 54.64 GB | 10.064B tok iter 76800 | train 2.750 | val 2.799 | lr 3.31e-04 | 102,487 tok/s | 54.64 GB | 10.066B tok iter 76820 | train 2.817 | val 2.772 | lr 3.31e-04 | 102,487 tok/s | 54.64 GB | 10.069B tok iter 76840 | train 2.789 | val 2.774 | lr 3.31e-04 | 102,489 tok/s | 54.64 GB | 10.072B tok iter 76860 | train 2.693 | val 2.753 | lr 3.31e-04 | 102,492 tok/s | 54.64 GB | 10.074B tok iter 76880 | train 2.767 | val 2.746 | lr 3.31e-04 | 102,494 tok/s | 54.64 GB | 10.077B tok iter 76900 | train 2.735 | val 2.775 | lr 3.31e-04 | 102,494 tok/s | 54.64 GB | 10.080B tok iter 76920 | train 2.796 | val 2.775 | lr 3.31e-04 | 102,496 tok/s | 54.64 GB | 10.082B tok iter 76940 | train 2.705 | val 2.789 | lr 3.31e-04 | 102,498 tok/s | 54.64 GB | 10.085B tok iter 76960 | train 2.682 | val 2.791 | lr 3.31e-04 | 102,500 tok/s | 54.64 GB | 10.087B tok iter 76980 | train 2.856 | val 2.757 | lr 3.30e-04 | 102,500 tok/s | 54.64 GB | 10.090B tok checkpoint @ iter 76990 -> checkpoints/500m_latest.pt iter 77000 | train 2.714 | val 2.748 | lr 3.30e-04 | 102,489 tok/s | 54.64 GB | 10.093B tok checkpoint @ iter 77000 -> checkpoints/500m_latest.pt iter 77020 | train 2.770 | val 2.793 | lr 3.30e-04 | 102,463 tok/s | 54.64 GB | 10.095B tok iter 77040 | train 2.770 | val 2.832 | lr 3.30e-04 | 102,466 tok/s | 54.64 GB | 10.098B tok iter 77060 | train 2.787 | val 2.831 | lr 3.30e-04 | 102,466 tok/s | 54.64 GB | 10.101B tok iter 77080 | train 2.783 | val 2.782 | lr 3.30e-04 | 102,468 tok/s | 54.64 GB | 10.103B tok iter 77100 | train 2.863 | val 2.798 | lr 3.30e-04 | 102,470 tok/s | 54.64 GB | 10.106B tok iter 77120 | train 2.793 | val 2.789 | lr 3.30e-04 | 102,472 tok/s | 54.64 GB | 10.108B tok iter 77140 | train 2.785 | val 2.837 | lr 3.30e-04 | 102,473 tok/s | 54.64 GB | 10.111B tok iter 77160 | train 2.766 | val 2.783 | lr 3.29e-04 | 102,475 tok/s | 54.64 GB | 10.114B tok iter 77180 | train 2.760 | val 2.756 | lr 3.29e-04 | 102,477 tok/s | 54.64 GB | 10.116B tok iter 77200 | train 2.793 | val 2.801 | lr 3.29e-04 | 102,479 tok/s | 54.64 GB | 10.119B tok iter 77220 | train 2.801 | val 2.765 | lr 3.29e-04 | 102,479 tok/s | 54.64 GB | 10.122B tok iter 77240 | train 2.923 | val 2.806 | lr 3.29e-04 | 102,481 tok/s | 54.64 GB | 10.124B tok iter 77260 | train 2.780 | val 2.792 | lr 3.29e-04 | 102,483 tok/s | 54.64 GB | 10.127B tok iter 77280 | train 2.673 | val 2.793 | lr 3.29e-04 | 102,486 tok/s | 54.64 GB | 10.129B tok iter 77300 | train 2.824 | val 2.794 | lr 3.29e-04 | 102,486 tok/s | 54.64 GB | 10.132B tok iter 77320 | train 2.705 | val 2.805 | lr 3.29e-04 | 102,488 tok/s | 54.64 GB | 10.135B tok iter 77340 | train 2.779 | val 2.757 | lr 3.28e-04 | 102,490 tok/s | 54.64 GB | 10.137B tok iter 77360 | train 2.836 | val 2.779 | lr 3.28e-04 | 102,492 tok/s | 54.64 GB | 10.140B tok iter 77380 | train 2.766 | val 2.829 | lr 3.28e-04 | 102,492 tok/s | 54.64 GB | 10.142B tok iter 77400 | train 2.766 | val 2.787 | lr 3.28e-04 | 102,494 tok/s | 54.64 GB | 10.145B tok iter 77420 | train 2.773 | val 2.748 | lr 3.28e-04 | 102,497 tok/s | 54.64 GB | 10.148B tok iter 77440 | train 2.787 | val 2.755 | lr 3.28e-04 | 102,499 tok/s | 54.64 GB | 10.150B tok iter 77460 | train 2.740 | val 2.794 | lr 3.28e-04 | 102,499 tok/s | 54.64 GB | 10.153B tok iter 77480 | train 2.656 | val 2.775 | lr 3.28e-04 | 102,501 tok/s | 54.64 GB | 10.156B tok checkpoint @ iter 77490 -> checkpoints/500m_latest.pt iter 77500 | train 2.822 | val 2.798 | lr 3.28e-04 | 102,489 tok/s | 54.64 GB | 10.158B tok checkpoint @ iter 77500 -> checkpoints/500m_latest.pt iter 77520 | train 2.860 | val 2.797 | lr 3.27e-04 | 102,463 tok/s | 54.64 GB | 10.161B tok iter 77540 | train 2.825 | val 2.747 | lr 3.27e-04 | 102,463 tok/s | 54.64 GB | 10.163B tok iter 77560 | train 2.741 | val 2.830 | lr 3.27e-04 | 102,465 tok/s | 54.64 GB | 10.166B tok iter 77580 | train 2.619 | val 2.750 | lr 3.27e-04 | 102,467 tok/s | 54.64 GB | 10.169B tok iter 77600 | train 2.727 | val 2.811 | lr 3.27e-04 | 102,469 tok/s | 54.64 GB | 10.171B tok iter 77620 | train 2.717 | val 2.814 | lr 3.27e-04 | 102,470 tok/s | 54.64 GB | 10.174B tok iter 77640 | train 2.816 | val 2.778 | lr 3.27e-04 | 102,472 tok/s | 54.64 GB | 10.177B tok iter 77660 | train 2.809 | val 2.788 | lr 3.27e-04 | 102,474 tok/s | 54.64 GB | 10.179B tok iter 77680 | train 2.772 | val 2.826 | lr 3.26e-04 | 102,476 tok/s | 54.64 GB | 10.182B tok iter 77700 | train 2.817 | val 2.744 | lr 3.26e-04 | 102,476 tok/s | 54.64 GB | 10.184B tok iter 77720 | train 2.705 | val 2.735 | lr 3.26e-04 | 102,478 tok/s | 54.64 GB | 10.187B tok iter 77740 | train 2.807 | val 2.756 | lr 3.26e-04 | 102,481 tok/s | 54.64 GB | 10.190B tok iter 77760 | train 2.870 | val 2.790 | lr 3.26e-04 | 102,483 tok/s | 54.64 GB | 10.192B tok iter 77780 | train 2.703 | val 2.783 | lr 3.26e-04 | 102,483 tok/s | 54.64 GB | 10.195B tok iter 77800 | train 2.941 | val 2.765 | lr 3.26e-04 | 102,485 tok/s | 54.64 GB | 10.198B tok iter 77820 | train 2.796 | val 2.765 | lr 3.26e-04 | 102,487 tok/s | 54.64 GB | 10.200B tok iter 77840 | train 2.760 | val 2.802 | lr 3.26e-04 | 102,489 tok/s | 54.64 GB | 10.203B tok iter 77860 | train 2.870 | val 2.867 | lr 3.25e-04 | 102,489 tok/s | 54.64 GB | 10.205B tok iter 77880 | train 2.782 | val 2.768 | lr 3.25e-04 | 102,491 tok/s | 54.64 GB | 10.208B tok iter 77900 | train 2.704 | val 2.765 | lr 3.25e-04 | 102,493 tok/s | 54.64 GB | 10.211B tok iter 77920 | train 2.635 | val 2.798 | lr 3.25e-04 | 102,496 tok/s | 54.64 GB | 10.213B tok iter 77940 | train 2.857 | val 2.748 | lr 3.25e-04 | 102,496 tok/s | 54.64 GB | 10.216B tok iter 77960 | train 2.758 | val 2.754 | lr 3.25e-04 | 102,498 tok/s | 54.64 GB | 10.219B tok iter 77980 | train 2.724 | val 2.752 | lr 3.25e-04 | 102,500 tok/s | 54.64 GB | 10.221B tok checkpoint @ iter 77990 -> checkpoints/500m_latest.pt iter 78000 | train 2.706 | val 2.796 | lr 3.25e-04 | 102,488 tok/s | 54.64 GB | 10.224B tok checkpoint @ iter 78000 -> checkpoints/500m_latest.pt iter 78020 | train 2.834 | val 2.745 | lr 3.25e-04 | 102,461 tok/s | 54.64 GB | 10.226B tok iter 78040 | train 2.664 | val 2.787 | lr 3.24e-04 | 102,463 tok/s | 54.64 GB | 10.229B tok iter 78060 | train 2.715 | val 2.756 | lr 3.24e-04 | 102,465 tok/s | 54.64 GB | 10.232B tok iter 78080 | train 2.859 | val 2.752 | lr 3.24e-04 | 102,467 tok/s | 54.64 GB | 10.234B tok iter 78100 | train 2.789 | val 2.789 | lr 3.24e-04 | 102,467 tok/s | 54.64 GB | 10.237B tok iter 78120 | train 2.743 | val 2.784 | lr 3.24e-04 | 102,470 tok/s | 54.64 GB | 10.239B tok iter 78140 | train 2.865 | val 2.774 | lr 3.24e-04 | 102,472 tok/s | 54.64 GB | 10.242B tok iter 78160 | train 2.792 | val 2.754 | lr 3.24e-04 | 102,474 tok/s | 54.64 GB | 10.245B tok iter 78180 | train 2.697 | val 2.783 | lr 3.24e-04 | 102,474 tok/s | 54.64 GB | 10.247B tok iter 78200 | train 2.838 | val 2.825 | lr 3.24e-04 | 102,476 tok/s | 54.64 GB | 10.250B tok iter 78220 | train 2.820 | val 2.829 | lr 3.23e-04 | 102,478 tok/s | 54.64 GB | 10.253B tok iter 78240 | train 2.908 | val 2.785 | lr 3.23e-04 | 102,480 tok/s | 54.64 GB | 10.255B tok iter 78260 | train 2.761 | val 2.771 | lr 3.23e-04 | 102,480 tok/s | 54.64 GB | 10.258B tok iter 78280 | train 2.752 | val 2.769 | lr 3.23e-04 | 102,483 tok/s | 54.64 GB | 10.260B tok iter 78300 | train 2.778 | val 2.775 | lr 3.23e-04 | 102,485 tok/s | 54.64 GB | 10.263B tok iter 78320 | train 2.869 | val 2.808 | lr 3.23e-04 | 102,487 tok/s | 54.64 GB | 10.266B tok iter 78340 | train 2.888 | val 2.775 | lr 3.23e-04 | 102,487 tok/s | 54.64 GB | 10.268B tok iter 78360 | train 2.707 | val 2.794 | lr 3.23e-04 | 102,489 tok/s | 54.64 GB | 10.271B tok iter 78380 | train 2.705 | val 2.821 | lr 3.23e-04 | 102,491 tok/s | 54.64 GB | 10.274B tok iter 78400 | train 2.834 | val 2.787 | lr 3.22e-04 | 102,493 tok/s | 54.64 GB | 10.276B tok iter 78420 | train 2.730 | val 2.810 | lr 3.22e-04 | 102,493 tok/s | 54.64 GB | 10.279B tok iter 78440 | train 2.950 | val 2.790 | lr 3.22e-04 | 102,495 tok/s | 54.64 GB | 10.281B tok iter 78460 | train 2.728 | val 2.778 | lr 3.22e-04 | 102,498 tok/s | 54.64 GB | 10.284B tok iter 78480 | train 2.790 | val 2.758 | lr 3.22e-04 | 102,500 tok/s | 54.64 GB | 10.287B tok checkpoint @ iter 78490 -> checkpoints/500m_latest.pt iter 78500 | train 2.683 | val 2.790 | lr 3.22e-04 | 102,486 tok/s | 54.64 GB | 10.289B tok checkpoint @ iter 78500 -> checkpoints/500m_latest.pt iter 78520 | train 2.796 | val 2.795 | lr 3.22e-04 | 102,461 tok/s | 54.64 GB | 10.292B tok iter 78540 | train 2.844 | val 2.773 | lr 3.22e-04 | 102,463 tok/s | 54.64 GB | 10.295B tok iter 78560 | train 2.825 | val 2.818 | lr 3.22e-04 | 102,465 tok/s | 54.64 GB | 10.297B tok iter 78580 | train 2.838 | val 2.723 | lr 3.21e-04 | 102,465 tok/s | 54.64 GB | 10.300B tok iter 78600 | train 2.875 | val 2.773 | lr 3.21e-04 | 102,467 tok/s | 54.64 GB | 10.302B tok iter 78620 | train 2.872 | val 2.756 | lr 3.21e-04 | 102,469 tok/s | 54.64 GB | 10.305B tok iter 78640 | train 2.841 | val 2.823 | lr 3.21e-04 | 102,471 tok/s | 54.64 GB | 10.308B tok iter 78660 | train 2.755 | val 2.791 | lr 3.21e-04 | 102,471 tok/s | 54.64 GB | 10.310B tok iter 78680 | train 2.641 | val 2.743 | lr 3.21e-04 | 102,474 tok/s | 54.64 GB | 10.313B tok iter 78700 | train 2.690 | val 2.790 | lr 3.21e-04 | 102,476 tok/s | 54.64 GB | 10.315B tok iter 78720 | train 2.916 | val 2.770 | lr 3.21e-04 | 102,478 tok/s | 54.64 GB | 10.318B tok iter 78740 | train 2.821 | val 2.798 | lr 3.21e-04 | 102,478 tok/s | 54.64 GB | 10.321B tok iter 78760 | train 2.784 | val 2.746 | lr 3.20e-04 | 102,480 tok/s | 54.64 GB | 10.323B tok iter 78780 | train 2.779 | val 2.798 | lr 3.20e-04 | 102,482 tok/s | 54.64 GB | 10.326B tok iter 78800 | train 2.893 | val 2.771 | lr 3.20e-04 | 102,484 tok/s | 54.64 GB | 10.329B tok iter 78820 | train 2.880 | val 2.786 | lr 3.20e-04 | 102,484 tok/s | 54.64 GB | 10.331B tok iter 78840 | train 2.846 | val 2.803 | lr 3.20e-04 | 102,487 tok/s | 54.64 GB | 10.334B tok iter 78860 | train 2.858 | val 2.779 | lr 3.20e-04 | 102,489 tok/s | 54.64 GB | 10.336B tok iter 78880 | train 2.991 | val 2.718 | lr 3.20e-04 | 102,491 tok/s | 54.64 GB | 10.339B tok iter 78900 | train 2.736 | val 2.747 | lr 3.20e-04 | 102,491 tok/s | 54.64 GB | 10.342B tok iter 78920 | train 2.780 | val 2.828 | lr 3.20e-04 | 102,493 tok/s | 54.64 GB | 10.344B tok iter 78940 | train 2.787 | val 2.815 | lr 3.19e-04 | 102,495 tok/s | 54.64 GB | 10.347B tok iter 78960 | train 2.798 | val 2.809 | lr 3.19e-04 | 102,497 tok/s | 54.64 GB | 10.350B tok iter 78980 | train 2.751 | val 2.797 | lr 3.19e-04 | 102,497 tok/s | 54.64 GB | 10.352B tok checkpoint @ iter 78990 -> checkpoints/500m_latest.pt iter 79000 | train 2.780 | val 2.789 | lr 3.19e-04 | 102,485 tok/s | 54.64 GB | 10.355B tok checkpoint @ iter 79000 -> checkpoints/500m_latest.pt iter 79020 | train 2.846 | val 2.794 | lr 3.19e-04 | 102,459 tok/s | 54.64 GB | 10.357B tok iter 79040 | train 2.793 | val 2.787 | lr 3.19e-04 | 102,461 tok/s | 54.64 GB | 10.360B tok iter 79060 | train 2.816 | val 2.806 | lr 3.19e-04 | 102,461 tok/s | 54.64 GB | 10.363B tok iter 79080 | train 2.831 | val 2.813 | lr 3.19e-04 | 102,463 tok/s | 54.64 GB | 10.365B tok iter 79100 | train 2.790 | val 2.797 | lr 3.19e-04 | 102,465 tok/s | 54.64 GB | 10.368B tok iter 79120 | train 2.774 | val 2.771 | lr 3.18e-04 | 102,467 tok/s | 54.64 GB | 10.371B tok iter 79140 | train 2.840 | val 2.782 | lr 3.18e-04 | 102,468 tok/s | 54.64 GB | 10.373B tok iter 79160 | train 2.877 | val 2.833 | lr 3.18e-04 | 102,470 tok/s | 54.64 GB | 10.376B tok iter 79180 | train 2.630 | val 2.845 | lr 3.18e-04 | 102,472 tok/s | 54.64 GB | 10.378B tok iter 79200 | train 2.786 | val 2.793 | lr 3.18e-04 | 102,474 tok/s | 54.64 GB | 10.381B tok iter 79220 | train 2.783 | val 2.744 | lr 3.18e-04 | 102,474 tok/s | 54.64 GB | 10.384B tok iter 79240 | train 2.901 | val 2.776 | lr 3.18e-04 | 102,476 tok/s | 54.64 GB | 10.386B tok iter 79260 | train 2.669 | val 2.760 | lr 3.18e-04 | 102,478 tok/s | 54.64 GB | 10.389B tok iter 79280 | train 2.797 | val 2.782 | lr 3.18e-04 | 102,480 tok/s | 54.64 GB | 10.392B tok iter 79300 | train 2.816 | val 2.749 | lr 3.17e-04 | 102,481 tok/s | 54.64 GB | 10.394B tok iter 79320 | train 2.878 | val 2.774 | lr 3.17e-04 | 102,483 tok/s | 54.64 GB | 10.397B tok iter 79340 | train 2.611 | val 2.808 | lr 3.17e-04 | 102,485 tok/s | 54.64 GB | 10.399B tok iter 79360 | train 2.650 | val 2.774 | lr 3.17e-04 | 102,487 tok/s | 54.64 GB | 10.402B tok iter 79380 | train 2.647 | val 2.760 | lr 3.17e-04 | 102,487 tok/s | 54.64 GB | 10.405B tok iter 79400 | train 2.738 | val 2.767 | lr 3.17e-04 | 102,489 tok/s | 54.64 GB | 10.407B tok iter 79420 | train 2.821 | val 2.787 | lr 3.17e-04 | 102,491 tok/s | 54.64 GB | 10.410B tok iter 79440 | train 2.810 | val 2.764 | lr 3.17e-04 | 102,493 tok/s | 54.64 GB | 10.412B tok iter 79460 | train 2.826 | val 2.823 | lr 3.17e-04 | 102,493 tok/s | 54.64 GB | 10.415B tok iter 79480 | train 2.778 | val 2.760 | lr 3.16e-04 | 102,495 tok/s | 54.64 GB | 10.418B tok checkpoint @ iter 79489 -> checkpoints/500m_latest.pt iter 79500 | train 2.863 | val 2.804 | lr 3.16e-04 | 102,484 tok/s | 54.64 GB | 10.420B tok checkpoint @ iter 79500 -> checkpoints/500m_latest.pt iter 79520 | train 2.721 | val 2.749 | lr 3.16e-04 | 102,460 tok/s | 54.64 GB | 10.423B tok iter 79540 | train 2.715 | val 2.773 | lr 3.16e-04 | 102,460 tok/s | 54.64 GB | 10.426B tok iter 79560 | train 2.757 | val 2.805 | lr 3.16e-04 | 102,462 tok/s | 54.64 GB | 10.428B tok iter 79580 | train 2.756 | val 2.767 | lr 3.16e-04 | 102,464 tok/s | 54.64 GB | 10.431B tok iter 79600 | train 2.829 | val 2.792 | lr 3.16e-04 | 102,466 tok/s | 54.64 GB | 10.433B tok iter 79620 | train 2.883 | val 2.794 | lr 3.16e-04 | 102,466 tok/s | 54.64 GB | 10.436B tok iter 79640 | train 2.782 | val 2.767 | lr 3.16e-04 | 102,468 tok/s | 54.64 GB | 10.439B tok iter 79660 | train 2.836 | val 2.793 | lr 3.15e-04 | 102,470 tok/s | 54.64 GB | 10.441B tok iter 79680 | train 2.891 | val 2.811 | lr 3.15e-04 | 102,472 tok/s | 54.64 GB | 10.444B tok iter 79700 | train 2.757 | val 2.798 | lr 3.15e-04 | 102,473 tok/s | 54.64 GB | 10.447B tok iter 79720 | train 2.849 | val 2.797 | lr 3.15e-04 | 102,475 tok/s | 54.64 GB | 10.449B tok iter 79740 | train 2.773 | val 2.744 | lr 3.15e-04 | 102,477 tok/s | 54.64 GB | 10.452B tok iter 79760 | train 2.643 | val 2.723 | lr 3.15e-04 | 102,479 tok/s | 54.64 GB | 10.454B tok iter 79780 | train 2.654 | val 2.799 | lr 3.15e-04 | 102,479 tok/s | 54.64 GB | 10.457B tok iter 79800 | train 2.784 | val 2.793 | lr 3.15e-04 | 102,481 tok/s | 54.64 GB | 10.460B tok iter 79820 | train 2.650 | val 2.848 | lr 3.14e-04 | 102,483 tok/s | 54.64 GB | 10.462B tok iter 79840 | train 2.853 | val 2.774 | lr 3.14e-04 | 102,485 tok/s | 54.64 GB | 10.465B tok iter 79860 | train 2.693 | val 2.842 | lr 3.14e-04 | 102,486 tok/s | 54.64 GB | 10.468B tok iter 79880 | train 2.827 | val 2.773 | lr 3.14e-04 | 102,488 tok/s | 54.64 GB | 10.470B tok iter 79900 | train 2.858 | val 2.799 | lr 3.14e-04 | 102,490 tok/s | 54.64 GB | 10.473B tok iter 79920 | train 2.741 | val 2.729 | lr 3.14e-04 | 102,492 tok/s | 54.64 GB | 10.475B tok iter 79940 | train 2.828 | val 2.791 | lr 3.14e-04 | 102,492 tok/s | 54.64 GB | 10.478B tok iter 79960 | train 2.805 | val 2.790 | lr 3.14e-04 | 102,494 tok/s | 54.64 GB | 10.481B tok iter 79980 | train 2.806 | val 2.772 | lr 3.14e-04 | 102,496 tok/s | 54.64 GB | 10.483B tok checkpoint @ iter 79991 -> checkpoints/500m_latest.pt iter 80000 | train 2.721 | val 2.726 | lr 3.13e-04 | 102,484 tok/s | 54.64 GB | 10.486B tok checkpoint @ iter 80000 -> checkpoints/500m_latest.pt iter 80020 | train 2.807 | val 2.807 | lr 3.13e-04 | 102,458 tok/s | 54.64 GB | 10.489B tok iter 80040 | train 2.715 | val 2.764 | lr 3.13e-04 | 102,460 tok/s | 54.64 GB | 10.491B tok iter 80060 | train 2.855 | val 2.811 | lr 3.13e-04 | 102,462 tok/s | 54.64 GB | 10.494B tok iter 80080 | train 2.688 | val 2.748 | lr 3.13e-04 | 102,464 tok/s | 54.64 GB | 10.496B tok iter 80100 | train 2.800 | val 2.771 | lr 3.13e-04 | 102,464 tok/s | 54.64 GB | 10.499B tok iter 80120 | train 2.742 | val 2.776 | lr 3.13e-04 | 102,466 tok/s | 54.64 GB | 10.502B tok iter 80140 | train 2.821 | val 2.756 | lr 3.13e-04 | 102,468 tok/s | 54.64 GB | 10.504B tok iter 80160 | train 2.784 | val 2.785 | lr 3.13e-04 | 102,470 tok/s | 54.64 GB | 10.507B tok iter 80180 | train 2.811 | val 2.746 | lr 3.12e-04 | 102,471 tok/s | 54.64 GB | 10.509B tok iter 80200 | train 2.744 | val 2.737 | lr 3.12e-04 | 102,473 tok/s | 54.64 GB | 10.512B tok iter 80220 | train 2.933 | val 2.724 | lr 3.12e-04 | 102,475 tok/s | 54.64 GB | 10.515B tok iter 80240 | train 2.864 | val 2.768 | lr 3.12e-04 | 102,477 tok/s | 54.64 GB | 10.517B tok iter 80260 | train 2.690 | val 2.782 | lr 3.12e-04 | 102,477 tok/s | 54.64 GB | 10.520B tok iter 80280 | train 2.801 | val 2.790 | lr 3.12e-04 | 102,479 tok/s | 54.64 GB | 10.523B tok iter 80300 | train 2.903 | val 2.801 | lr 3.12e-04 | 102,481 tok/s | 54.64 GB | 10.525B tok iter 80320 | train 2.782 | val 2.715 | lr 3.12e-04 | 102,483 tok/s | 54.64 GB | 10.528B tok iter 80340 | train 2.711 | val 2.796 | lr 3.12e-04 | 102,483 tok/s | 54.64 GB | 10.530B tok iter 80360 | train 2.756 | val 2.763 | lr 3.11e-04 | 102,485 tok/s | 54.64 GB | 10.533B tok iter 80380 | train 2.824 | val 2.767 | lr 3.11e-04 | 102,487 tok/s | 54.64 GB | 10.536B tok iter 80400 | train 2.893 | val 2.762 | lr 3.11e-04 | 102,489 tok/s | 54.64 GB | 10.538B tok iter 80420 | train 2.891 | val 2.789 | lr 3.11e-04 | 102,489 tok/s | 54.64 GB | 10.541B tok iter 80440 | train 2.724 | val 2.830 | lr 3.11e-04 | 102,491 tok/s | 54.64 GB | 10.544B tok iter 80460 | train 2.917 | val 2.819 | lr 3.11e-04 | 102,493 tok/s | 54.64 GB | 10.546B tok iter 80480 | train 2.802 | val 2.799 | lr 3.11e-04 | 102,495 tok/s | 54.64 GB | 10.549B tok checkpoint @ iter 80490 -> checkpoints/500m_latest.pt iter 80500 | train 2.676 | val 2.749 | lr 3.11e-04 | 102,482 tok/s | 54.64 GB | 10.551B tok checkpoint @ iter 80500 -> checkpoints/500m_latest.pt iter 80520 | train 2.820 | val 2.769 | lr 3.11e-04 | 102,458 tok/s | 54.64 GB | 10.554B tok iter 80540 | train 2.848 | val 2.792 | lr 3.10e-04 | 102,460 tok/s | 54.64 GB | 10.557B tok iter 80560 | train 2.739 | val 2.802 | lr 3.10e-04 | 102,462 tok/s | 54.64 GB | 10.559B tok iter 80580 | train 2.724 | val 2.789 | lr 3.10e-04 | 102,462 tok/s | 54.64 GB | 10.562B tok iter 80600 | train 2.710 | val 2.787 | lr 3.10e-04 | 102,464 tok/s | 54.64 GB | 10.565B tok iter 80620 | train 2.747 | val 2.822 | lr 3.10e-04 | 102,466 tok/s | 54.64 GB | 10.567B tok iter 80640 | train 2.699 | val 2.758 | lr 3.10e-04 | 102,468 tok/s | 54.64 GB | 10.570B tok iter 80660 | train 2.681 | val 2.813 | lr 3.10e-04 | 102,469 tok/s | 54.64 GB | 10.572B tok iter 80680 | train 2.801 | val 2.741 | lr 3.10e-04 | 102,471 tok/s | 54.64 GB | 10.575B tok iter 80700 | train 2.688 | val 2.776 | lr 3.10e-04 | 102,473 tok/s | 54.64 GB | 10.578B tok iter 80720 | train 2.733 | val 2.806 | lr 3.09e-04 | 102,475 tok/s | 54.64 GB | 10.580B tok iter 80740 | train 2.747 | val 2.782 | lr 3.09e-04 | 102,475 tok/s | 54.64 GB | 10.583B tok iter 80760 | train 2.743 | val 2.750 | lr 3.09e-04 | 102,477 tok/s | 54.64 GB | 10.586B tok iter 80780 | train 2.747 | val 2.774 | lr 3.09e-04 | 102,479 tok/s | 54.64 GB | 10.588B tok iter 80800 | train 2.740 | val 2.764 | lr 3.09e-04 | 102,481 tok/s | 54.64 GB | 10.591B tok iter 80820 | train 2.773 | val 2.762 | lr 3.09e-04 | 102,481 tok/s | 54.64 GB | 10.593B tok iter 80840 | train 2.834 | val 2.757 | lr 3.09e-04 | 102,483 tok/s | 54.64 GB | 10.596B tok iter 80860 | train 2.607 | val 2.769 | lr 3.09e-04 | 102,485 tok/s | 54.64 GB | 10.599B tok iter 80880 | train 2.755 | val 2.712 | lr 3.09e-04 | 102,487 tok/s | 54.64 GB | 10.601B tok iter 80900 | train 2.702 | val 2.808 | lr 3.08e-04 | 102,487 tok/s | 54.64 GB | 10.604B tok iter 80920 | train 2.833 | val 2.792 | lr 3.08e-04 | 102,489 tok/s | 54.64 GB | 10.606B tok iter 80940 | train 2.813 | val 2.760 | lr 3.08e-04 | 102,491 tok/s | 54.64 GB | 10.609B tok iter 80960 | train 2.729 | val 2.813 | lr 3.08e-04 | 102,493 tok/s | 54.64 GB | 10.612B tok iter 80980 | train 2.826 | val 2.783 | lr 3.08e-04 | 102,494 tok/s | 54.64 GB | 10.614B tok checkpoint @ iter 80990 -> checkpoints/500m_latest.pt iter 81000 | train 2.762 | val 2.779 | lr 3.08e-04 | 102,482 tok/s | 54.64 GB | 10.617B tok checkpoint @ iter 81000 -> checkpoints/500m_latest.pt iter 81020 | train 2.863 | val 2.709 | lr 3.08e-04 | 102,458 tok/s | 54.64 GB | 10.620B tok iter 81040 | train 2.830 | val 2.783 | lr 3.08e-04 | 102,460 tok/s | 54.64 GB | 10.622B tok iter 81060 | train 2.956 | val 2.765 | lr 3.08e-04 | 102,460 tok/s | 54.64 GB | 10.625B tok iter 81080 | train 2.807 | val 2.779 | lr 3.07e-04 | 102,462 tok/s | 54.64 GB | 10.627B tok iter 81100 | train 2.796 | val 2.791 | lr 3.07e-04 | 102,464 tok/s | 54.64 GB | 10.630B tok iter 81120 | train 2.890 | val 2.851 | lr 3.07e-04 | 102,466 tok/s | 54.64 GB | 10.633B tok iter 81140 | train 2.634 | val 2.794 | lr 3.07e-04 | 102,467 tok/s | 54.64 GB | 10.635B tok iter 81160 | train 2.687 | val 2.785 | lr 3.07e-04 | 102,469 tok/s | 54.64 GB | 10.638B tok iter 81180 | train 2.763 | val 2.786 | lr 3.07e-04 | 102,470 tok/s | 54.64 GB | 10.641B tok iter 81200 | train 2.780 | val 2.823 | lr 3.07e-04 | 102,472 tok/s | 54.64 GB | 10.643B tok iter 81220 | train 2.822 | val 2.743 | lr 3.07e-04 | 102,473 tok/s | 54.64 GB | 10.646B tok iter 81240 | train 2.823 | val 2.790 | lr 3.07e-04 | 102,475 tok/s | 54.64 GB | 10.648B tok iter 81260 | train 2.820 | val 2.761 | lr 3.06e-04 | 102,477 tok/s | 54.64 GB | 10.651B tok iter 81280 | train 2.782 | val 2.767 | lr 3.06e-04 | 102,479 tok/s | 54.64 GB | 10.654B tok iter 81300 | train 2.853 | val 2.813 | lr 3.06e-04 | 102,479 tok/s | 54.64 GB | 10.656B tok iter 81320 | train 2.852 | val 2.775 | lr 3.06e-04 | 102,481 tok/s | 54.64 GB | 10.659B tok iter 81340 | train 2.654 | val 2.801 | lr 3.06e-04 | 102,483 tok/s | 54.64 GB | 10.662B tok iter 81360 | train 2.905 | val 2.781 | lr 3.06e-04 | 102,485 tok/s | 54.64 GB | 10.664B tok iter 81380 | train 2.792 | val 2.780 | lr 3.06e-04 | 102,485 tok/s | 54.64 GB | 10.667B tok iter 81400 | train 2.745 | val 2.760 | lr 3.06e-04 | 102,487 tok/s | 54.64 GB | 10.669B tok iter 81420 | train 2.703 | val 2.785 | lr 3.06e-04 | 102,489 tok/s | 54.64 GB | 10.672B tok iter 81440 | train 2.720 | val 2.793 | lr 3.05e-04 | 102,491 tok/s | 54.64 GB | 10.675B tok iter 81460 | train 2.636 | val 2.771 | lr 3.05e-04 | 102,491 tok/s | 54.64 GB | 10.677B tok iter 81480 | train 2.718 | val 2.795 | lr 3.05e-04 | 102,493 tok/s | 54.64 GB | 10.680B tok checkpoint @ iter 81490 -> checkpoints/500m_latest.pt iter 81500 | train 2.712 | val 2.825 | lr 3.05e-04 | 102,482 tok/s | 54.64 GB | 10.682B tok checkpoint @ iter 81500 -> checkpoints/500m_latest.pt iter 81520 | train 2.702 | val 2.778 | lr 3.05e-04 | 102,458 tok/s | 54.64 GB | 10.685B tok iter 81540 | train 2.775 | val 2.773 | lr 3.05e-04 | 102,458 tok/s | 54.64 GB | 10.688B tok iter 81560 | train 2.690 | val 2.764 | lr 3.05e-04 | 102,460 tok/s | 54.64 GB | 10.690B tok iter 81580 | train 2.781 | val 2.755 | lr 3.05e-04 | 102,462 tok/s | 54.64 GB | 10.693B tok iter 81600 | train 2.859 | val 2.736 | lr 3.05e-04 | 102,464 tok/s | 54.64 GB | 10.696B tok iter 81620 | train 2.784 | val 2.761 | lr 3.04e-04 | 102,464 tok/s | 54.64 GB | 10.698B tok iter 81640 | train 2.676 | val 2.803 | lr 3.04e-04 | 102,466 tok/s | 54.64 GB | 10.701B tok iter 81660 | train 2.740 | val 2.759 | lr 3.04e-04 | 102,468 tok/s | 54.64 GB | 10.703B tok iter 81680 | train 2.769 | val 2.755 | lr 3.04e-04 | 102,470 tok/s | 54.64 GB | 10.706B tok iter 81700 | train 2.729 | val 2.772 | lr 3.04e-04 | 102,470 tok/s | 54.64 GB | 10.709B tok iter 81720 | train 2.839 | val 2.750 | lr 3.04e-04 | 102,472 tok/s | 54.64 GB | 10.711B tok iter 81740 | train 2.856 | val 2.765 | lr 3.04e-04 | 102,474 tok/s | 54.64 GB | 10.714B tok iter 81760 | train 2.844 | val 2.783 | lr 3.04e-04 | 102,476 tok/s | 54.64 GB | 10.717B tok iter 81780 | train 2.793 | val 2.747 | lr 3.04e-04 | 102,476 tok/s | 54.64 GB | 10.719B tok iter 81800 | train 2.829 | val 2.779 | lr 3.03e-04 | 102,478 tok/s | 54.64 GB | 10.722B tok iter 81820 | train 2.847 | val 2.771 | lr 3.03e-04 | 102,480 tok/s | 54.64 GB | 10.724B tok iter 81840 | train 2.808 | val 2.821 | lr 3.03e-04 | 102,482 tok/s | 54.64 GB | 10.727B tok iter 81860 | train 2.768 | val 2.694 | lr 3.03e-04 | 102,483 tok/s | 54.64 GB | 10.730B tok iter 81880 | train 2.795 | val 2.804 | lr 3.03e-04 | 102,485 tok/s | 54.64 GB | 10.732B tok iter 81900 | train 2.794 | val 2.779 | lr 3.03e-04 | 102,487 tok/s | 54.64 GB | 10.735B tok iter 81920 | train 2.811 | val 2.714 | lr 3.03e-04 | 102,489 tok/s | 54.64 GB | 10.738B tok iter 81940 | train 2.660 | val 2.798 | lr 3.03e-04 | 102,489 tok/s | 54.64 GB | 10.740B tok iter 81960 | train 2.821 | val 2.789 | lr 3.03e-04 | 102,491 tok/s | 54.64 GB | 10.743B tok iter 81980 | train 2.843 | val 2.720 | lr 3.02e-04 | 102,493 tok/s | 54.64 GB | 10.745B tok checkpoint @ iter 81990 -> checkpoints/500m_latest.pt iter 82000 | train 2.746 | val 2.804 | lr 3.02e-04 | 102,482 tok/s | 54.64 GB | 10.748B tok checkpoint @ iter 82000 -> checkpoints/500m_latest.pt iter 82020 | train 2.741 | val 2.782 | lr 3.02e-04 | 102,457 tok/s | 54.64 GB | 10.751B tok iter 82040 | train 2.726 | val 2.784 | lr 3.02e-04 | 102,459 tok/s | 54.64 GB | 10.753B tok iter 82060 | train 2.800 | val 2.706 | lr 3.02e-04 | 102,461 tok/s | 54.64 GB | 10.756B tok iter 82080 | train 2.759 | val 2.788 | lr 3.02e-04 | 102,463 tok/s | 54.64 GB | 10.759B tok iter 82100 | train 2.723 | val 2.808 | lr 3.02e-04 | 102,463 tok/s | 54.64 GB | 10.761B tok iter 82120 | train 2.745 | val 2.757 | lr 3.02e-04 | 102,465 tok/s | 54.64 GB | 10.764B tok iter 82140 | train 2.799 | val 2.751 | lr 3.02e-04 | 102,467 tok/s | 54.64 GB | 10.766B tok iter 82160 | train 2.919 | val 2.732 | lr 3.01e-04 | 102,469 tok/s | 54.64 GB | 10.769B tok iter 82180 | train 2.682 | val 2.737 | lr 3.01e-04 | 102,470 tok/s | 54.64 GB | 10.772B tok iter 82200 | train 2.734 | val 2.825 | lr 3.01e-04 | 102,472 tok/s | 54.64 GB | 10.774B tok iter 82220 | train 2.872 | val 2.787 | lr 3.01e-04 | 102,473 tok/s | 54.64 GB | 10.777B tok iter 82240 | train 2.696 | val 2.763 | lr 3.01e-04 | 102,475 tok/s | 54.64 GB | 10.779B tok iter 82260 | train 2.895 | val 2.788 | lr 3.01e-04 | 102,476 tok/s | 54.64 GB | 10.782B tok iter 82280 | train 2.790 | val 2.817 | lr 3.01e-04 | 102,478 tok/s | 54.64 GB | 10.785B tok iter 82300 | train 2.891 | val 2.785 | lr 3.01e-04 | 102,480 tok/s | 54.64 GB | 10.787B tok iter 82320 | train 2.771 | val 2.775 | lr 3.01e-04 | 102,482 tok/s | 54.64 GB | 10.790B tok iter 82340 | train 2.750 | val 2.719 | lr 3.00e-04 | 102,482 tok/s | 54.64 GB | 10.793B tok iter 82360 | train 2.750 | val 2.790 | lr 3.00e-04 | 102,484 tok/s | 54.64 GB | 10.795B tok iter 82380 | train 2.692 | val 2.737 | lr 3.00e-04 | 102,486 tok/s | 54.64 GB | 10.798B tok iter 82400 | train 2.845 | val 2.778 | lr 3.00e-04 | 102,488 tok/s | 54.64 GB | 10.800B tok iter 82420 | train 2.762 | val 2.759 | lr 3.00e-04 | 102,488 tok/s | 54.64 GB | 10.803B tok iter 82440 | train 2.832 | val 2.790 | lr 3.00e-04 | 102,490 tok/s | 54.64 GB | 10.806B tok iter 82460 | train 2.845 | val 2.749 | lr 3.00e-04 | 102,492 tok/s | 54.64 GB | 10.808B tok iter 82480 | train 2.794 | val 2.783 | lr 3.00e-04 | 102,494 tok/s | 54.64 GB | 10.811B tok checkpoint @ iter 82491 -> checkpoints/500m_latest.pt iter 82500 | train 2.785 | val 2.788 | lr 2.99e-04 | 102,481 tok/s | 54.64 GB | 10.814B tok checkpoint @ iter 82500 -> checkpoints/500m_latest.pt iter 82520 | train 2.867 | val 2.770 | lr 2.99e-04 | 102,457 tok/s | 54.64 GB | 10.816B tok iter 82540 | train 2.820 | val 2.785 | lr 2.99e-04 | 102,459 tok/s | 54.64 GB | 10.819B tok iter 82560 | train 2.697 | val 2.783 | lr 2.99e-04 | 102,461 tok/s | 54.64 GB | 10.821B tok iter 82580 | train 2.830 | val 2.803 | lr 2.99e-04 | 102,461 tok/s | 54.64 GB | 10.824B tok iter 82600 | train 2.769 | val 2.781 | lr 2.99e-04 | 102,463 tok/s | 54.64 GB | 10.827B tok iter 82620 | train 2.675 | val 2.815 | lr 2.99e-04 | 102,465 tok/s | 54.64 GB | 10.829B tok iter 82640 | train 2.835 | val 2.733 | lr 2.99e-04 | 102,467 tok/s | 54.64 GB | 10.832B tok iter 82660 | train 2.741 | val 2.804 | lr 2.99e-04 | 102,467 tok/s | 54.64 GB | 10.835B tok iter 82680 | train 2.909 | val 2.794 | lr 2.98e-04 | 102,469 tok/s | 54.64 GB | 10.837B tok iter 82700 | train 2.919 | val 2.788 | lr 2.98e-04 | 102,471 tok/s | 54.64 GB | 10.840B tok iter 82720 | train 2.627 | val 2.785 | lr 2.98e-04 | 102,473 tok/s | 54.64 GB | 10.842B tok iter 82740 | train 2.664 | val 2.793 | lr 2.98e-04 | 102,473 tok/s | 54.64 GB | 10.845B tok iter 82760 | train 2.752 | val 2.834 | lr 2.98e-04 | 102,475 tok/s | 54.64 GB | 10.848B tok iter 82780 | train 2.822 | val 2.754 | lr 2.98e-04 | 102,477 tok/s | 54.64 GB | 10.850B tok iter 82800 | train 2.775 | val 2.760 | lr 2.98e-04 | 102,479 tok/s | 54.64 GB | 10.853B tok iter 82820 | train 2.763 | val 2.799 | lr 2.98e-04 | 102,479 tok/s | 54.64 GB | 10.856B tok iter 82840 | train 2.746 | val 2.779 | lr 2.98e-04 | 102,481 tok/s | 54.64 GB | 10.858B tok iter 82860 | train 2.710 | val 2.807 | lr 2.97e-04 | 102,483 tok/s | 54.64 GB | 10.861B tok iter 82880 | train 2.788 | val 2.732 | lr 2.97e-04 | 102,485 tok/s | 54.64 GB | 10.863B tok iter 82900 | train 2.771 | val 2.782 | lr 2.97e-04 | 102,485 tok/s | 54.64 GB | 10.866B tok iter 82920 | train 2.777 | val 2.755 | lr 2.97e-04 | 102,487 tok/s | 54.64 GB | 10.869B tok iter 82940 | train 2.730 | val 2.771 | lr 2.97e-04 | 102,489 tok/s | 54.64 GB | 10.871B tok iter 82960 | train 2.832 | val 2.783 | lr 2.97e-04 | 102,491 tok/s | 54.64 GB | 10.874B tok iter 82980 | train 2.700 | val 2.801 | lr 2.97e-04 | 102,492 tok/s | 54.64 GB | 10.876B tok checkpoint @ iter 82990 -> checkpoints/500m_latest.pt iter 83000 | train 2.765 | val 2.783 | lr 2.97e-04 | 102,481 tok/s | 54.64 GB | 10.879B tok checkpoint @ iter 83000 -> checkpoints/500m_latest.pt iter 83020 | train 2.713 | val 2.800 | lr 2.97e-04 | 102,460 tok/s | 54.64 GB | 10.882B tok iter 83040 | train 2.759 | val 2.781 | lr 2.96e-04 | 102,462 tok/s | 54.64 GB | 10.884B tok iter 83060 | train 2.884 | val 2.801 | lr 2.96e-04 | 102,463 tok/s | 54.64 GB | 10.887B tok iter 83080 | train 2.742 | val 2.752 | lr 2.96e-04 | 102,465 tok/s | 54.64 GB | 10.890B tok iter 83100 | train 2.931 | val 2.831 | lr 2.96e-04 | 102,466 tok/s | 54.64 GB | 10.892B tok iter 83120 | train 2.810 | val 2.726 | lr 2.96e-04 | 102,468 tok/s | 54.64 GB | 10.895B tok iter 83140 | train 2.775 | val 2.751 | lr 2.96e-04 | 102,469 tok/s | 54.64 GB | 10.897B tok iter 83160 | train 2.687 | val 2.749 | lr 2.96e-04 | 102,471 tok/s | 54.64 GB | 10.900B tok iter 83180 | train 2.762 | val 2.712 | lr 2.96e-04 | 102,473 tok/s | 54.64 GB | 10.903B tok iter 83200 | train 2.834 | val 2.742 | lr 2.96e-04 | 102,475 tok/s | 54.64 GB | 10.905B tok iter 83220 | train 2.721 | val 2.764 | lr 2.95e-04 | 102,475 tok/s | 54.64 GB | 10.908B tok iter 83240 | train 2.841 | val 2.758 | lr 2.95e-04 | 102,477 tok/s | 54.64 GB | 10.911B tok iter 83260 | train 2.754 | val 2.753 | lr 2.95e-04 | 102,479 tok/s | 54.64 GB | 10.913B tok iter 83280 | train 2.796 | val 2.808 | lr 2.95e-04 | 102,481 tok/s | 54.64 GB | 10.916B tok iter 83300 | train 2.776 | val 2.729 | lr 2.95e-04 | 102,481 tok/s | 54.64 GB | 10.918B tok iter 83320 | train 2.744 | val 2.757 | lr 2.95e-04 | 102,483 tok/s | 54.64 GB | 10.921B tok iter 83340 | train 2.854 | val 2.776 | lr 2.95e-04 | 102,485 tok/s | 54.64 GB | 10.924B tok iter 83360 | train 2.635 | val 2.770 | lr 2.95e-04 | 102,487 tok/s | 54.64 GB | 10.926B tok iter 83380 | train 2.823 | val 2.777 | lr 2.95e-04 | 102,487 tok/s | 54.64 GB | 10.929B tok iter 83400 | train 2.848 | val 2.811 | lr 2.94e-04 | 102,489 tok/s | 54.64 GB | 10.932B tok iter 83420 | train 2.786 | val 2.758 | lr 2.94e-04 | 102,491 tok/s | 54.64 GB | 10.934B tok iter 83440 | train 2.786 | val 2.729 | lr 2.94e-04 | 102,493 tok/s | 54.64 GB | 10.937B tok iter 83460 | train 2.748 | val 2.756 | lr 2.94e-04 | 102,493 tok/s | 54.64 GB | 10.939B tok iter 83480 | train 2.803 | val 2.793 | lr 2.94e-04 | 102,495 tok/s | 54.64 GB | 10.942B tok checkpoint @ iter 83492 -> checkpoints/500m_latest.pt iter 83500 | train 2.697 | val 2.685 | lr 2.94e-04 | 102,485 tok/s | 54.64 GB | 10.945B tok checkpoint @ iter 83500 -> checkpoints/500m_latest.pt iter 83520 | train 2.804 | val 2.782 | lr 2.94e-04 | 102,461 tok/s | 54.64 GB | 10.947B tok iter 83540 | train 2.707 | val 2.772 | lr 2.94e-04 | 102,461 tok/s | 54.64 GB | 10.950B tok iter 83560 | train 2.775 | val 2.788 | lr 2.94e-04 | 102,463 tok/s | 54.64 GB | 10.953B tok iter 83580 | train 2.677 | val 2.750 | lr 2.93e-04 | 102,465 tok/s | 54.64 GB | 10.955B tok iter 83600 | train 2.801 | val 2.759 | lr 2.93e-04 | 102,467 tok/s | 54.64 GB | 10.958B tok iter 83620 | train 2.828 | val 2.777 | lr 2.93e-04 | 102,467 tok/s | 54.64 GB | 10.960B tok iter 83640 | train 2.859 | val 2.720 | lr 2.93e-04 | 102,469 tok/s | 54.64 GB | 10.963B tok iter 83660 | train 2.805 | val 2.793 | lr 2.93e-04 | 102,471 tok/s | 54.64 GB | 10.966B tok iter 83680 | train 2.798 | val 2.759 | lr 2.93e-04 | 102,473 tok/s | 54.64 GB | 10.968B tok iter 83700 | train 2.703 | val 2.802 | lr 2.93e-04 | 102,473 tok/s | 54.64 GB | 10.971B tok iter 83720 | train 2.804 | val 2.785 | lr 2.93e-04 | 102,475 tok/s | 54.64 GB | 10.973B tok iter 83740 | train 2.852 | val 2.802 | lr 2.93e-04 | 102,477 tok/s | 54.64 GB | 10.976B tok iter 83760 | train 2.922 | val 2.759 | lr 2.92e-04 | 102,479 tok/s | 54.64 GB | 10.979B tok iter 83780 | train 2.762 | val 2.701 | lr 2.92e-04 | 102,479 tok/s | 54.64 GB | 10.981B tok iter 83800 | train 2.768 | val 2.749 | lr 2.92e-04 | 102,481 tok/s | 54.64 GB | 10.984B tok iter 83820 | train 2.667 | val 2.771 | lr 2.92e-04 | 102,483 tok/s | 54.64 GB | 10.987B tok iter 83840 | train 2.732 | val 2.793 | lr 2.92e-04 | 102,485 tok/s | 54.64 GB | 10.989B tok iter 83860 | train 2.751 | val 2.724 | lr 2.92e-04 | 102,485 tok/s | 54.64 GB | 10.992B tok iter 83880 | train 2.867 | val 2.797 | lr 2.92e-04 | 102,487 tok/s | 54.64 GB | 10.994B tok iter 83900 | train 2.752 | val 2.790 | lr 2.92e-04 | 102,489 tok/s | 54.64 GB | 10.997B tok iter 83920 | train 2.703 | val 2.801 | lr 2.92e-04 | 102,491 tok/s | 54.64 GB | 11.000B tok iter 83940 | train 2.737 | val 2.684 | lr 2.91e-04 | 102,491 tok/s | 54.64 GB | 11.002B tok iter 83960 | train 2.755 | val 2.709 | lr 2.91e-04 | 102,493 tok/s | 54.64 GB | 11.005B tok iter 83980 | train 2.745 | val 2.744 | lr 2.91e-04 | 102,495 tok/s | 54.64 GB | 11.008B tok checkpoint @ iter 83990 -> checkpoints/500m_latest.pt iter 84000 | train 2.644 | val 2.738 | lr 2.91e-04 | 102,485 tok/s | 54.64 GB | 11.010B tok checkpoint @ iter 84000 -> checkpoints/500m_latest.pt iter 84020 | train 2.749 | val 2.780 | lr 2.91e-04 | 102,460 tok/s | 54.64 GB | 11.013B tok iter 84040 | train 2.764 | val 2.771 | lr 2.91e-04 | 102,462 tok/s | 54.64 GB | 11.015B tok iter 84060 | train 2.795 | val 2.769 | lr 2.91e-04 | 102,464 tok/s | 54.64 GB | 11.018B tok iter 84080 | train 2.781 | val 2.782 | lr 2.91e-04 | 102,466 tok/s | 54.64 GB | 11.021B tok iter 84100 | train 2.658 | val 2.797 | lr 2.91e-04 | 102,466 tok/s | 54.64 GB | 11.023B tok iter 84120 | train 2.785 | val 2.762 | lr 2.90e-04 | 102,468 tok/s | 54.64 GB | 11.026B tok iter 84140 | train 2.746 | val 2.734 | lr 2.90e-04 | 102,470 tok/s | 54.64 GB | 11.029B tok iter 84160 | train 2.677 | val 2.767 | lr 2.90e-04 | 102,472 tok/s | 54.64 GB | 11.031B tok iter 84180 | train 2.725 | val 2.755 | lr 2.90e-04 | 102,472 tok/s | 54.64 GB | 11.034B tok iter 84200 | train 2.735 | val 2.732 | lr 2.90e-04 | 102,474 tok/s | 54.64 GB | 11.036B tok iter 84220 | train 2.718 | val 2.795 | lr 2.90e-04 | 102,476 tok/s | 54.64 GB | 11.039B tok iter 84240 | train 2.770 | val 2.739 | lr 2.90e-04 | 102,478 tok/s | 54.64 GB | 11.042B tok iter 84260 | train 2.516 | val 2.781 | lr 2.90e-04 | 102,478 tok/s | 54.64 GB | 11.044B tok iter 84280 | train 2.753 | val 2.758 | lr 2.90e-04 | 102,480 tok/s | 54.64 GB | 11.047B tok iter 84300 | train 2.723 | val 2.760 | lr 2.89e-04 | 102,482 tok/s | 54.64 GB | 11.050B tok iter 84320 | train 2.832 | val 2.725 | lr 2.89e-04 | 102,484 tok/s | 54.64 GB | 11.052B tok iter 84340 | train 2.774 | val 2.774 | lr 2.89e-04 | 102,484 tok/s | 54.64 GB | 11.055B tok iter 84360 | train 2.811 | val 2.769 | lr 2.89e-04 | 102,486 tok/s | 54.64 GB | 11.057B tok iter 84380 | train 2.779 | val 2.746 | lr 2.89e-04 | 102,488 tok/s | 54.64 GB | 11.060B tok iter 84400 | train 2.805 | val 2.813 | lr 2.89e-04 | 102,490 tok/s | 54.64 GB | 11.063B tok iter 84420 | train 2.782 | val 2.803 | lr 2.89e-04 | 102,490 tok/s | 54.64 GB | 11.065B tok iter 84440 | train 2.797 | val 2.773 | lr 2.89e-04 | 102,492 tok/s | 54.64 GB | 11.068B tok iter 84460 | train 2.791 | val 2.788 | lr 2.89e-04 | 102,494 tok/s | 54.64 GB | 11.070B tok iter 84480 | train 2.866 | val 2.760 | lr 2.88e-04 | 102,496 tok/s | 54.64 GB | 11.073B tok checkpoint @ iter 84490 -> checkpoints/500m_latest.pt iter 84500 | train 2.814 | val 2.726 | lr 2.88e-04 | 102,484 tok/s | 54.64 GB | 11.076B tok checkpoint @ iter 84500 -> checkpoints/500m_latest.pt iter 84520 | train 2.790 | val 2.783 | lr 2.88e-04 | 102,460 tok/s | 54.64 GB | 11.078B tok iter 84540 | train 2.799 | val 2.769 | lr 2.88e-04 | 102,462 tok/s | 54.64 GB | 11.081B tok iter 84560 | train 2.828 | val 2.775 | lr 2.88e-04 | 102,464 tok/s | 54.64 GB | 11.084B tok iter 84580 | train 2.698 | val 2.762 | lr 2.88e-04 | 102,464 tok/s | 54.64 GB | 11.086B tok iter 84600 | train 2.797 | val 2.737 | lr 2.88e-04 | 102,466 tok/s | 54.64 GB | 11.089B tok iter 84620 | train 2.703 | val 2.754 | lr 2.88e-04 | 102,468 tok/s | 54.64 GB | 11.091B tok iter 84640 | train 2.778 | val 2.776 | lr 2.88e-04 | 102,470 tok/s | 54.64 GB | 11.094B tok iter 84660 | train 2.852 | val 2.781 | lr 2.87e-04 | 102,470 tok/s | 54.64 GB | 11.097B tok iter 84680 | train 2.800 | val 2.776 | lr 2.87e-04 | 102,472 tok/s | 54.64 GB | 11.099B tok iter 84700 | train 2.760 | val 2.725 | lr 2.87e-04 | 102,474 tok/s | 54.64 GB | 11.102B tok iter 84720 | train 2.829 | val 2.785 | lr 2.87e-04 | 102,476 tok/s | 54.64 GB | 11.105B tok iter 84740 | train 2.780 | val 2.757 | lr 2.87e-04 | 102,476 tok/s | 54.64 GB | 11.107B tok iter 84760 | train 2.771 | val 2.735 | lr 2.87e-04 | 102,478 tok/s | 54.64 GB | 11.110B tok iter 84780 | train 2.799 | val 2.733 | lr 2.87e-04 | 102,480 tok/s | 54.64 GB | 11.112B tok iter 84800 | train 2.675 | val 2.774 | lr 2.87e-04 | 102,482 tok/s | 54.64 GB | 11.115B tok iter 84820 | train 2.745 | val 2.747 | lr 2.87e-04 | 102,482 tok/s | 54.64 GB | 11.118B tok iter 84840 | train 2.667 | val 2.763 | lr 2.86e-04 | 102,484 tok/s | 54.64 GB | 11.120B tok iter 84860 | train 2.746 | val 2.748 | lr 2.86e-04 | 102,486 tok/s | 54.64 GB | 11.123B tok iter 84880 | train 2.842 | val 2.752 | lr 2.86e-04 | 102,488 tok/s | 54.64 GB | 11.126B tok iter 84900 | train 2.796 | val 2.732 | lr 2.86e-04 | 102,488 tok/s | 54.64 GB | 11.128B tok iter 84920 | train 2.682 | val 2.752 | lr 2.86e-04 | 102,490 tok/s | 54.64 GB | 11.131B tok iter 84940 | train 2.778 | val 2.813 | lr 2.86e-04 | 102,492 tok/s | 54.64 GB | 11.133B tok iter 84960 | train 2.656 | val 2.744 | lr 2.86e-04 | 102,494 tok/s | 54.64 GB | 11.136B tok iter 84980 | train 2.777 | val 2.775 | lr 2.86e-04 | 102,494 tok/s | 54.64 GB | 11.139B tok checkpoint @ iter 84990 -> checkpoints/500m_latest.pt iter 85000 | train 2.666 | val 2.709 | lr 2.86e-04 | 102,483 tok/s | 54.64 GB | 11.141B tok checkpoint @ iter 85000 -> checkpoints/500m_latest.pt iter 85020 | train 2.792 | val 2.855 | lr 2.85e-04 | 102,459 tok/s | 54.64 GB | 11.144B tok iter 85040 | train 2.678 | val 2.773 | lr 2.85e-04 | 102,461 tok/s | 54.64 GB | 11.146B tok iter 85060 | train 2.826 | val 2.796 | lr 2.85e-04 | 102,461 tok/s | 54.64 GB | 11.149B tok iter 85080 | train 2.805 | val 2.770 | lr 2.85e-04 | 102,463 tok/s | 54.64 GB | 11.152B tok iter 85100 | train 2.807 | val 2.794 | lr 2.85e-04 | 102,465 tok/s | 54.64 GB | 11.154B tok iter 85120 | train 2.753 | val 2.751 | lr 2.85e-04 | 102,467 tok/s | 54.64 GB | 11.157B tok iter 85140 | train 2.776 | val 2.780 | lr 2.85e-04 | 102,467 tok/s | 54.64 GB | 11.160B tok iter 85160 | train 2.733 | val 2.762 | lr 2.85e-04 | 102,469 tok/s | 54.64 GB | 11.162B tok iter 85180 | train 2.747 | val 2.796 | lr 2.85e-04 | 102,471 tok/s | 54.64 GB | 11.165B tok iter 85200 | train 2.760 | val 2.808 | lr 2.84e-04 | 102,473 tok/s | 54.64 GB | 11.167B tok iter 85220 | train 2.709 | val 2.775 | lr 2.84e-04 | 102,473 tok/s | 54.64 GB | 11.170B tok iter 85240 | train 2.687 | val 2.783 | lr 2.84e-04 | 102,475 tok/s | 54.64 GB | 11.173B tok iter 85260 | train 2.839 | val 2.751 | lr 2.84e-04 | 102,477 tok/s | 54.64 GB | 11.175B tok iter 85280 | train 2.821 | val 2.770 | lr 2.84e-04 | 102,479 tok/s | 54.64 GB | 11.178B tok iter 85300 | train 2.852 | val 2.786 | lr 2.84e-04 | 102,479 tok/s | 54.64 GB | 11.181B tok iter 85320 | train 2.815 | val 2.751 | lr 2.84e-04 | 102,481 tok/s | 54.64 GB | 11.183B tok iter 85340 | train 2.758 | val 2.792 | lr 2.84e-04 | 102,483 tok/s | 54.64 GB | 11.186B tok iter 85360 | train 2.765 | val 2.758 | lr 2.84e-04 | 102,485 tok/s | 54.64 GB | 11.188B tok iter 85380 | train 2.789 | val 2.726 | lr 2.83e-04 | 102,485 tok/s | 54.64 GB | 11.191B tok iter 85400 | train 2.698 | val 2.751 | lr 2.83e-04 | 102,487 tok/s | 54.64 GB | 11.194B tok iter 85420 | train 2.773 | val 2.782 | lr 2.83e-04 | 102,489 tok/s | 54.64 GB | 11.196B tok iter 85440 | train 2.728 | val 2.849 | lr 2.83e-04 | 102,491 tok/s | 54.64 GB | 11.199B tok iter 85460 | train 2.839 | val 2.761 | lr 2.83e-04 | 102,491 tok/s | 54.64 GB | 11.202B tok iter 85480 | train 2.684 | val 2.778 | lr 2.83e-04 | 102,493 tok/s | 54.64 GB | 11.204B tok checkpoint @ iter 85489 -> checkpoints/500m_latest.pt iter 85500 | train 2.664 | val 2.770 | lr 2.83e-04 | 102,482 tok/s | 54.64 GB | 11.207B tok checkpoint @ iter 85500 -> checkpoints/500m_latest.pt iter 85520 | train 2.769 | val 2.775 | lr 2.83e-04 | 102,460 tok/s | 54.64 GB | 11.209B tok iter 85540 | train 2.826 | val 2.769 | lr 2.83e-04 | 102,460 tok/s | 54.64 GB | 11.212B tok iter 85560 | train 2.610 | val 2.664 | lr 2.83e-04 | 102,462 tok/s | 54.64 GB | 11.215B tok iter 85580 | train 2.764 | val 2.721 | lr 2.82e-04 | 102,464 tok/s | 54.64 GB | 11.217B tok iter 85600 | train 2.786 | val 2.753 | lr 2.82e-04 | 102,466 tok/s | 54.64 GB | 11.220B tok iter 85620 | train 2.891 | val 2.813 | lr 2.82e-04 | 102,466 tok/s | 54.64 GB | 11.223B tok iter 85640 | train 2.822 | val 2.761 | lr 2.82e-04 | 102,468 tok/s | 54.64 GB | 11.225B tok iter 85660 | train 2.871 | val 2.741 | lr 2.82e-04 | 102,470 tok/s | 54.64 GB | 11.228B tok iter 85680 | train 2.811 | val 2.721 | lr 2.82e-04 | 102,472 tok/s | 54.64 GB | 11.230B tok iter 85700 | train 2.715 | val 2.795 | lr 2.82e-04 | 102,472 tok/s | 54.64 GB | 11.233B tok iter 85720 | train 2.820 | val 2.760 | lr 2.82e-04 | 102,474 tok/s | 54.64 GB | 11.236B tok iter 85740 | train 2.808 | val 2.790 | lr 2.82e-04 | 102,476 tok/s | 54.64 GB | 11.238B tok iter 85760 | train 2.768 | val 2.821 | lr 2.81e-04 | 102,478 tok/s | 54.64 GB | 11.241B tok iter 85780 | train 2.765 | val 2.762 | lr 2.81e-04 | 102,478 tok/s | 54.64 GB | 11.243B tok iter 85800 | train 2.701 | val 2.741 | lr 2.81e-04 | 102,480 tok/s | 54.64 GB | 11.246B tok iter 85820 | train 2.761 | val 2.741 | lr 2.81e-04 | 102,482 tok/s | 54.64 GB | 11.249B tok iter 85840 | train 2.744 | val 2.772 | lr 2.81e-04 | 102,484 tok/s | 54.64 GB | 11.251B tok iter 85860 | train 2.792 | val 2.788 | lr 2.81e-04 | 102,484 tok/s | 54.64 GB | 11.254B tok iter 85880 | train 2.777 | val 2.746 | lr 2.81e-04 | 102,486 tok/s | 54.64 GB | 11.257B tok iter 85900 | train 2.740 | val 2.724 | lr 2.81e-04 | 102,488 tok/s | 54.64 GB | 11.259B tok iter 85920 | train 2.636 | val 2.760 | lr 2.81e-04 | 102,490 tok/s | 54.64 GB | 11.262B tok iter 85940 | train 2.805 | val 2.720 | lr 2.80e-04 | 102,490 tok/s | 54.64 GB | 11.264B tok iter 85960 | train 2.635 | val 2.771 | lr 2.80e-04 | 102,492 tok/s | 54.64 GB | 11.267B tok iter 85980 | train 2.748 | val 2.752 | lr 2.80e-04 | 102,494 tok/s | 54.64 GB | 11.270B tok checkpoint @ iter 85991 -> checkpoints/500m_latest.pt iter 86000 | train 2.837 | val 2.768 | lr 2.80e-04 | 102,483 tok/s | 54.64 GB | 11.272B tok checkpoint @ iter 86000 -> checkpoints/500m_latest.pt iter 86020 | train 2.746 | val 2.768 | lr 2.80e-04 | 102,458 tok/s | 54.64 GB | 11.275B tok iter 86040 | train 2.811 | val 2.790 | lr 2.80e-04 | 102,460 tok/s | 54.64 GB | 11.278B tok iter 86060 | train 2.670 | val 2.767 | lr 2.80e-04 | 102,461 tok/s | 54.64 GB | 11.280B tok iter 86080 | train 2.761 | val 2.736 | lr 2.80e-04 | 102,463 tok/s | 54.64 GB | 11.283B tok iter 86100 | train 2.841 | val 2.764 | lr 2.80e-04 | 102,464 tok/s | 54.64 GB | 11.285B tok iter 86120 | train 2.570 | val 2.783 | lr 2.79e-04 | 102,466 tok/s | 54.64 GB | 11.288B tok iter 86140 | train 2.722 | val 2.780 | lr 2.79e-04 | 102,467 tok/s | 54.64 GB | 11.291B tok iter 86160 | train 2.755 | val 2.741 | lr 2.79e-04 | 102,469 tok/s | 54.64 GB | 11.293B tok iter 86180 | train 2.838 | val 2.760 | lr 2.79e-04 | 102,469 tok/s | 54.64 GB | 11.296B tok iter 86200 | train 2.737 | val 2.784 | lr 2.79e-04 | 102,471 tok/s | 54.64 GB | 11.299B tok iter 86220 | train 2.694 | val 2.806 | lr 2.79e-04 | 102,473 tok/s | 54.64 GB | 11.301B tok iter 86240 | train 2.718 | val 2.765 | lr 2.79e-04 | 102,475 tok/s | 54.64 GB | 11.304B tok iter 86260 | train 2.821 | val 2.741 | lr 2.79e-04 | 102,475 tok/s | 54.64 GB | 11.306B tok iter 86280 | train 2.695 | val 2.739 | lr 2.79e-04 | 102,477 tok/s | 54.64 GB | 11.309B tok iter 86300 | train 2.802 | val 2.785 | lr 2.78e-04 | 102,479 tok/s | 54.64 GB | 11.312B tok iter 86320 | train 2.851 | val 2.743 | lr 2.78e-04 | 102,481 tok/s | 54.64 GB | 11.314B tok iter 86340 | train 2.758 | val 2.785 | lr 2.78e-04 | 102,481 tok/s | 54.64 GB | 11.317B tok iter 86360 | train 2.795 | val 2.746 | lr 2.78e-04 | 102,483 tok/s | 54.64 GB | 11.320B tok iter 86380 | train 2.760 | val 2.755 | lr 2.78e-04 | 102,485 tok/s | 54.64 GB | 11.322B tok iter 86400 | train 2.929 | val 2.737 | lr 2.78e-04 | 102,486 tok/s | 54.64 GB | 11.325B tok iter 86420 | train 2.722 | val 2.793 | lr 2.78e-04 | 102,487 tok/s | 54.64 GB | 11.327B tok iter 86440 | train 2.761 | val 2.726 | lr 2.78e-04 | 102,489 tok/s | 54.64 GB | 11.330B tok iter 86460 | train 2.771 | val 2.741 | lr 2.78e-04 | 102,490 tok/s | 54.64 GB | 11.333B tok iter 86480 | train 2.655 | val 2.768 | lr 2.77e-04 | 102,492 tok/s | 54.64 GB | 11.335B tok checkpoint @ iter 86489 -> checkpoints/500m_latest.pt iter 86500 | train 2.718 | val 2.742 | lr 2.77e-04 | 102,480 tok/s | 54.64 GB | 11.338B tok checkpoint @ iter 86500 -> checkpoints/500m_latest.pt iter 86520 | train 2.782 | val 2.765 | lr 2.77e-04 | 102,457 tok/s | 54.64 GB | 11.340B tok iter 86540 | train 2.755 | val 2.721 | lr 2.77e-04 | 102,459 tok/s | 54.64 GB | 11.343B tok iter 86560 | train 2.745 | val 2.793 | lr 2.77e-04 | 102,461 tok/s | 54.64 GB | 11.346B tok iter 86580 | train 2.794 | val 2.756 | lr 2.77e-04 | 102,461 tok/s | 54.64 GB | 11.348B tok iter 86600 | train 2.759 | val 2.751 | lr 2.77e-04 | 102,463 tok/s | 54.64 GB | 11.351B tok iter 86620 | train 2.734 | val 2.730 | lr 2.77e-04 | 102,465 tok/s | 54.64 GB | 11.354B tok iter 86640 | train 2.767 | val 2.746 | lr 2.77e-04 | 102,466 tok/s | 54.64 GB | 11.356B tok iter 86660 | train 2.950 | val 2.787 | lr 2.76e-04 | 102,467 tok/s | 54.64 GB | 11.359B tok iter 86680 | train 2.657 | val 2.740 | lr 2.76e-04 | 102,469 tok/s | 54.64 GB | 11.361B tok iter 86700 | train 2.716 | val 2.727 | lr 2.76e-04 | 102,470 tok/s | 54.64 GB | 11.364B tok iter 86720 | train 2.792 | val 2.750 | lr 2.76e-04 | 102,472 tok/s | 54.64 GB | 11.367B tok iter 86740 | train 2.649 | val 2.755 | lr 2.76e-04 | 102,473 tok/s | 54.64 GB | 11.369B tok iter 86760 | train 2.782 | val 2.705 | lr 2.76e-04 | 102,474 tok/s | 54.64 GB | 11.372B tok iter 86780 | train 2.612 | val 2.756 | lr 2.76e-04 | 102,476 tok/s | 54.64 GB | 11.375B tok iter 86800 | train 2.703 | val 2.789 | lr 2.76e-04 | 102,478 tok/s | 54.64 GB | 11.377B tok iter 86820 | train 2.769 | val 2.743 | lr 2.76e-04 | 102,478 tok/s | 54.64 GB | 11.380B tok iter 86840 | train 2.651 | val 2.757 | lr 2.75e-04 | 102,480 tok/s | 54.64 GB | 11.382B tok iter 86860 | train 2.655 | val 2.780 | lr 2.75e-04 | 102,482 tok/s | 54.64 GB | 11.385B tok iter 86880 | train 2.910 | val 2.813 | lr 2.75e-04 | 102,484 tok/s | 54.64 GB | 11.388B tok iter 86900 | train 2.719 | val 2.764 | lr 2.75e-04 | 102,484 tok/s | 54.64 GB | 11.390B tok iter 86920 | train 2.712 | val 2.772 | lr 2.75e-04 | 102,486 tok/s | 54.64 GB | 11.393B tok iter 86940 | train 2.830 | val 2.789 | lr 2.75e-04 | 102,488 tok/s | 54.64 GB | 11.396B tok iter 86960 | train 2.762 | val 2.816 | lr 2.75e-04 | 102,490 tok/s | 54.64 GB | 11.398B tok iter 86980 | train 2.718 | val 2.762 | lr 2.75e-04 | 102,490 tok/s | 54.64 GB | 11.401B tok checkpoint @ iter 86990 -> checkpoints/500m_latest.pt iter 87000 | train 2.771 | val 2.792 | lr 2.75e-04 | 102,480 tok/s | 54.64 GB | 11.403B tok checkpoint @ iter 87000 -> checkpoints/500m_latest.pt iter 87020 | train 2.896 | val 2.714 | lr 2.74e-04 | 102,456 tok/s | 54.64 GB | 11.406B tok iter 87040 | train 2.806 | val 2.744 | lr 2.74e-04 | 102,458 tok/s | 54.64 GB | 11.409B tok iter 87060 | train 2.777 | val 2.709 | lr 2.74e-04 | 102,458 tok/s | 54.64 GB | 11.411B tok iter 87080 | train 2.815 | val 2.731 | lr 2.74e-04 | 102,460 tok/s | 54.64 GB | 11.414B tok iter 87100 | train 2.579 | val 2.717 | lr 2.74e-04 | 102,462 tok/s | 54.64 GB | 11.417B tok iter 87120 | train 2.885 | val 2.771 | lr 2.74e-04 | 102,464 tok/s | 54.64 GB | 11.419B tok iter 87140 | train 2.806 | val 2.756 | lr 2.74e-04 | 102,464 tok/s | 54.64 GB | 11.422B tok iter 87160 | train 2.689 | val 2.787 | lr 2.74e-04 | 102,466 tok/s | 54.64 GB | 11.424B tok iter 87180 | train 2.657 | val 2.761 | lr 2.74e-04 | 102,468 tok/s | 54.64 GB | 11.427B tok iter 87200 | train 2.819 | val 2.773 | lr 2.73e-04 | 102,470 tok/s | 54.64 GB | 11.430B tok iter 87220 | train 2.733 | val 2.746 | lr 2.73e-04 | 102,470 tok/s | 54.64 GB | 11.432B tok iter 87240 | train 2.659 | val 2.725 | lr 2.73e-04 | 102,472 tok/s | 54.64 GB | 11.435B tok iter 87260 | train 2.845 | val 2.783 | lr 2.73e-04 | 102,474 tok/s | 54.64 GB | 11.437B tok iter 87280 | train 2.628 | val 2.803 | lr 2.73e-04 | 102,476 tok/s | 54.64 GB | 11.440B tok iter 87300 | train 2.709 | val 2.751 | lr 2.73e-04 | 102,476 tok/s | 54.64 GB | 11.443B tok iter 87320 | train 2.737 | val 2.750 | lr 2.73e-04 | 102,478 tok/s | 54.64 GB | 11.445B tok iter 87340 | train 2.742 | val 2.750 | lr 2.73e-04 | 102,480 tok/s | 54.64 GB | 11.448B tok iter 87360 | train 2.720 | val 2.789 | lr 2.73e-04 | 102,481 tok/s | 54.64 GB | 11.451B tok iter 87380 | train 2.723 | val 2.720 | lr 2.72e-04 | 102,482 tok/s | 54.64 GB | 11.453B tok iter 87400 | train 2.895 | val 2.792 | lr 2.72e-04 | 102,484 tok/s | 54.64 GB | 11.456B tok iter 87420 | train 2.882 | val 2.752 | lr 2.72e-04 | 102,485 tok/s | 54.64 GB | 11.458B tok iter 87440 | train 2.717 | val 2.753 | lr 2.72e-04 | 102,487 tok/s | 54.64 GB | 11.461B tok iter 87460 | train 2.765 | val 2.739 | lr 2.72e-04 | 102,487 tok/s | 54.64 GB | 11.464B tok iter 87480 | train 2.835 | val 2.778 | lr 2.72e-04 | 102,489 tok/s | 54.64 GB | 11.466B tok checkpoint @ iter 87490 -> checkpoints/500m_latest.pt iter 87500 | train 2.688 | val 2.732 | lr 2.72e-04 | 102,479 tok/s | 54.64 GB | 11.469B tok checkpoint @ iter 87500 -> checkpoints/500m_latest.pt iter 87520 | train 2.717 | val 2.735 | lr 2.72e-04 | 102,457 tok/s | 54.64 GB | 11.472B tok iter 87540 | train 2.802 | val 2.783 | lr 2.72e-04 | 102,458 tok/s | 54.64 GB | 11.474B tok iter 87560 | train 2.762 | val 2.778 | lr 2.71e-04 | 102,460 tok/s | 54.64 GB | 11.477B tok iter 87580 | train 2.745 | val 2.757 | lr 2.71e-04 | 102,461 tok/s | 54.64 GB | 11.479B tok iter 87600 | train 2.861 | val 2.773 | lr 2.71e-04 | 102,463 tok/s | 54.64 GB | 11.482B tok iter 87620 | train 2.700 | val 2.749 | lr 2.71e-04 | 102,464 tok/s | 54.64 GB | 11.485B tok iter 87640 | train 2.675 | val 2.794 | lr 2.71e-04 | 102,465 tok/s | 54.64 GB | 11.487B tok iter 87660 | train 2.698 | val 2.718 | lr 2.71e-04 | 102,467 tok/s | 54.64 GB | 11.490B tok iter 87680 | train 2.809 | val 2.723 | lr 2.71e-04 | 102,469 tok/s | 54.64 GB | 11.493B tok iter 87700 | train 2.632 | val 2.729 | lr 2.71e-04 | 102,469 tok/s | 54.64 GB | 11.495B tok iter 87720 | train 2.830 | val 2.740 | lr 2.71e-04 | 102,471 tok/s | 54.64 GB | 11.498B tok iter 87740 | train 2.751 | val 2.739 | lr 2.71e-04 | 102,473 tok/s | 54.64 GB | 11.500B tok iter 87760 | train 2.674 | val 2.718 | lr 2.70e-04 | 102,475 tok/s | 54.64 GB | 11.503B tok iter 87780 | train 2.742 | val 2.776 | lr 2.70e-04 | 102,475 tok/s | 54.64 GB | 11.506B tok iter 87800 | train 2.764 | val 2.724 | lr 2.70e-04 | 102,477 tok/s | 54.64 GB | 11.508B tok iter 87820 | train 2.634 | val 2.795 | lr 2.70e-04 | 102,479 tok/s | 54.64 GB | 11.511B tok iter 87840 | train 2.921 | val 2.727 | lr 2.70e-04 | 102,481 tok/s | 54.64 GB | 11.513B tok iter 87860 | train 2.731 | val 2.771 | lr 2.70e-04 | 102,481 tok/s | 54.64 GB | 11.516B tok iter 87880 | train 2.738 | val 2.786 | lr 2.70e-04 | 102,483 tok/s | 54.64 GB | 11.519B tok iter 87900 | train 2.681 | val 2.796 | lr 2.70e-04 | 102,485 tok/s | 54.64 GB | 11.521B tok iter 87920 | train 2.710 | val 2.752 | lr 2.70e-04 | 102,486 tok/s | 54.64 GB | 11.524B tok iter 87940 | train 2.629 | val 2.783 | lr 2.69e-04 | 102,487 tok/s | 54.64 GB | 11.527B tok iter 87960 | train 2.693 | val 2.769 | lr 2.69e-04 | 102,488 tok/s | 54.64 GB | 11.529B tok iter 87980 | train 2.887 | val 2.764 | lr 2.69e-04 | 102,490 tok/s | 54.64 GB | 11.532B tok checkpoint @ iter 87991 -> checkpoints/500m_latest.pt iter 88000 | train 2.768 | val 2.815 | lr 2.69e-04 | 102,480 tok/s | 54.64 GB | 11.534B tok checkpoint @ iter 88000 -> checkpoints/500m_latest.pt iter 88020 | train 2.678 | val 2.811 | lr 2.69e-04 | 102,457 tok/s | 54.64 GB | 11.537B tok iter 88040 | train 2.820 | val 2.752 | lr 2.69e-04 | 102,459 tok/s | 54.64 GB | 11.540B tok iter 88060 | train 2.798 | val 2.795 | lr 2.69e-04 | 102,461 tok/s | 54.64 GB | 11.542B tok iter 88080 | train 2.715 | val 2.727 | lr 2.69e-04 | 102,463 tok/s | 54.64 GB | 11.545B tok iter 88100 | train 2.814 | val 2.744 | lr 2.69e-04 | 102,463 tok/s | 54.64 GB | 11.548B tok iter 88120 | train 2.603 | val 2.724 | lr 2.68e-04 | 102,465 tok/s | 54.64 GB | 11.550B tok iter 88140 | train 2.725 | val 2.770 | lr 2.68e-04 | 102,467 tok/s | 54.64 GB | 11.553B tok iter 88160 | train 2.786 | val 2.810 | lr 2.68e-04 | 102,468 tok/s | 54.64 GB | 11.555B tok iter 88180 | train 2.700 | val 2.750 | lr 2.68e-04 | 102,469 tok/s | 54.64 GB | 11.558B tok iter 88200 | train 2.847 | val 2.785 | lr 2.68e-04 | 102,470 tok/s | 54.64 GB | 11.561B tok iter 88220 | train 2.759 | val 2.747 | lr 2.68e-04 | 102,472 tok/s | 54.64 GB | 11.563B tok iter 88240 | train 2.613 | val 2.791 | lr 2.68e-04 | 102,474 tok/s | 54.64 GB | 11.566B tok iter 88260 | train 2.736 | val 2.775 | lr 2.68e-04 | 102,474 tok/s | 54.64 GB | 11.569B tok iter 88280 | train 2.720 | val 2.775 | lr 2.68e-04 | 102,476 tok/s | 54.64 GB | 11.571B tok iter 88300 | train 2.732 | val 2.715 | lr 2.67e-04 | 102,478 tok/s | 54.64 GB | 11.574B tok iter 88320 | train 2.786 | val 2.781 | lr 2.67e-04 | 102,480 tok/s | 54.64 GB | 11.576B tok iter 88340 | train 2.757 | val 2.757 | lr 2.67e-04 | 102,480 tok/s | 54.64 GB | 11.579B tok iter 88360 | train 2.777 | val 2.797 | lr 2.67e-04 | 102,482 tok/s | 54.64 GB | 11.582B tok iter 88380 | train 2.713 | val 2.766 | lr 2.67e-04 | 102,484 tok/s | 54.64 GB | 11.584B tok iter 88400 | train 2.701 | val 2.750 | lr 2.67e-04 | 102,486 tok/s | 54.64 GB | 11.587B tok iter 88420 | train 2.780 | val 2.746 | lr 2.67e-04 | 102,486 tok/s | 54.64 GB | 11.590B tok iter 88440 | train 2.746 | val 2.773 | lr 2.67e-04 | 102,488 tok/s | 54.64 GB | 11.592B tok iter 88460 | train 2.716 | val 2.739 | lr 2.67e-04 | 102,489 tok/s | 54.64 GB | 11.595B tok iter 88480 | train 2.775 | val 2.730 | lr 2.66e-04 | 102,491 tok/s | 54.64 GB | 11.597B tok checkpoint @ iter 88490 -> checkpoints/500m_latest.pt iter 88500 | train 2.634 | val 2.765 | lr 2.66e-04 | 102,479 tok/s | 54.64 GB | 11.600B tok checkpoint @ iter 88500 -> checkpoints/500m_latest.pt iter 88520 | train 2.856 | val 2.814 | lr 2.66e-04 | 102,458 tok/s | 54.64 GB | 11.603B tok iter 88540 | train 2.846 | val 2.807 | lr 2.66e-04 | 102,460 tok/s | 54.64 GB | 11.605B tok iter 88560 | train 2.609 | val 2.731 | lr 2.66e-04 | 102,462 tok/s | 54.64 GB | 11.608B tok iter 88580 | train 2.760 | val 2.770 | lr 2.66e-04 | 102,462 tok/s | 54.64 GB | 11.610B tok iter 88600 | train 2.702 | val 2.789 | lr 2.66e-04 | 102,464 tok/s | 54.64 GB | 11.613B tok iter 88620 | train 2.745 | val 2.762 | lr 2.66e-04 | 102,466 tok/s | 54.64 GB | 11.616B tok iter 88640 | train 2.765 | val 2.761 | lr 2.66e-04 | 102,467 tok/s | 54.64 GB | 11.618B tok iter 88660 | train 2.673 | val 2.740 | lr 2.65e-04 | 102,468 tok/s | 54.64 GB | 11.621B tok iter 88680 | train 2.794 | val 2.784 | lr 2.65e-04 | 102,469 tok/s | 54.64 GB | 11.624B tok iter 88700 | train 2.685 | val 2.758 | lr 2.65e-04 | 102,471 tok/s | 54.64 GB | 11.626B tok iter 88720 | train 2.791 | val 2.764 | lr 2.65e-04 | 102,473 tok/s | 54.64 GB | 11.629B tok iter 88740 | train 2.794 | val 2.787 | lr 2.65e-04 | 102,473 tok/s | 54.64 GB | 11.631B tok iter 88760 | train 2.725 | val 2.794 | lr 2.65e-04 | 102,475 tok/s | 54.64 GB | 11.634B tok iter 88780 | train 2.871 | val 2.735 | lr 2.65e-04 | 102,477 tok/s | 54.64 GB | 11.637B tok iter 88800 | train 2.596 | val 2.747 | lr 2.65e-04 | 102,479 tok/s | 54.64 GB | 11.639B tok iter 88820 | train 2.743 | val 2.792 | lr 2.65e-04 | 102,479 tok/s | 54.64 GB | 11.642B tok iter 88840 | train 2.767 | val 2.728 | lr 2.64e-04 | 102,481 tok/s | 54.64 GB | 11.645B tok iter 88860 | train 2.850 | val 2.762 | lr 2.64e-04 | 102,483 tok/s | 54.64 GB | 11.647B tok iter 88880 | train 2.802 | val 2.749 | lr 2.64e-04 | 102,484 tok/s | 54.64 GB | 11.650B tok iter 88900 | train 2.808 | val 2.777 | lr 2.64e-04 | 102,485 tok/s | 54.64 GB | 11.652B tok iter 88920 | train 2.759 | val 2.752 | lr 2.64e-04 | 102,486 tok/s | 54.64 GB | 11.655B tok iter 88940 | train 2.783 | val 2.772 | lr 2.64e-04 | 102,488 tok/s | 54.64 GB | 11.658B tok iter 88960 | train 2.615 | val 2.760 | lr 2.64e-04 | 102,490 tok/s | 54.64 GB | 11.660B tok iter 88980 | train 2.682 | val 2.708 | lr 2.64e-04 | 102,490 tok/s | 54.64 GB | 11.663B tok checkpoint @ iter 88990 -> checkpoints/500m_latest.pt iter 89000 | train 2.878 | val 2.755 | lr 2.64e-04 | 102,479 tok/s | 54.64 GB | 11.666B tok checkpoint @ iter 89000 -> checkpoints/500m_latest.pt iter 89020 | train 2.767 | val 2.774 | lr 2.64e-04 | 102,453 tok/s | 54.64 GB | 11.668B tok iter 89040 | train 2.635 | val 2.730 | lr 2.63e-04 | 102,455 tok/s | 54.64 GB | 11.671B tok iter 89060 | train 2.906 | val 2.758 | lr 2.63e-04 | 102,455 tok/s | 54.64 GB | 11.673B tok iter 89080 | train 2.929 | val 2.762 | lr 2.63e-04 | 102,457 tok/s | 54.64 GB | 11.676B tok iter 89100 | train 2.846 | val 2.757 | lr 2.63e-04 | 102,458 tok/s | 54.64 GB | 11.679B tok iter 89120 | train 2.709 | val 2.767 | lr 2.63e-04 | 102,460 tok/s | 54.64 GB | 11.681B tok iter 89140 | train 2.844 | val 2.747 | lr 2.63e-04 | 102,461 tok/s | 54.64 GB | 11.684B tok iter 89160 | train 2.748 | val 2.782 | lr 2.63e-04 | 102,462 tok/s | 54.64 GB | 11.687B tok iter 89180 | train 2.482 | val 2.744 | lr 2.63e-04 | 102,464 tok/s | 54.64 GB | 11.689B tok iter 89200 | train 2.742 | val 2.770 | lr 2.63e-04 | 102,466 tok/s | 54.64 GB | 11.692B tok iter 89220 | train 2.797 | val 2.761 | lr 2.62e-04 | 102,466 tok/s | 54.64 GB | 11.694B tok iter 89240 | train 2.820 | val 2.752 | lr 2.62e-04 | 102,468 tok/s | 54.64 GB | 11.697B tok iter 89260 | train 2.695 | val 2.772 | lr 2.62e-04 | 102,470 tok/s | 54.64 GB | 11.700B tok iter 89280 | train 2.781 | val 2.747 | lr 2.62e-04 | 102,472 tok/s | 54.64 GB | 11.702B tok iter 89300 | train 2.843 | val 2.730 | lr 2.62e-04 | 102,472 tok/s | 54.64 GB | 11.705B tok iter 89320 | train 2.695 | val 2.762 | lr 2.62e-04 | 102,474 tok/s | 54.64 GB | 11.707B tok iter 89340 | train 2.826 | val 2.754 | lr 2.62e-04 | 102,475 tok/s | 54.64 GB | 11.710B tok iter 89360 | train 2.798 | val 2.755 | lr 2.62e-04 | 102,477 tok/s | 54.64 GB | 11.713B tok iter 89380 | train 2.775 | val 2.758 | lr 2.62e-04 | 102,478 tok/s | 54.64 GB | 11.715B tok iter 89400 | train 2.796 | val 2.750 | lr 2.61e-04 | 102,479 tok/s | 54.64 GB | 11.718B tok iter 89420 | train 2.776 | val 2.770 | lr 2.61e-04 | 102,481 tok/s | 54.64 GB | 11.721B tok iter 89440 | train 2.696 | val 2.787 | lr 2.61e-04 | 102,483 tok/s | 54.64 GB | 11.723B tok iter 89460 | train 2.600 | val 2.827 | lr 2.61e-04 | 102,483 tok/s | 54.64 GB | 11.726B tok iter 89480 | train 2.657 | val 2.799 | lr 2.61e-04 | 102,485 tok/s | 54.64 GB | 11.728B tok checkpoint @ iter 89487 -> checkpoints/500m_latest.pt iter 89500 | train 2.837 | val 2.741 | lr 2.61e-04 | 102,476 tok/s | 54.64 GB | 11.731B tok checkpoint @ iter 89500 -> checkpoints/500m_latest.pt iter 89520 | train 2.663 | val 2.769 | lr 2.61e-04 | 102,453 tok/s | 54.64 GB | 11.734B tok iter 89540 | train 2.799 | val 2.760 | lr 2.61e-04 | 102,453 tok/s | 54.64 GB | 11.736B tok iter 89560 | train 2.688 | val 2.760 | lr 2.61e-04 | 102,455 tok/s | 54.64 GB | 11.739B tok iter 89580 | train 2.656 | val 2.769 | lr 2.60e-04 | 102,457 tok/s | 54.64 GB | 11.742B tok iter 89600 | train 2.721 | val 2.758 | lr 2.60e-04 | 102,459 tok/s | 54.64 GB | 11.744B tok iter 89620 | train 2.645 | val 2.764 | lr 2.60e-04 | 102,459 tok/s | 54.64 GB | 11.747B tok iter 89640 | train 2.768 | val 2.785 | lr 2.60e-04 | 102,461 tok/s | 54.64 GB | 11.749B tok iter 89660 | train 2.808 | val 2.764 | lr 2.60e-04 | 102,463 tok/s | 54.64 GB | 11.752B tok iter 89680 | train 2.768 | val 2.769 | lr 2.60e-04 | 102,464 tok/s | 54.64 GB | 11.755B tok iter 89700 | train 2.776 | val 2.780 | lr 2.60e-04 | 102,465 tok/s | 54.64 GB | 11.757B tok iter 89720 | train 2.782 | val 2.755 | lr 2.60e-04 | 102,466 tok/s | 54.64 GB | 11.760B tok iter 89740 | train 2.634 | val 2.763 | lr 2.60e-04 | 102,468 tok/s | 54.64 GB | 11.763B tok iter 89760 | train 2.830 | val 2.789 | lr 2.59e-04 | 102,470 tok/s | 54.64 GB | 11.765B tok iter 89780 | train 2.767 | val 2.758 | lr 2.59e-04 | 102,470 tok/s | 54.64 GB | 11.768B tok iter 89800 | train 2.766 | val 2.746 | lr 2.59e-04 | 102,472 tok/s | 54.64 GB | 11.770B tok iter 89820 | train 2.687 | val 2.731 | lr 2.59e-04 | 102,474 tok/s | 54.64 GB | 11.773B tok iter 89840 | train 2.763 | val 2.783 | lr 2.59e-04 | 102,476 tok/s | 54.64 GB | 11.776B tok iter 89860 | train 2.836 | val 2.774 | lr 2.59e-04 | 102,476 tok/s | 54.64 GB | 11.778B tok iter 89880 | train 2.833 | val 2.801 | lr 2.59e-04 | 102,478 tok/s | 54.64 GB | 11.781B tok iter 89900 | train 2.702 | val 2.727 | lr 2.59e-04 | 102,479 tok/s | 54.64 GB | 11.784B tok iter 89920 | train 2.817 | val 2.785 | lr 2.59e-04 | 102,481 tok/s | 54.64 GB | 11.786B tok iter 89940 | train 2.672 | val 2.728 | lr 2.59e-04 | 102,481 tok/s | 54.64 GB | 11.789B tok iter 89960 | train 2.874 | val 2.793 | lr 2.58e-04 | 102,483 tok/s | 54.64 GB | 11.791B tok iter 89980 | train 2.703 | val 2.753 | lr 2.58e-04 | 102,485 tok/s | 54.64 GB | 11.794B tok checkpoint @ iter 89990 -> checkpoints/500m_latest.pt iter 90000 | train 2.660 | val 2.768 | lr 2.58e-04 | 102,474 tok/s | 54.64 GB | 11.797B tok checkpoint @ iter 90000 -> checkpoints/500m_latest.pt iter 90020 | train 2.664 | val 2.753 | lr 2.58e-04 | 102,450 tok/s | 54.64 GB | 11.799B tok iter 90040 | train 2.832 | val 2.752 | lr 2.58e-04 | 102,452 tok/s | 54.64 GB | 11.802B tok iter 90060 | train 2.749 | val 2.743 | lr 2.58e-04 | 102,454 tok/s | 54.64 GB | 11.804B tok iter 90080 | train 2.763 | val 2.755 | lr 2.58e-04 | 102,456 tok/s | 54.64 GB | 11.807B tok iter 90100 | train 2.796 | val 2.732 | lr 2.58e-04 | 102,456 tok/s | 54.64 GB | 11.810B tok iter 90120 | train 2.628 | val 2.737 | lr 2.58e-04 | 102,458 tok/s | 54.64 GB | 11.812B tok iter 90140 | train 2.693 | val 2.758 | lr 2.57e-04 | 102,460 tok/s | 54.64 GB | 11.815B tok iter 90160 | train 2.815 | val 2.739 | lr 2.57e-04 | 102,462 tok/s | 54.64 GB | 11.818B tok iter 90180 | train 2.757 | val 2.765 | lr 2.57e-04 | 102,462 tok/s | 54.64 GB | 11.820B tok iter 90200 | train 2.733 | val 2.789 | lr 2.57e-04 | 102,464 tok/s | 54.64 GB | 11.823B tok iter 90220 | train 2.582 | val 2.741 | lr 2.57e-04 | 102,465 tok/s | 54.64 GB | 11.825B tok iter 90240 | train 2.851 | val 2.782 | lr 2.57e-04 | 102,467 tok/s | 54.64 GB | 11.828B tok iter 90260 | train 2.723 | val 2.731 | lr 2.57e-04 | 102,468 tok/s | 54.64 GB | 11.831B tok iter 90280 | train 2.921 | val 2.777 | lr 2.57e-04 | 102,469 tok/s | 54.64 GB | 11.833B tok iter 90300 | train 2.813 | val 2.760 | lr 2.57e-04 | 102,471 tok/s | 54.64 GB | 11.836B tok iter 90320 | train 2.712 | val 2.756 | lr 2.56e-04 | 102,473 tok/s | 54.64 GB | 11.839B tok iter 90340 | train 2.715 | val 2.722 | lr 2.56e-04 | 102,473 tok/s | 54.64 GB | 11.841B tok iter 90360 | train 2.763 | val 2.776 | lr 2.56e-04 | 102,475 tok/s | 54.64 GB | 11.844B tok iter 90380 | train 2.803 | val 2.772 | lr 2.56e-04 | 102,477 tok/s | 54.64 GB | 11.846B tok iter 90400 | train 2.822 | val 2.753 | lr 2.56e-04 | 102,478 tok/s | 54.64 GB | 11.849B tok iter 90420 | train 2.719 | val 2.742 | lr 2.56e-04 | 102,479 tok/s | 54.64 GB | 11.852B tok iter 90440 | train 2.812 | val 2.752 | lr 2.56e-04 | 102,481 tok/s | 54.64 GB | 11.854B tok iter 90460 | train 2.820 | val 2.744 | lr 2.56e-04 | 102,482 tok/s | 54.64 GB | 11.857B tok iter 90480 | train 2.753 | val 2.732 | lr 2.56e-04 | 102,484 tok/s | 54.64 GB | 11.860B tok checkpoint @ iter 90489 -> checkpoints/500m_latest.pt iter 90500 | train 2.688 | val 2.720 | lr 2.55e-04 | 102,472 tok/s | 54.64 GB | 11.862B tok checkpoint @ iter 90500 -> checkpoints/500m_latest.pt iter 90520 | train 2.699 | val 2.754 | lr 2.55e-04 | 102,450 tok/s | 54.64 GB | 11.865B tok iter 90540 | train 2.825 | val 2.713 | lr 2.55e-04 | 102,451 tok/s | 54.64 GB | 11.867B tok iter 90560 | train 2.776 | val 2.795 | lr 2.55e-04 | 102,453 tok/s | 54.64 GB | 11.870B tok iter 90580 | train 2.774 | val 2.753 | lr 2.55e-04 | 102,454 tok/s | 54.64 GB | 11.873B tok iter 90600 | train 2.629 | val 2.758 | lr 2.55e-04 | 102,455 tok/s | 54.64 GB | 11.875B tok iter 90620 | train 2.646 | val 2.785 | lr 2.55e-04 | 102,457 tok/s | 54.64 GB | 11.878B tok iter 90640 | train 2.738 | val 2.781 | lr 2.55e-04 | 102,459 tok/s | 54.64 GB | 11.880B tok iter 90660 | train 2.742 | val 2.722 | lr 2.55e-04 | 102,459 tok/s | 54.64 GB | 11.883B tok iter 90680 | train 2.707 | val 2.769 | lr 2.55e-04 | 102,461 tok/s | 54.64 GB | 11.886B tok iter 90700 | train 2.756 | val 2.737 | lr 2.54e-04 | 102,463 tok/s | 54.64 GB | 11.888B tok iter 90720 | train 2.835 | val 2.772 | lr 2.54e-04 | 102,465 tok/s | 54.64 GB | 11.891B tok iter 90740 | train 2.738 | val 2.726 | lr 2.54e-04 | 102,465 tok/s | 54.64 GB | 11.894B tok iter 90760 | train 2.849 | val 2.762 | lr 2.54e-04 | 102,467 tok/s | 54.64 GB | 11.896B tok iter 90780 | train 2.684 | val 2.718 | lr 2.54e-04 | 102,468 tok/s | 54.64 GB | 11.899B tok iter 90800 | train 2.698 | val 2.821 | lr 2.54e-04 | 102,470 tok/s | 54.64 GB | 11.901B tok iter 90820 | train 2.659 | val 2.740 | lr 2.54e-04 | 102,470 tok/s | 54.64 GB | 11.904B tok iter 90840 | train 2.795 | val 2.791 | lr 2.54e-04 | 102,472 tok/s | 54.64 GB | 11.907B tok iter 90860 | train 2.652 | val 2.753 | lr 2.54e-04 | 102,474 tok/s | 54.64 GB | 11.909B tok iter 90880 | train 2.776 | val 2.729 | lr 2.53e-04 | 102,476 tok/s | 54.64 GB | 11.912B tok iter 90900 | train 2.702 | val 2.732 | lr 2.53e-04 | 102,476 tok/s | 54.64 GB | 11.915B tok iter 90920 | train 2.682 | val 2.705 | lr 2.53e-04 | 102,478 tok/s | 54.64 GB | 11.917B tok iter 90940 | train 2.779 | val 2.709 | lr 2.53e-04 | 102,480 tok/s | 54.64 GB | 11.920B tok iter 90960 | train 2.845 | val 2.769 | lr 2.53e-04 | 102,481 tok/s | 54.64 GB | 11.922B tok iter 90980 | train 2.745 | val 2.751 | lr 2.53e-04 | 102,482 tok/s | 54.64 GB | 11.925B tok checkpoint @ iter 90990 -> checkpoints/500m_latest.pt iter 91000 | train 2.825 | val 2.793 | lr 2.53e-04 | 102,472 tok/s | 54.64 GB | 11.928B tok checkpoint @ iter 91000 -> checkpoints/500m_latest.pt iter 91020 | train 2.670 | val 2.770 | lr 2.53e-04 | 102,450 tok/s | 54.64 GB | 11.930B tok iter 91040 | train 2.792 | val 2.801 | lr 2.53e-04 | 102,452 tok/s | 54.64 GB | 11.933B tok iter 91060 | train 2.759 | val 2.739 | lr 2.52e-04 | 102,452 tok/s | 54.64 GB | 11.936B tok iter 91080 | train 2.791 | val 2.764 | lr 2.52e-04 | 102,454 tok/s | 54.64 GB | 11.938B tok iter 91100 | train 2.848 | val 2.766 | lr 2.52e-04 | 102,456 tok/s | 54.64 GB | 11.941B tok iter 91120 | train 2.773 | val 2.743 | lr 2.52e-04 | 102,458 tok/s | 54.64 GB | 11.943B tok iter 91140 | train 2.738 | val 2.763 | lr 2.52e-04 | 102,458 tok/s | 54.64 GB | 11.946B tok iter 91160 | train 2.809 | val 2.766 | lr 2.52e-04 | 102,460 tok/s | 54.64 GB | 11.949B tok iter 91180 | train 2.662 | val 2.748 | lr 2.52e-04 | 102,462 tok/s | 54.64 GB | 11.951B tok iter 91200 | train 2.707 | val 2.729 | lr 2.52e-04 | 102,463 tok/s | 54.64 GB | 11.954B tok iter 91220 | train 2.728 | val 2.751 | lr 2.52e-04 | 102,464 tok/s | 54.64 GB | 11.957B tok iter 91240 | train 2.673 | val 2.756 | lr 2.52e-04 | 102,465 tok/s | 54.64 GB | 11.959B tok iter 91260 | train 2.862 | val 2.760 | lr 2.51e-04 | 102,467 tok/s | 54.64 GB | 11.962B tok iter 91280 | train 2.676 | val 2.732 | lr 2.51e-04 | 102,469 tok/s | 54.64 GB | 11.964B tok iter 91300 | train 2.824 | val 2.768 | lr 2.51e-04 | 102,469 tok/s | 54.64 GB | 11.967B tok iter 91320 | train 2.845 | val 2.778 | lr 2.51e-04 | 102,471 tok/s | 54.64 GB | 11.970B tok iter 91340 | train 2.740 | val 2.801 | lr 2.51e-04 | 102,472 tok/s | 54.64 GB | 11.972B tok iter 91360 | train 2.602 | val 2.790 | lr 2.51e-04 | 102,474 tok/s | 54.64 GB | 11.975B tok iter 91380 | train 2.673 | val 2.765 | lr 2.51e-04 | 102,475 tok/s | 54.64 GB | 11.977B tok iter 91400 | train 2.636 | val 2.783 | lr 2.51e-04 | 102,476 tok/s | 54.64 GB | 11.980B tok iter 91420 | train 2.811 | val 2.740 | lr 2.51e-04 | 102,478 tok/s | 54.64 GB | 11.983B tok iter 91440 | train 2.629 | val 2.815 | lr 2.50e-04 | 102,480 tok/s | 54.64 GB | 11.985B tok iter 91460 | train 2.617 | val 2.793 | lr 2.50e-04 | 102,480 tok/s | 54.64 GB | 11.988B tok iter 91480 | train 2.830 | val 2.697 | lr 2.50e-04 | 102,482 tok/s | 54.64 GB | 11.991B tok checkpoint @ iter 91490 -> checkpoints/500m_latest.pt iter 91500 | train 2.772 | val 2.725 | lr 2.50e-04 | 102,472 tok/s | 54.64 GB | 11.993B tok checkpoint @ iter 91500 -> checkpoints/500m_latest.pt iter 91520 | train 2.849 | val 2.777 | lr 2.50e-04 | 102,451 tok/s | 54.64 GB | 11.996B tok iter 91540 | train 2.684 | val 2.753 | lr 2.50e-04 | 102,452 tok/s | 54.64 GB | 11.998B tok iter 91560 | train 2.704 | val 2.730 | lr 2.50e-04 | 102,454 tok/s | 54.64 GB | 12.001B tok iter 91580 | train 2.666 | val 2.727 | lr 2.50e-04 | 102,455 tok/s | 54.64 GB | 12.004B tok iter 91600 | train 2.726 | val 2.728 | lr 2.50e-04 | 102,457 tok/s | 54.64 GB | 12.006B tok iter 91620 | train 2.683 | val 2.753 | lr 2.49e-04 | 102,457 tok/s | 54.64 GB | 12.009B tok iter 91640 | train 2.756 | val 2.757 | lr 2.49e-04 | 102,459 tok/s | 54.64 GB | 12.012B tok iter 91660 | train 2.670 | val 2.752 | lr 2.49e-04 | 102,461 tok/s | 54.64 GB | 12.014B tok iter 91680 | train 2.800 | val 2.749 | lr 2.49e-04 | 102,463 tok/s | 54.64 GB | 12.017B tok iter 91700 | train 2.760 | val 2.722 | lr 2.49e-04 | 102,463 tok/s | 54.64 GB | 12.019B tok iter 91720 | train 2.730 | val 2.816 | lr 2.49e-04 | 102,465 tok/s | 54.64 GB | 12.022B tok iter 91740 | train 2.786 | val 2.747 | lr 2.49e-04 | 102,466 tok/s | 54.64 GB | 12.025B tok iter 91760 | train 2.805 | val 2.729 | lr 2.49e-04 | 102,468 tok/s | 54.64 GB | 12.027B tok iter 91780 | train 2.739 | val 2.744 | lr 2.49e-04 | 102,468 tok/s | 54.64 GB | 12.030B tok iter 91800 | train 2.834 | val 2.743 | lr 2.49e-04 | 102,470 tok/s | 54.64 GB | 12.033B tok iter 91820 | train 2.855 | val 2.766 | lr 2.48e-04 | 102,472 tok/s | 54.64 GB | 12.035B tok iter 91840 | train 2.727 | val 2.779 | lr 2.48e-04 | 102,474 tok/s | 54.64 GB | 12.038B tok iter 91860 | train 2.742 | val 2.756 | lr 2.48e-04 | 102,474 tok/s | 54.64 GB | 12.040B tok iter 91880 | train 2.696 | val 2.728 | lr 2.48e-04 | 102,475 tok/s | 54.64 GB | 12.043B tok iter 91900 | train 2.786 | val 2.768 | lr 2.48e-04 | 102,477 tok/s | 54.64 GB | 12.046B tok iter 91920 | train 2.703 | val 2.737 | lr 2.48e-04 | 102,479 tok/s | 54.64 GB | 12.048B tok iter 91940 | train 2.771 | val 2.707 | lr 2.48e-04 | 102,479 tok/s | 54.64 GB | 12.051B tok iter 91960 | train 2.849 | val 2.759 | lr 2.48e-04 | 102,481 tok/s | 54.64 GB | 12.054B tok iter 91980 | train 2.793 | val 2.746 | lr 2.48e-04 | 102,483 tok/s | 54.64 GB | 12.056B tok checkpoint @ iter 91990 -> checkpoints/500m_latest.pt iter 92000 | train 2.738 | val 2.778 | lr 2.47e-04 | 102,473 tok/s | 54.64 GB | 12.059B tok checkpoint @ iter 92000 -> checkpoints/500m_latest.pt iter 92020 | train 2.708 | val 2.749 | lr 2.47e-04 | 102,450 tok/s | 54.64 GB | 12.061B tok iter 92040 | train 2.734 | val 2.735 | lr 2.47e-04 | 102,452 tok/s | 54.64 GB | 12.064B tok iter 92060 | train 2.813 | val 2.693 | lr 2.47e-04 | 102,454 tok/s | 54.64 GB | 12.067B tok iter 92080 | train 2.793 | val 2.783 | lr 2.47e-04 | 102,456 tok/s | 54.64 GB | 12.069B tok iter 92100 | train 2.573 | val 2.749 | lr 2.47e-04 | 102,456 tok/s | 54.64 GB | 12.072B tok iter 92120 | train 2.762 | val 2.739 | lr 2.47e-04 | 102,458 tok/s | 54.64 GB | 12.074B tok iter 92140 | train 2.739 | val 2.739 | lr 2.47e-04 | 102,459 tok/s | 54.64 GB | 12.077B tok iter 92160 | train 2.780 | val 2.731 | lr 2.47e-04 | 102,461 tok/s | 54.64 GB | 12.080B tok iter 92180 | train 2.685 | val 2.743 | lr 2.46e-04 | 102,461 tok/s | 54.64 GB | 12.082B tok iter 92200 | train 2.715 | val 2.726 | lr 2.46e-04 | 102,463 tok/s | 54.64 GB | 12.085B tok iter 92220 | train 2.898 | val 2.733 | lr 2.46e-04 | 102,465 tok/s | 54.64 GB | 12.088B tok iter 92240 | train 2.860 | val 2.740 | lr 2.46e-04 | 102,466 tok/s | 54.64 GB | 12.090B tok iter 92260 | train 2.681 | val 2.771 | lr 2.46e-04 | 102,467 tok/s | 54.64 GB | 12.093B tok iter 92280 | train 2.608 | val 2.746 | lr 2.46e-04 | 102,469 tok/s | 54.64 GB | 12.095B tok iter 92300 | train 2.650 | val 2.704 | lr 2.46e-04 | 102,470 tok/s | 54.64 GB | 12.098B tok iter 92320 | train 2.662 | val 2.720 | lr 2.46e-04 | 102,472 tok/s | 54.64 GB | 12.101B tok iter 92340 | train 2.787 | val 2.730 | lr 2.46e-04 | 102,472 tok/s | 54.64 GB | 12.103B tok iter 92360 | train 2.739 | val 2.724 | lr 2.46e-04 | 102,474 tok/s | 54.64 GB | 12.106B tok iter 92380 | train 2.662 | val 2.755 | lr 2.45e-04 | 102,476 tok/s | 54.64 GB | 12.109B tok iter 92400 | train 2.874 | val 2.736 | lr 2.45e-04 | 102,477 tok/s | 54.64 GB | 12.111B tok iter 92420 | train 2.894 | val 2.755 | lr 2.45e-04 | 102,478 tok/s | 54.64 GB | 12.114B tok iter 92440 | train 2.777 | val 2.794 | lr 2.45e-04 | 102,480 tok/s | 54.64 GB | 12.116B tok iter 92460 | train 2.805 | val 2.790 | lr 2.45e-04 | 102,481 tok/s | 54.64 GB | 12.119B tok iter 92480 | train 2.761 | val 2.725 | lr 2.45e-04 | 102,483 tok/s | 54.64 GB | 12.122B tok checkpoint @ iter 92490 -> checkpoints/500m_latest.pt iter 92500 | train 2.572 | val 2.785 | lr 2.45e-04 | 102,471 tok/s | 54.64 GB | 12.124B tok checkpoint @ iter 92500 -> checkpoints/500m_latest.pt iter 92520 | train 2.724 | val 2.792 | lr 2.45e-04 | 102,449 tok/s | 54.64 GB | 12.127B tok iter 92540 | train 2.724 | val 2.721 | lr 2.45e-04 | 102,450 tok/s | 54.64 GB | 12.130B tok iter 92560 | train 2.876 | val 2.726 | lr 2.44e-04 | 102,452 tok/s | 54.64 GB | 12.132B tok iter 92580 | train 2.782 | val 2.752 | lr 2.44e-04 | 102,452 tok/s | 54.64 GB | 12.135B tok iter 92600 | train 2.684 | val 2.783 | lr 2.44e-04 | 102,454 tok/s | 54.64 GB | 12.137B tok iter 92620 | train 2.818 | val 2.691 | lr 2.44e-04 | 102,456 tok/s | 54.64 GB | 12.140B tok iter 92640 | train 2.718 | val 2.704 | lr 2.44e-04 | 102,458 tok/s | 54.64 GB | 12.143B tok iter 92660 | train 2.737 | val 2.793 | lr 2.44e-04 | 102,458 tok/s | 54.64 GB | 12.145B tok iter 92680 | train 2.698 | val 2.791 | lr 2.44e-04 | 102,460 tok/s | 54.64 GB | 12.148B tok iter 92700 | train 2.647 | val 2.731 | lr 2.44e-04 | 102,461 tok/s | 54.64 GB | 12.151B tok iter 92720 | train 2.776 | val 2.789 | lr 2.44e-04 | 102,463 tok/s | 54.64 GB | 12.153B tok iter 92740 | train 2.824 | val 2.730 | lr 2.43e-04 | 102,463 tok/s | 54.64 GB | 12.156B tok iter 92760 | train 2.726 | val 2.702 | lr 2.43e-04 | 102,465 tok/s | 54.64 GB | 12.158B tok iter 92780 | train 2.700 | val 2.752 | lr 2.43e-04 | 102,467 tok/s | 54.64 GB | 12.161B tok iter 92800 | train 2.782 | val 2.776 | lr 2.43e-04 | 102,469 tok/s | 54.64 GB | 12.164B tok iter 92820 | train 2.801 | val 2.728 | lr 2.43e-04 | 102,469 tok/s | 54.64 GB | 12.166B tok iter 92840 | train 2.747 | val 2.745 | lr 2.43e-04 | 102,471 tok/s | 54.64 GB | 12.169B tok iter 92860 | train 2.639 | val 2.758 | lr 2.43e-04 | 102,472 tok/s | 54.64 GB | 12.171B tok iter 92880 | train 2.688 | val 2.782 | lr 2.43e-04 | 102,474 tok/s | 54.64 GB | 12.174B tok iter 92900 | train 2.759 | val 2.749 | lr 2.43e-04 | 102,474 tok/s | 54.64 GB | 12.177B tok iter 92920 | train 2.818 | val 2.718 | lr 2.43e-04 | 102,476 tok/s | 54.64 GB | 12.179B tok iter 92940 | train 2.853 | val 2.780 | lr 2.42e-04 | 102,478 tok/s | 54.64 GB | 12.182B tok iter 92960 | train 2.908 | val 2.740 | lr 2.42e-04 | 102,480 tok/s | 54.64 GB | 12.185B tok iter 92980 | train 2.730 | val 2.732 | lr 2.42e-04 | 102,480 tok/s | 54.64 GB | 12.187B tok checkpoint @ iter 92990 -> checkpoints/500m_latest.pt iter 93000 | train 2.646 | val 2.683 | lr 2.42e-04 | 102,469 tok/s | 54.64 GB | 12.190B tok checkpoint @ iter 93000 -> checkpoints/500m_latest.pt iter 93020 | train 2.716 | val 2.787 | lr 2.42e-04 | 102,447 tok/s | 54.64 GB | 12.192B tok iter 93040 | train 2.744 | val 2.796 | lr 2.42e-04 | 102,449 tok/s | 54.64 GB | 12.195B tok iter 93060 | train 2.815 | val 2.737 | lr 2.42e-04 | 102,449 tok/s | 54.64 GB | 12.198B tok iter 93080 | train 2.785 | val 2.764 | lr 2.42e-04 | 102,451 tok/s | 54.64 GB | 12.200B tok iter 93100 | train 2.744 | val 2.798 | lr 2.42e-04 | 102,453 tok/s | 54.64 GB | 12.203B tok iter 93120 | train 2.686 | val 2.723 | lr 2.41e-04 | 102,454 tok/s | 54.64 GB | 12.206B tok iter 93140 | train 2.734 | val 2.718 | lr 2.41e-04 | 102,455 tok/s | 54.64 GB | 12.208B tok iter 93160 | train 2.644 | val 2.782 | lr 2.41e-04 | 102,456 tok/s | 54.64 GB | 12.211B tok iter 93180 | train 2.682 | val 2.712 | lr 2.41e-04 | 102,458 tok/s | 54.64 GB | 12.213B tok iter 93200 | train 2.896 | val 2.768 | lr 2.41e-04 | 102,460 tok/s | 54.64 GB | 12.216B tok iter 93220 | train 2.679 | val 2.747 | lr 2.41e-04 | 102,460 tok/s | 54.64 GB | 12.219B tok iter 93240 | train 2.812 | val 2.729 | lr 2.41e-04 | 102,462 tok/s | 54.64 GB | 12.221B tok iter 93260 | train 2.680 | val 2.778 | lr 2.41e-04 | 102,464 tok/s | 54.64 GB | 12.224B tok iter 93280 | train 2.842 | val 2.734 | lr 2.41e-04 | 102,465 tok/s | 54.64 GB | 12.227B tok iter 93300 | train 2.822 | val 2.731 | lr 2.41e-04 | 102,466 tok/s | 54.64 GB | 12.229B tok iter 93320 | train 2.739 | val 2.759 | lr 2.40e-04 | 102,467 tok/s | 54.64 GB | 12.232B tok iter 93340 | train 2.658 | val 2.736 | lr 2.40e-04 | 102,469 tok/s | 54.64 GB | 12.234B tok iter 93360 | train 2.647 | val 2.739 | lr 2.40e-04 | 102,471 tok/s | 54.64 GB | 12.237B tok iter 93380 | train 2.721 | val 2.739 | lr 2.40e-04 | 102,471 tok/s | 54.64 GB | 12.240B tok iter 93400 | train 2.787 | val 2.810 | lr 2.40e-04 | 102,473 tok/s | 54.64 GB | 12.242B tok iter 93420 | train 2.745 | val 2.697 | lr 2.40e-04 | 102,474 tok/s | 54.64 GB | 12.245B tok iter 93440 | train 2.843 | val 2.796 | lr 2.40e-04 | 102,476 tok/s | 54.64 GB | 12.247B tok iter 93460 | train 2.802 | val 2.776 | lr 2.40e-04 | 102,476 tok/s | 54.64 GB | 12.250B tok iter 93480 | train 2.711 | val 2.793 | lr 2.40e-04 | 102,478 tok/s | 54.64 GB | 12.253B tok checkpoint @ iter 93490 -> checkpoints/500m_latest.pt iter 93500 | train 2.763 | val 2.717 | lr 2.39e-04 | 102,468 tok/s | 54.64 GB | 12.255B tok checkpoint @ iter 93500 -> checkpoints/500m_latest.pt iter 93520 | train 2.789 | val 2.794 | lr 2.39e-04 | 102,447 tok/s | 54.64 GB | 12.258B tok iter 93540 | train 2.859 | val 2.700 | lr 2.39e-04 | 102,447 tok/s | 54.64 GB | 12.261B tok iter 93560 | train 2.659 | val 2.718 | lr 2.39e-04 | 102,449 tok/s | 54.64 GB | 12.263B tok iter 93580 | train 2.702 | val 2.749 | lr 2.39e-04 | 102,451 tok/s | 54.64 GB | 12.266B tok iter 93600 | train 2.756 | val 2.775 | lr 2.39e-04 | 102,452 tok/s | 54.64 GB | 12.268B tok iter 93620 | train 2.831 | val 2.699 | lr 2.39e-04 | 102,453 tok/s | 54.64 GB | 12.271B tok iter 93640 | train 2.755 | val 2.695 | lr 2.39e-04 | 102,454 tok/s | 54.64 GB | 12.274B tok iter 93660 | train 2.761 | val 2.729 | lr 2.39e-04 | 102,456 tok/s | 54.64 GB | 12.276B tok iter 93680 | train 2.583 | val 2.736 | lr 2.39e-04 | 102,458 tok/s | 54.64 GB | 12.279B tok iter 93700 | train 2.788 | val 2.751 | lr 2.38e-04 | 102,458 tok/s | 54.64 GB | 12.282B tok iter 93720 | train 2.852 | val 2.726 | lr 2.38e-04 | 102,460 tok/s | 54.64 GB | 12.284B tok iter 93740 | train 2.701 | val 2.765 | lr 2.38e-04 | 102,461 tok/s | 54.64 GB | 12.287B tok iter 93760 | train 2.728 | val 2.773 | lr 2.38e-04 | 102,463 tok/s | 54.64 GB | 12.289B tok iter 93780 | train 2.734 | val 2.708 | lr 2.38e-04 | 102,463 tok/s | 54.64 GB | 12.292B tok iter 93800 | train 2.767 | val 2.764 | lr 2.38e-04 | 102,465 tok/s | 54.64 GB | 12.295B tok iter 93820 | train 2.859 | val 2.721 | lr 2.38e-04 | 102,467 tok/s | 54.64 GB | 12.297B tok iter 93840 | train 2.728 | val 2.686 | lr 2.38e-04 | 102,468 tok/s | 54.64 GB | 12.300B tok iter 93860 | train 2.709 | val 2.731 | lr 2.38e-04 | 102,469 tok/s | 54.64 GB | 12.303B tok iter 93880 | train 2.601 | val 2.780 | lr 2.37e-04 | 102,470 tok/s | 54.64 GB | 12.305B tok iter 93900 | train 2.706 | val 2.706 | lr 2.37e-04 | 102,472 tok/s | 54.64 GB | 12.308B tok iter 93920 | train 2.776 | val 2.764 | lr 2.37e-04 | 102,474 tok/s | 54.64 GB | 12.310B tok iter 93940 | train 2.823 | val 2.754 | lr 2.37e-04 | 102,474 tok/s | 54.64 GB | 12.313B tok iter 93960 | train 2.713 | val 2.781 | lr 2.37e-04 | 102,476 tok/s | 54.64 GB | 12.316B tok iter 93980 | train 2.756 | val 2.706 | lr 2.37e-04 | 102,477 tok/s | 54.64 GB | 12.318B tok checkpoint @ iter 93990 -> checkpoints/500m_latest.pt iter 94000 | train 2.822 | val 2.744 | lr 2.37e-04 | 102,467 tok/s | 54.64 GB | 12.321B tok checkpoint @ iter 94000 -> checkpoints/500m_latest.pt iter 94020 | train 2.773 | val 2.725 | lr 2.37e-04 | 102,443 tok/s | 54.64 GB | 12.324B tok iter 94040 | train 2.710 | val 2.749 | lr 2.37e-04 | 102,445 tok/s | 54.64 GB | 12.326B tok iter 94060 | train 2.653 | val 2.758 | lr 2.37e-04 | 102,447 tok/s | 54.64 GB | 12.329B tok iter 94080 | train 2.560 | val 2.750 | lr 2.36e-04 | 102,449 tok/s | 54.64 GB | 12.331B tok iter 94100 | train 2.795 | val 2.768 | lr 2.36e-04 | 102,449 tok/s | 54.64 GB | 12.334B tok iter 94120 | train 2.672 | val 2.734 | lr 2.36e-04 | 102,451 tok/s | 54.64 GB | 12.337B tok iter 94140 | train 2.665 | val 2.768 | lr 2.36e-04 | 102,452 tok/s | 54.64 GB | 12.339B tok iter 94160 | train 2.784 | val 2.768 | lr 2.36e-04 | 102,454 tok/s | 54.64 GB | 12.342B tok iter 94180 | train 2.638 | val 2.777 | lr 2.36e-04 | 102,454 tok/s | 54.64 GB | 12.344B tok iter 94200 | train 2.942 | val 2.708 | lr 2.36e-04 | 102,456 tok/s | 54.64 GB | 12.347B tok iter 94220 | train 2.745 | val 2.740 | lr 2.36e-04 | 102,458 tok/s | 54.64 GB | 12.350B tok iter 94240 | train 2.814 | val 2.740 | lr 2.36e-04 | 102,459 tok/s | 54.64 GB | 12.352B tok iter 94260 | train 2.863 | val 2.756 | lr 2.35e-04 | 102,460 tok/s | 54.64 GB | 12.355B tok iter 94280 | train 2.823 | val 2.801 | lr 2.35e-04 | 102,461 tok/s | 54.64 GB | 12.358B tok iter 94300 | train 2.749 | val 2.712 | lr 2.35e-04 | 102,463 tok/s | 54.64 GB | 12.360B tok iter 94320 | train 2.724 | val 2.765 | lr 2.35e-04 | 102,465 tok/s | 54.64 GB | 12.363B tok iter 94340 | train 2.811 | val 2.674 | lr 2.35e-04 | 102,465 tok/s | 54.64 GB | 12.365B tok iter 94360 | train 2.793 | val 2.794 | lr 2.35e-04 | 102,467 tok/s | 54.64 GB | 12.368B tok iter 94380 | train 2.740 | val 2.729 | lr 2.35e-04 | 102,469 tok/s | 54.64 GB | 12.371B tok iter 94400 | train 2.821 | val 2.716 | lr 2.35e-04 | 102,470 tok/s | 54.64 GB | 12.373B tok iter 94420 | train 2.580 | val 2.768 | lr 2.35e-04 | 102,471 tok/s | 54.64 GB | 12.376B tok iter 94440 | train 2.972 | val 2.704 | lr 2.35e-04 | 102,472 tok/s | 54.64 GB | 12.379B tok iter 94460 | train 2.721 | val 2.752 | lr 2.34e-04 | 102,474 tok/s | 54.64 GB | 12.381B tok iter 94480 | train 2.731 | val 2.754 | lr 2.34e-04 | 102,476 tok/s | 54.64 GB | 12.384B tok checkpoint @ iter 94490 -> checkpoints/500m_latest.pt iter 94500 | train 2.811 | val 2.747 | lr 2.34e-04 | 102,464 tok/s | 54.64 GB | 12.386B tok checkpoint @ iter 94500 -> checkpoints/500m_latest.pt iter 94520 | train 2.643 | val 2.689 | lr 2.34e-04 | 102,444 tok/s | 54.64 GB | 12.389B tok iter 94540 | train 2.749 | val 2.742 | lr 2.34e-04 | 102,445 tok/s | 54.64 GB | 12.392B tok iter 94560 | train 2.900 | val 2.720 | lr 2.34e-04 | 102,447 tok/s | 54.64 GB | 12.394B tok iter 94580 | train 2.903 | val 2.781 | lr 2.34e-04 | 102,447 tok/s | 54.64 GB | 12.397B tok iter 94600 | train 2.710 | val 2.754 | lr 2.34e-04 | 102,449 tok/s | 54.64 GB | 12.400B tok iter 94620 | train 2.822 | val 2.727 | lr 2.34e-04 | 102,451 tok/s | 54.64 GB | 12.402B tok iter 94640 | train 2.640 | val 2.741 | lr 2.33e-04 | 102,453 tok/s | 54.64 GB | 12.405B tok iter 94660 | train 2.806 | val 2.767 | lr 2.33e-04 | 102,453 tok/s | 54.64 GB | 12.407B tok iter 94680 | train 2.798 | val 2.739 | lr 2.33e-04 | 102,455 tok/s | 54.64 GB | 12.410B tok iter 94700 | train 2.820 | val 2.739 | lr 2.33e-04 | 102,456 tok/s | 54.64 GB | 12.413B tok iter 94720 | train 2.801 | val 2.727 | lr 2.33e-04 | 102,458 tok/s | 54.64 GB | 12.415B tok iter 94740 | train 2.676 | val 2.749 | lr 2.33e-04 | 102,458 tok/s | 54.64 GB | 12.418B tok iter 94760 | train 2.753 | val 2.691 | lr 2.33e-04 | 102,460 tok/s | 54.64 GB | 12.421B tok iter 94780 | train 2.696 | val 2.767 | lr 2.33e-04 | 102,461 tok/s | 54.64 GB | 12.423B tok iter 94800 | train 2.828 | val 2.760 | lr 2.33e-04 | 102,463 tok/s | 54.64 GB | 12.426B tok iter 94820 | train 2.696 | val 2.755 | lr 2.33e-04 | 102,463 tok/s | 54.64 GB | 12.428B tok iter 94840 | train 2.713 | val 2.741 | lr 2.32e-04 | 102,465 tok/s | 54.64 GB | 12.431B tok iter 94860 | train 2.766 | val 2.750 | lr 2.32e-04 | 102,467 tok/s | 54.64 GB | 12.434B tok iter 94880 | train 2.620 | val 2.711 | lr 2.32e-04 | 102,468 tok/s | 54.64 GB | 12.436B tok iter 94900 | train 2.606 | val 2.722 | lr 2.32e-04 | 102,469 tok/s | 54.64 GB | 12.439B tok iter 94920 | train 2.772 | val 2.751 | lr 2.32e-04 | 102,470 tok/s | 54.64 GB | 12.441B tok iter 94940 | train 2.652 | val 2.781 | lr 2.32e-04 | 102,472 tok/s | 54.64 GB | 12.444B tok iter 94960 | train 2.679 | val 2.677 | lr 2.32e-04 | 102,474 tok/s | 54.64 GB | 12.447B tok iter 94980 | train 2.813 | val 2.775 | lr 2.32e-04 | 102,474 tok/s | 54.64 GB | 12.449B tok checkpoint @ iter 94990 -> checkpoints/500m_latest.pt iter 95000 | train 2.930 | val 2.724 | lr 2.32e-04 | 102,463 tok/s | 54.64 GB | 12.452B tok checkpoint @ iter 95000 -> checkpoints/500m_latest.pt iter 95020 | train 2.620 | val 2.755 | lr 2.31e-04 | 102,439 tok/s | 54.64 GB | 12.455B tok iter 95040 | train 2.740 | val 2.761 | lr 2.31e-04 | 102,441 tok/s | 54.64 GB | 12.457B tok iter 95060 | train 2.740 | val 2.737 | lr 2.31e-04 | 102,441 tok/s | 54.64 GB | 12.460B tok iter 95080 | train 2.825 | val 2.728 | lr 2.31e-04 | 102,443 tok/s | 54.64 GB | 12.462B tok iter 95100 | train 2.815 | val 2.730 | lr 2.31e-04 | 102,445 tok/s | 54.64 GB | 12.465B tok iter 95120 | train 2.815 | val 2.733 | lr 2.31e-04 | 102,446 tok/s | 54.64 GB | 12.468B tok iter 95140 | train 2.840 | val 2.788 | lr 2.31e-04 | 102,447 tok/s | 54.64 GB | 12.470B tok iter 95160 | train 2.775 | val 2.753 | lr 2.31e-04 | 102,449 tok/s | 54.64 GB | 12.473B tok iter 95180 | train 2.798 | val 2.741 | lr 2.31e-04 | 102,450 tok/s | 54.64 GB | 12.476B tok iter 95200 | train 2.731 | val 2.706 | lr 2.31e-04 | 102,452 tok/s | 54.64 GB | 12.478B tok iter 95220 | train 2.648 | val 2.736 | lr 2.30e-04 | 102,452 tok/s | 54.64 GB | 12.481B tok iter 95240 | train 2.574 | val 2.743 | lr 2.30e-04 | 102,454 tok/s | 54.64 GB | 12.483B tok iter 95260 | train 2.772 | val 2.721 | lr 2.30e-04 | 102,456 tok/s | 54.64 GB | 12.486B tok iter 95280 | train 2.690 | val 2.743 | lr 2.30e-04 | 102,457 tok/s | 54.64 GB | 12.489B tok iter 95300 | train 2.740 | val 2.744 | lr 2.30e-04 | 102,458 tok/s | 54.64 GB | 12.491B tok iter 95320 | train 2.728 | val 2.734 | lr 2.30e-04 | 102,460 tok/s | 54.64 GB | 12.494B tok iter 95340 | train 2.663 | val 2.746 | lr 2.30e-04 | 102,461 tok/s | 54.64 GB | 12.497B tok iter 95360 | train 2.777 | val 2.717 | lr 2.30e-04 | 102,463 tok/s | 54.64 GB | 12.499B tok iter 95380 | train 2.693 | val 2.756 | lr 2.30e-04 | 102,463 tok/s | 54.64 GB | 12.502B tok iter 95400 | train 2.743 | val 2.743 | lr 2.29e-04 | 102,465 tok/s | 54.64 GB | 12.504B tok iter 95420 | train 2.792 | val 2.804 | lr 2.29e-04 | 102,467 tok/s | 54.64 GB | 12.507B tok iter 95440 | train 2.766 | val 2.737 | lr 2.29e-04 | 102,468 tok/s | 54.64 GB | 12.510B tok iter 95460 | train 2.723 | val 2.768 | lr 2.29e-04 | 102,469 tok/s | 54.64 GB | 12.512B tok iter 95480 | train 2.762 | val 2.718 | lr 2.29e-04 | 102,470 tok/s | 54.64 GB | 12.515B tok checkpoint @ iter 95490 -> checkpoints/500m_latest.pt iter 95500 | train 2.925 | val 2.754 | lr 2.29e-04 | 102,460 tok/s | 54.64 GB | 12.518B tok checkpoint @ iter 95500 -> checkpoints/500m_latest.pt iter 95520 | train 2.697 | val 2.715 | lr 2.29e-04 | 102,438 tok/s | 54.64 GB | 12.520B tok iter 95540 | train 2.701 | val 2.723 | lr 2.29e-04 | 102,438 tok/s | 54.64 GB | 12.523B tok iter 95560 | train 2.715 | val 2.718 | lr 2.29e-04 | 102,440 tok/s | 54.64 GB | 12.525B tok iter 95580 | train 2.657 | val 2.707 | lr 2.29e-04 | 102,442 tok/s | 54.64 GB | 12.528B tok iter 95600 | train 2.690 | val 2.756 | lr 2.28e-04 | 102,444 tok/s | 54.64 GB | 12.531B tok iter 95620 | train 2.745 | val 2.755 | lr 2.28e-04 | 102,444 tok/s | 54.64 GB | 12.533B tok iter 95640 | train 2.761 | val 2.701 | lr 2.28e-04 | 102,446 tok/s | 54.64 GB | 12.536B tok iter 95660 | train 2.649 | val 2.722 | lr 2.28e-04 | 102,447 tok/s | 54.64 GB | 12.538B tok iter 95680 | train 2.729 | val 2.754 | lr 2.28e-04 | 102,449 tok/s | 54.64 GB | 12.541B tok iter 95700 | train 2.814 | val 2.759 | lr 2.28e-04 | 102,449 tok/s | 54.64 GB | 12.544B tok iter 95720 | train 2.658 | val 2.686 | lr 2.28e-04 | 102,451 tok/s | 54.64 GB | 12.546B tok iter 95740 | train 2.590 | val 2.772 | lr 2.28e-04 | 102,453 tok/s | 54.64 GB | 12.549B tok iter 95760 | train 2.777 | val 2.751 | lr 2.28e-04 | 102,455 tok/s | 54.64 GB | 12.552B tok iter 95780 | train 2.765 | val 2.714 | lr 2.28e-04 | 102,455 tok/s | 54.64 GB | 12.554B tok iter 95800 | train 2.711 | val 2.739 | lr 2.27e-04 | 102,457 tok/s | 54.64 GB | 12.557B tok iter 95820 | train 2.916 | val 2.763 | lr 2.27e-04 | 102,458 tok/s | 54.64 GB | 12.559B tok iter 95840 | train 2.764 | val 2.767 | lr 2.27e-04 | 102,460 tok/s | 54.64 GB | 12.562B tok iter 95860 | train 2.643 | val 2.751 | lr 2.27e-04 | 102,460 tok/s | 54.64 GB | 12.565B tok iter 95880 | train 2.885 | val 2.676 | lr 2.27e-04 | 102,462 tok/s | 54.64 GB | 12.567B tok iter 95900 | train 2.904 | val 2.701 | lr 2.27e-04 | 102,464 tok/s | 54.64 GB | 12.570B tok iter 95920 | train 2.760 | val 2.699 | lr 2.27e-04 | 102,465 tok/s | 54.64 GB | 12.573B tok iter 95940 | train 2.893 | val 2.731 | lr 2.27e-04 | 102,466 tok/s | 54.64 GB | 12.575B tok iter 95960 | train 2.771 | val 2.763 | lr 2.27e-04 | 102,468 tok/s | 54.64 GB | 12.578B tok iter 95980 | train 2.814 | val 2.779 | lr 2.26e-04 | 102,469 tok/s | 54.64 GB | 12.580B tok checkpoint @ iter 95990 -> checkpoints/500m_latest.pt iter 96000 | train 2.732 | val 2.736 | lr 2.26e-04 | 102,460 tok/s | 54.64 GB | 12.583B tok checkpoint @ iter 96000 -> checkpoints/500m_latest.pt iter 96020 | train 2.806 | val 2.760 | lr 2.26e-04 | 102,438 tok/s | 54.64 GB | 12.586B tok iter 96040 | train 2.747 | val 2.726 | lr 2.26e-04 | 102,440 tok/s | 54.64 GB | 12.588B tok iter 96060 | train 2.756 | val 2.713 | lr 2.26e-04 | 102,441 tok/s | 54.64 GB | 12.591B tok iter 96080 | train 2.703 | val 2.732 | lr 2.26e-04 | 102,443 tok/s | 54.64 GB | 12.594B tok iter 96100 | train 2.793 | val 2.738 | lr 2.26e-04 | 102,444 tok/s | 54.64 GB | 12.596B tok iter 96120 | train 2.732 | val 2.725 | lr 2.26e-04 | 102,445 tok/s | 54.64 GB | 12.599B tok iter 96140 | train 2.657 | val 2.710 | lr 2.26e-04 | 102,447 tok/s | 54.64 GB | 12.601B tok iter 96160 | train 2.796 | val 2.784 | lr 2.26e-04 | 102,449 tok/s | 54.64 GB | 12.604B tok iter 96180 | train 2.770 | val 2.730 | lr 2.25e-04 | 102,449 tok/s | 54.64 GB | 12.607B tok iter 96200 | train 2.781 | val 2.727 | lr 2.25e-04 | 102,451 tok/s | 54.64 GB | 12.609B tok iter 96220 | train 2.690 | val 2.745 | lr 2.25e-04 | 102,452 tok/s | 54.64 GB | 12.612B tok iter 96240 | train 2.773 | val 2.746 | lr 2.25e-04 | 102,454 tok/s | 54.64 GB | 12.615B tok iter 96260 | train 2.690 | val 2.704 | lr 2.25e-04 | 102,454 tok/s | 54.64 GB | 12.617B tok iter 96280 | train 2.812 | val 2.791 | lr 2.25e-04 | 102,456 tok/s | 54.64 GB | 12.620B tok iter 96300 | train 2.666 | val 2.734 | lr 2.25e-04 | 102,458 tok/s | 54.64 GB | 12.622B tok iter 96320 | train 2.686 | val 2.720 | lr 2.25e-04 | 102,459 tok/s | 54.64 GB | 12.625B tok iter 96340 | train 2.568 | val 2.725 | lr 2.25e-04 | 102,460 tok/s | 54.64 GB | 12.628B tok iter 96360 | train 2.815 | val 2.749 | lr 2.24e-04 | 102,461 tok/s | 54.64 GB | 12.630B tok iter 96380 | train 2.605 | val 2.730 | lr 2.24e-04 | 102,463 tok/s | 54.64 GB | 12.633B tok iter 96400 | train 2.804 | val 2.737 | lr 2.24e-04 | 102,465 tok/s | 54.64 GB | 12.635B tok iter 96420 | train 2.623 | val 2.733 | lr 2.24e-04 | 102,465 tok/s | 54.64 GB | 12.638B tok iter 96440 | train 2.686 | val 2.734 | lr 2.24e-04 | 102,467 tok/s | 54.64 GB | 12.641B tok iter 96460 | train 2.983 | val 2.740 | lr 2.24e-04 | 102,468 tok/s | 54.64 GB | 12.643B tok iter 96480 | train 2.749 | val 2.706 | lr 2.24e-04 | 102,470 tok/s | 54.64 GB | 12.646B tok checkpoint @ iter 96491 -> checkpoints/500m_latest.pt iter 96500 | train 2.696 | val 2.755 | lr 2.24e-04 | 102,459 tok/s | 54.64 GB | 12.649B tok checkpoint @ iter 96500 -> checkpoints/500m_latest.pt iter 96520 | train 3.069 | val 2.737 | lr 2.24e-04 | 102,439 tok/s | 54.64 GB | 12.651B tok iter 96540 | train 2.729 | val 2.725 | lr 2.24e-04 | 102,441 tok/s | 54.64 GB | 12.654B tok iter 96560 | train 2.663 | val 2.767 | lr 2.23e-04 | 102,443 tok/s | 54.64 GB | 12.656B tok iter 96580 | train 2.862 | val 2.728 | lr 2.23e-04 | 102,443 tok/s | 54.64 GB | 12.659B tok iter 96600 | train 2.723 | val 2.720 | lr 2.23e-04 | 102,445 tok/s | 54.64 GB | 12.662B tok iter 96620 | train 2.807 | val 2.740 | lr 2.23e-04 | 102,446 tok/s | 54.64 GB | 12.664B tok iter 96640 | train 2.756 | val 2.719 | lr 2.23e-04 | 102,448 tok/s | 54.64 GB | 12.667B tok iter 96660 | train 2.911 | val 2.730 | lr 2.23e-04 | 102,448 tok/s | 54.64 GB | 12.670B tok iter 96680 | train 2.632 | val 2.718 | lr 2.23e-04 | 102,450 tok/s | 54.64 GB | 12.672B tok iter 96700 | train 2.785 | val 2.728 | lr 2.23e-04 | 102,452 tok/s | 54.64 GB | 12.675B tok iter 96720 | train 2.746 | val 2.775 | lr 2.23e-04 | 102,453 tok/s | 54.64 GB | 12.677B tok iter 96740 | train 2.721 | val 2.759 | lr 2.23e-04 | 102,454 tok/s | 54.64 GB | 12.680B tok iter 96760 | train 2.754 | val 2.728 | lr 2.22e-04 | 102,455 tok/s | 54.64 GB | 12.683B tok iter 96780 | train 2.696 | val 2.777 | lr 2.22e-04 | 102,457 tok/s | 54.64 GB | 12.685B tok iter 96800 | train 2.746 | val 2.778 | lr 2.22e-04 | 102,459 tok/s | 54.64 GB | 12.688B tok iter 96820 | train 2.735 | val 2.759 | lr 2.22e-04 | 102,459 tok/s | 54.64 GB | 12.691B tok iter 96840 | train 2.847 | val 2.735 | lr 2.22e-04 | 102,461 tok/s | 54.64 GB | 12.693B tok iter 96860 | train 2.767 | val 2.770 | lr 2.22e-04 | 102,463 tok/s | 54.64 GB | 12.696B tok iter 96880 | train 2.813 | val 2.737 | lr 2.22e-04 | 102,464 tok/s | 54.64 GB | 12.698B tok iter 96900 | train 2.641 | val 2.775 | lr 2.22e-04 | 102,465 tok/s | 54.64 GB | 12.701B tok iter 96920 | train 2.790 | val 2.783 | lr 2.22e-04 | 102,466 tok/s | 54.64 GB | 12.704B tok iter 96940 | train 2.657 | val 2.745 | lr 2.22e-04 | 102,468 tok/s | 54.64 GB | 12.706B tok iter 96960 | train 2.684 | val 2.749 | lr 2.21e-04 | 102,470 tok/s | 54.64 GB | 12.709B tok iter 96980 | train 2.710 | val 2.770 | lr 2.21e-04 | 102,470 tok/s | 54.64 GB | 12.711B tok checkpoint @ iter 96991 -> checkpoints/500m_latest.pt iter 97000 | train 2.654 | val 2.784 | lr 2.21e-04 | 102,460 tok/s | 54.64 GB | 12.714B tok checkpoint @ iter 97000 -> checkpoints/500m_latest.pt iter 97020 | train 2.705 | val 2.663 | lr 2.21e-04 | 102,438 tok/s | 54.64 GB | 12.717B tok iter 97040 | train 2.752 | val 2.778 | lr 2.21e-04 | 102,440 tok/s | 54.64 GB | 12.719B tok iter 97060 | train 2.798 | val 2.759 | lr 2.21e-04 | 102,440 tok/s | 54.64 GB | 12.722B tok iter 97080 | train 2.660 | val 2.710 | lr 2.21e-04 | 102,442 tok/s | 54.64 GB | 12.725B tok iter 97100 | train 2.732 | val 2.743 | lr 2.21e-04 | 102,444 tok/s | 54.64 GB | 12.727B tok iter 97120 | train 2.695 | val 2.717 | lr 2.21e-04 | 102,445 tok/s | 54.64 GB | 12.730B tok iter 97140 | train 2.663 | val 2.711 | lr 2.20e-04 | 102,446 tok/s | 54.64 GB | 12.732B tok iter 97160 | train 2.690 | val 2.708 | lr 2.20e-04 | 102,448 tok/s | 54.64 GB | 12.735B tok iter 97180 | train 2.883 | val 2.729 | lr 2.20e-04 | 102,449 tok/s | 54.64 GB | 12.738B tok iter 97200 | train 2.720 | val 2.751 | lr 2.20e-04 | 102,451 tok/s | 54.64 GB | 12.740B tok iter 97220 | train 2.798 | val 2.740 | lr 2.20e-04 | 102,451 tok/s | 54.64 GB | 12.743B tok iter 97240 | train 2.679 | val 2.719 | lr 2.20e-04 | 102,453 tok/s | 54.64 GB | 12.746B tok iter 97260 | train 2.729 | val 2.732 | lr 2.20e-04 | 102,454 tok/s | 54.64 GB | 12.748B tok iter 97280 | train 2.719 | val 2.749 | lr 2.20e-04 | 102,456 tok/s | 54.64 GB | 12.751B tok iter 97300 | train 2.717 | val 2.724 | lr 2.20e-04 | 102,456 tok/s | 54.64 GB | 12.753B tok iter 97320 | train 2.626 | val 2.731 | lr 2.20e-04 | 102,458 tok/s | 54.64 GB | 12.756B tok iter 97340 | train 2.578 | val 2.745 | lr 2.19e-04 | 102,460 tok/s | 54.64 GB | 12.759B tok iter 97360 | train 2.792 | val 2.824 | lr 2.19e-04 | 102,461 tok/s | 54.64 GB | 12.761B tok iter 97380 | train 2.815 | val 2.753 | lr 2.19e-04 | 102,462 tok/s | 54.64 GB | 12.764B tok iter 97400 | train 2.758 | val 2.759 | lr 2.19e-04 | 102,463 tok/s | 54.64 GB | 12.767B tok iter 97420 | train 2.752 | val 2.755 | lr 2.19e-04 | 102,465 tok/s | 54.64 GB | 12.769B tok iter 97440 | train 2.703 | val 2.699 | lr 2.19e-04 | 102,467 tok/s | 54.64 GB | 12.772B tok iter 97460 | train 2.675 | val 2.762 | lr 2.19e-04 | 102,467 tok/s | 54.64 GB | 12.774B tok iter 97480 | train 2.690 | val 2.769 | lr 2.19e-04 | 102,469 tok/s | 54.64 GB | 12.777B tok checkpoint @ iter 97490 -> checkpoints/500m_latest.pt iter 97500 | train 2.544 | val 2.680 | lr 2.19e-04 | 102,459 tok/s | 54.64 GB | 12.780B tok checkpoint @ iter 97500 -> checkpoints/500m_latest.pt iter 97520 | train 2.588 | val 2.728 | lr 2.19e-04 | 102,437 tok/s | 54.64 GB | 12.782B tok iter 97540 | train 2.586 | val 2.738 | lr 2.18e-04 | 102,438 tok/s | 54.64 GB | 12.785B tok iter 97560 | train 2.650 | val 2.749 | lr 2.18e-04 | 102,439 tok/s | 54.64 GB | 12.788B tok iter 97580 | train 2.817 | val 2.699 | lr 2.18e-04 | 102,441 tok/s | 54.64 GB | 12.790B tok iter 97600 | train 2.622 | val 2.721 | lr 2.18e-04 | 102,443 tok/s | 54.64 GB | 12.793B tok iter 97620 | train 2.655 | val 2.695 | lr 2.18e-04 | 102,443 tok/s | 54.64 GB | 12.795B tok iter 97640 | train 2.780 | val 2.733 | lr 2.18e-04 | 102,445 tok/s | 54.64 GB | 12.798B tok iter 97660 | train 2.728 | val 2.721 | lr 2.18e-04 | 102,446 tok/s | 54.64 GB | 12.801B tok iter 97680 | train 2.721 | val 2.760 | lr 2.18e-04 | 102,448 tok/s | 54.64 GB | 12.803B tok iter 97700 | train 2.718 | val 2.770 | lr 2.18e-04 | 102,448 tok/s | 54.64 GB | 12.806B tok iter 97720 | train 2.738 | val 2.729 | lr 2.18e-04 | 102,450 tok/s | 54.64 GB | 12.808B tok iter 97740 | train 2.638 | val 2.746 | lr 2.17e-04 | 102,452 tok/s | 54.64 GB | 12.811B tok iter 97760 | train 2.907 | val 2.744 | lr 2.17e-04 | 102,453 tok/s | 54.64 GB | 12.814B tok iter 97780 | train 2.854 | val 2.735 | lr 2.17e-04 | 102,454 tok/s | 54.64 GB | 12.816B tok iter 97800 | train 2.760 | val 2.714 | lr 2.17e-04 | 102,455 tok/s | 54.64 GB | 12.819B tok iter 97820 | train 2.775 | val 2.778 | lr 2.17e-04 | 102,457 tok/s | 54.64 GB | 12.822B tok iter 97840 | train 2.799 | val 2.675 | lr 2.17e-04 | 102,459 tok/s | 54.64 GB | 12.824B tok iter 97860 | train 2.840 | val 2.704 | lr 2.17e-04 | 102,459 tok/s | 54.64 GB | 12.827B tok iter 97880 | train 2.745 | val 2.729 | lr 2.17e-04 | 102,461 tok/s | 54.64 GB | 12.829B tok iter 97900 | train 2.623 | val 2.751 | lr 2.17e-04 | 102,462 tok/s | 54.64 GB | 12.832B tok iter 97920 | train 2.639 | val 2.748 | lr 2.16e-04 | 102,464 tok/s | 54.64 GB | 12.835B tok iter 97940 | train 2.766 | val 2.698 | lr 2.16e-04 | 102,464 tok/s | 54.64 GB | 12.837B tok iter 97960 | train 2.692 | val 2.756 | lr 2.16e-04 | 102,466 tok/s | 54.64 GB | 12.840B tok iter 97980 | train 2.703 | val 2.689 | lr 2.16e-04 | 102,468 tok/s | 54.64 GB | 12.843B tok checkpoint @ iter 97990 -> checkpoints/500m_latest.pt iter 98000 | train 2.727 | val 2.709 | lr 2.16e-04 | 102,458 tok/s | 54.64 GB | 12.845B tok checkpoint @ iter 98000 -> checkpoints/500m_latest.pt iter 98020 | train 2.753 | val 2.749 | lr 2.16e-04 | 102,437 tok/s | 54.64 GB | 12.848B tok iter 98040 | train 2.751 | val 2.740 | lr 2.16e-04 | 102,438 tok/s | 54.64 GB | 12.850B tok iter 98060 | train 2.694 | val 2.716 | lr 2.16e-04 | 102,440 tok/s | 54.64 GB | 12.853B tok iter 98080 | train 2.787 | val 2.737 | lr 2.16e-04 | 102,442 tok/s | 54.64 GB | 12.856B tok iter 98100 | train 2.757 | val 2.681 | lr 2.16e-04 | 102,442 tok/s | 54.64 GB | 12.858B tok iter 98120 | train 2.681 | val 2.711 | lr 2.15e-04 | 102,444 tok/s | 54.64 GB | 12.861B tok iter 98140 | train 2.707 | val 2.761 | lr 2.15e-04 | 102,445 tok/s | 54.64 GB | 12.864B tok iter 98160 | train 2.718 | val 2.760 | lr 2.15e-04 | 102,447 tok/s | 54.64 GB | 12.866B tok iter 98180 | train 2.769 | val 2.748 | lr 2.15e-04 | 102,447 tok/s | 54.64 GB | 12.869B tok iter 98200 | train 2.722 | val 2.698 | lr 2.15e-04 | 102,449 tok/s | 54.64 GB | 12.871B tok iter 98220 | train 2.808 | val 2.663 | lr 2.15e-04 | 102,451 tok/s | 54.64 GB | 12.874B tok iter 98240 | train 2.843 | val 2.758 | lr 2.15e-04 | 102,452 tok/s | 54.64 GB | 12.877B tok iter 98260 | train 2.720 | val 2.709 | lr 2.15e-04 | 102,453 tok/s | 54.64 GB | 12.879B tok iter 98280 | train 2.806 | val 2.751 | lr 2.15e-04 | 102,454 tok/s | 54.64 GB | 12.882B tok iter 98300 | train 2.714 | val 2.752 | lr 2.15e-04 | 102,456 tok/s | 54.64 GB | 12.885B tok iter 98320 | train 2.895 | val 2.692 | lr 2.14e-04 | 102,457 tok/s | 54.64 GB | 12.887B tok iter 98340 | train 2.768 | val 2.756 | lr 2.14e-04 | 102,458 tok/s | 54.64 GB | 12.890B tok iter 98360 | train 2.937 | val 2.726 | lr 2.14e-04 | 102,460 tok/s | 54.64 GB | 12.892B tok iter 98380 | train 2.714 | val 2.729 | lr 2.14e-04 | 102,461 tok/s | 54.64 GB | 12.895B tok iter 98400 | train 2.761 | val 2.719 | lr 2.14e-04 | 102,463 tok/s | 54.64 GB | 12.898B tok iter 98420 | train 2.747 | val 2.744 | lr 2.14e-04 | 102,463 tok/s | 54.64 GB | 12.900B tok iter 98440 | train 2.793 | val 2.791 | lr 2.14e-04 | 102,465 tok/s | 54.64 GB | 12.903B tok iter 98460 | train 2.692 | val 2.709 | lr 2.14e-04 | 102,466 tok/s | 54.64 GB | 12.905B tok iter 98480 | train 2.730 | val 2.768 | lr 2.14e-04 | 102,468 tok/s | 54.64 GB | 12.908B tok checkpoint @ iter 98491 -> checkpoints/500m_latest.pt iter 98500 | train 2.635 | val 2.753 | lr 2.14e-04 | 102,458 tok/s | 54.64 GB | 12.911B tok checkpoint @ iter 98500 -> checkpoints/500m_latest.pt iter 98520 | train 2.785 | val 2.730 | lr 2.13e-04 | 102,438 tok/s | 54.64 GB | 12.913B tok iter 98540 | train 2.554 | val 2.769 | lr 2.13e-04 | 102,440 tok/s | 54.64 GB | 12.916B tok iter 98560 | train 2.709 | val 2.738 | lr 2.13e-04 | 102,441 tok/s | 54.64 GB | 12.919B tok iter 98580 | train 2.659 | val 2.707 | lr 2.13e-04 | 102,442 tok/s | 54.64 GB | 12.921B tok iter 98600 | train 2.913 | val 2.718 | lr 2.13e-04 | 102,443 tok/s | 54.64 GB | 12.924B tok iter 98620 | train 2.781 | val 2.735 | lr 2.13e-04 | 102,445 tok/s | 54.64 GB | 12.926B tok iter 98640 | train 2.756 | val 2.727 | lr 2.13e-04 | 102,447 tok/s | 54.64 GB | 12.929B tok iter 98660 | train 2.646 | val 2.745 | lr 2.13e-04 | 102,447 tok/s | 54.64 GB | 12.932B tok iter 98680 | train 2.807 | val 2.728 | lr 2.13e-04 | 102,449 tok/s | 54.64 GB | 12.934B tok iter 98700 | train 2.654 | val 2.741 | lr 2.13e-04 | 102,450 tok/s | 54.64 GB | 12.937B tok iter 98720 | train 2.592 | val 2.699 | lr 2.12e-04 | 102,452 tok/s | 54.64 GB | 12.940B tok iter 98740 | train 2.715 | val 2.757 | lr 2.12e-04 | 102,452 tok/s | 54.64 GB | 12.942B tok iter 98760 | train 2.746 | val 2.728 | lr 2.12e-04 | 102,454 tok/s | 54.64 GB | 12.945B tok iter 98780 | train 2.794 | val 2.743 | lr 2.12e-04 | 102,455 tok/s | 54.64 GB | 12.947B tok iter 98800 | train 2.698 | val 2.726 | lr 2.12e-04 | 102,457 tok/s | 54.64 GB | 12.950B tok iter 98820 | train 2.723 | val 2.748 | lr 2.12e-04 | 102,457 tok/s | 54.64 GB | 12.953B tok iter 98840 | train 2.817 | val 2.732 | lr 2.12e-04 | 102,459 tok/s | 54.64 GB | 12.955B tok iter 98860 | train 2.642 | val 2.754 | lr 2.12e-04 | 102,461 tok/s | 54.64 GB | 12.958B tok iter 98880 | train 2.722 | val 2.733 | lr 2.12e-04 | 102,462 tok/s | 54.64 GB | 12.961B tok iter 98900 | train 2.770 | val 2.714 | lr 2.12e-04 | 102,463 tok/s | 54.64 GB | 12.963B tok iter 98920 | train 2.692 | val 2.723 | lr 2.11e-04 | 102,464 tok/s | 54.64 GB | 12.966B tok iter 98940 | train 2.785 | val 2.759 | lr 2.11e-04 | 102,466 tok/s | 54.64 GB | 12.968B tok iter 98960 | train 2.680 | val 2.723 | lr 2.11e-04 | 102,467 tok/s | 54.64 GB | 12.971B tok iter 98980 | train 2.674 | val 2.753 | lr 2.11e-04 | 102,468 tok/s | 54.64 GB | 12.974B tok checkpoint @ iter 98991 -> checkpoints/500m_latest.pt iter 99000 | train 2.782 | val 2.725 | lr 2.11e-04 | 102,459 tok/s | 54.64 GB | 12.976B tok checkpoint @ iter 99000 -> checkpoints/500m_latest.pt iter 99020 | train 2.758 | val 2.702 | lr 2.11e-04 | 102,439 tok/s | 54.64 GB | 12.979B tok iter 99040 | train 2.651 | val 2.731 | lr 2.11e-04 | 102,441 tok/s | 54.64 GB | 12.982B tok iter 99060 | train 2.670 | val 2.711 | lr 2.11e-04 | 102,441 tok/s | 54.64 GB | 12.984B tok iter 99080 | train 2.779 | val 2.712 | lr 2.11e-04 | 102,443 tok/s | 54.64 GB | 12.987B tok iter 99100 | train 2.741 | val 2.707 | lr 2.11e-04 | 102,444 tok/s | 54.64 GB | 12.989B tok iter 99120 | train 2.772 | val 2.698 | lr 2.10e-04 | 102,446 tok/s | 54.64 GB | 12.992B tok iter 99140 | train 2.615 | val 2.751 | lr 2.10e-04 | 102,446 tok/s | 54.64 GB | 12.995B tok iter 99160 | train 2.790 | val 2.748 | lr 2.10e-04 | 102,448 tok/s | 54.64 GB | 12.997B tok iter 99180 | train 2.820 | val 2.718 | lr 2.10e-04 | 102,450 tok/s | 54.64 GB | 13.000B tok iter 99200 | train 2.652 | val 2.740 | lr 2.10e-04 | 102,451 tok/s | 54.64 GB | 13.002B tok iter 99220 | train 2.740 | val 2.737 | lr 2.10e-04 | 102,452 tok/s | 54.64 GB | 13.005B tok iter 99240 | train 2.767 | val 2.750 | lr 2.10e-04 | 102,453 tok/s | 54.64 GB | 13.008B tok iter 99260 | train 2.902 | val 2.691 | lr 2.10e-04 | 102,455 tok/s | 54.64 GB | 13.010B tok iter 99280 | train 2.613 | val 2.794 | lr 2.10e-04 | 102,456 tok/s | 54.64 GB | 13.013B tok iter 99300 | train 2.726 | val 2.694 | lr 2.10e-04 | 102,457 tok/s | 54.64 GB | 13.016B tok iter 99320 | train 2.785 | val 2.719 | lr 2.09e-04 | 102,459 tok/s | 54.64 GB | 13.018B tok iter 99340 | train 2.748 | val 2.759 | lr 2.09e-04 | 102,460 tok/s | 54.64 GB | 13.021B tok iter 99360 | train 2.755 | val 2.718 | lr 2.09e-04 | 102,462 tok/s | 54.64 GB | 13.023B tok iter 99380 | train 2.723 | val 2.748 | lr 2.09e-04 | 102,462 tok/s | 54.64 GB | 13.026B tok iter 99400 | train 2.763 | val 2.706 | lr 2.09e-04 | 102,464 tok/s | 54.64 GB | 13.029B tok iter 99420 | train 2.763 | val 2.722 | lr 2.09e-04 | 102,465 tok/s | 54.64 GB | 13.031B tok iter 99440 | train 2.808 | val 2.708 | lr 2.09e-04 | 102,467 tok/s | 54.64 GB | 13.034B tok iter 99460 | train 2.731 | val 2.768 | lr 2.09e-04 | 102,467 tok/s | 54.64 GB | 13.037B tok iter 99480 | train 2.721 | val 2.696 | lr 2.09e-04 | 102,469 tok/s | 54.64 GB | 13.039B tok checkpoint @ iter 99491 -> checkpoints/500m_latest.pt iter 99500 | train 2.718 | val 2.762 | lr 2.09e-04 | 102,460 tok/s | 54.64 GB | 13.042B tok checkpoint @ iter 99500 -> checkpoints/500m_latest.pt iter 99520 | train 2.808 | val 2.736 | lr 2.08e-04 | 102,440 tok/s | 54.64 GB | 13.044B tok iter 99540 | train 2.613 | val 2.724 | lr 2.08e-04 | 102,440 tok/s | 54.64 GB | 13.047B tok iter 99560 | train 2.717 | val 2.717 | lr 2.08e-04 | 102,442 tok/s | 54.64 GB | 13.050B tok iter 99580 | train 2.594 | val 2.713 | lr 2.08e-04 | 102,444 tok/s | 54.64 GB | 13.052B tok iter 99600 | train 2.727 | val 2.724 | lr 2.08e-04 | 102,445 tok/s | 54.64 GB | 13.055B tok iter 99620 | train 2.731 | val 2.763 | lr 2.08e-04 | 102,446 tok/s | 54.64 GB | 13.058B tok iter 99640 | train 2.684 | val 2.723 | lr 2.08e-04 | 102,447 tok/s | 54.64 GB | 13.060B tok iter 99660 | train 2.783 | val 2.725 | lr 2.08e-04 | 102,449 tok/s | 54.64 GB | 13.063B tok iter 99680 | train 2.724 | val 2.693 | lr 2.08e-04 | 102,450 tok/s | 54.64 GB | 13.065B tok iter 99700 | train 2.699 | val 2.757 | lr 2.08e-04 | 102,451 tok/s | 54.64 GB | 13.068B tok iter 99720 | train 2.728 | val 2.774 | lr 2.07e-04 | 102,452 tok/s | 54.64 GB | 13.071B tok iter 99740 | train 2.692 | val 2.745 | lr 2.07e-04 | 102,454 tok/s | 54.64 GB | 13.073B tok iter 99760 | train 2.723 | val 2.749 | lr 2.07e-04 | 102,456 tok/s | 54.64 GB | 13.076B tok iter 99780 | train 2.760 | val 2.729 | lr 2.07e-04 | 102,456 tok/s | 54.64 GB | 13.078B tok iter 99800 | train 2.814 | val 2.717 | lr 2.07e-04 | 102,458 tok/s | 54.64 GB | 13.081B tok iter 99820 | train 2.732 | val 2.702 | lr 2.07e-04 | 102,459 tok/s | 54.64 GB | 13.084B tok iter 99840 | train 2.873 | val 2.719 | lr 2.07e-04 | 102,461 tok/s | 54.64 GB | 13.086B tok iter 99860 | train 2.896 | val 2.730 | lr 2.07e-04 | 102,461 tok/s | 54.64 GB | 13.089B tok iter 99880 | train 2.671 | val 2.742 | lr 2.07e-04 | 102,463 tok/s | 54.64 GB | 13.092B tok iter 99900 | train 2.633 | val 2.719 | lr 2.07e-04 | 102,464 tok/s | 54.64 GB | 13.094B tok iter 99920 | train 2.694 | val 2.707 | lr 2.06e-04 | 102,466 tok/s | 54.64 GB | 13.097B tok iter 99940 | train 2.781 | val 2.742 | lr 2.06e-04 | 102,466 tok/s | 54.64 GB | 13.099B tok iter 99960 | train 2.640 | val 2.694 | lr 2.06e-04 | 102,468 tok/s | 54.64 GB | 13.102B tok iter 99980 | train 2.749 | val 2.675 | lr 2.06e-04 | 102,469 tok/s | 54.64 GB | 13.105B tok checkpoint @ iter 99991 -> checkpoints/500m_latest.pt iter 100000 | train 2.711 | val 2.724 | lr 2.06e-04 | 102,460 tok/s | 54.64 GB | 13.107B tok checkpoint @ iter 100000 -> checkpoints/500m_latest.pt iter 100020 | train 2.757 | val 2.676 | lr 2.06e-04 | 102,439 tok/s | 54.64 GB | 13.110B tok iter 100040 | train 2.751 | val 2.723 | lr 2.06e-04 | 102,441 tok/s | 54.64 GB | 13.113B tok iter 100060 | train 2.682 | val 2.727 | lr 2.06e-04 | 102,443 tok/s | 54.64 GB | 13.115B tok iter 100080 | train 2.721 | val 2.742 | lr 2.06e-04 | 102,444 tok/s | 54.64 GB | 13.118B tok iter 100100 | train 2.691 | val 2.742 | lr 2.06e-04 | 102,445 tok/s | 54.64 GB | 13.120B tok iter 100120 | train 2.521 | val 2.694 | lr 2.05e-04 | 102,446 tok/s | 54.64 GB | 13.123B tok iter 100140 | train 2.717 | val 2.710 | lr 2.05e-04 | 102,448 tok/s | 54.64 GB | 13.126B tok iter 100160 | train 2.785 | val 2.771 | lr 2.05e-04 | 102,449 tok/s | 54.64 GB | 13.128B tok iter 100180 | train 2.546 | val 2.701 | lr 2.05e-04 | 102,450 tok/s | 54.64 GB | 13.131B tok iter 100200 | train 2.746 | val 2.670 | lr 2.05e-04 | 102,451 tok/s | 54.64 GB | 13.134B tok iter 100220 | train 2.596 | val 2.705 | lr 2.05e-04 | 102,453 tok/s | 54.64 GB | 13.136B tok iter 100240 | train 2.824 | val 2.756 | lr 2.05e-04 | 102,455 tok/s | 54.64 GB | 13.139B tok iter 100260 | train 2.721 | val 2.764 | lr 2.05e-04 | 102,455 tok/s | 54.64 GB | 13.141B tok iter 100280 | train 2.750 | val 2.729 | lr 2.05e-04 | 102,457 tok/s | 54.64 GB | 13.144B tok iter 100300 | train 2.825 | val 2.729 | lr 2.05e-04 | 102,458 tok/s | 54.64 GB | 13.147B tok iter 100320 | train 2.835 | val 2.729 | lr 2.04e-04 | 102,460 tok/s | 54.64 GB | 13.149B tok iter 100340 | train 2.671 | val 2.716 | lr 2.04e-04 | 102,460 tok/s | 54.64 GB | 13.152B tok iter 100360 | train 2.786 | val 2.664 | lr 2.04e-04 | 102,462 tok/s | 54.64 GB | 13.155B tok iter 100380 | train 2.664 | val 2.708 | lr 2.04e-04 | 102,463 tok/s | 54.64 GB | 13.157B tok iter 100400 | train 2.687 | val 2.739 | lr 2.04e-04 | 102,465 tok/s | 54.64 GB | 13.160B tok iter 100420 | train 2.749 | val 2.743 | lr 2.04e-04 | 102,465 tok/s | 54.64 GB | 13.162B tok iter 100440 | train 2.580 | val 2.756 | lr 2.04e-04 | 102,467 tok/s | 54.64 GB | 13.165B tok iter 100460 | train 2.697 | val 2.761 | lr 2.04e-04 | 102,469 tok/s | 54.64 GB | 13.168B tok iter 100480 | train 2.732 | val 2.733 | lr 2.04e-04 | 102,470 tok/s | 54.64 GB | 13.170B tok checkpoint @ iter 100491 -> checkpoints/500m_latest.pt iter 100500 | train 2.707 | val 2.712 | lr 2.04e-04 | 102,460 tok/s | 54.64 GB | 13.173B tok checkpoint @ iter 100500 -> checkpoints/500m_latest.pt iter 100520 | train 2.650 | val 2.724 | lr 2.03e-04 | 102,441 tok/s | 54.64 GB | 13.175B tok iter 100540 | train 2.722 | val 2.766 | lr 2.03e-04 | 102,443 tok/s | 54.64 GB | 13.178B tok iter 100560 | train 2.727 | val 2.738 | lr 2.03e-04 | 102,445 tok/s | 54.64 GB | 13.181B tok iter 100580 | train 2.760 | val 2.757 | lr 2.03e-04 | 102,445 tok/s | 54.64 GB | 13.183B tok iter 100600 | train 2.597 | val 2.692 | lr 2.03e-04 | 102,447 tok/s | 54.64 GB | 13.186B tok iter 100620 | train 2.598 | val 2.717 | lr 2.03e-04 | 102,448 tok/s | 54.64 GB | 13.189B tok iter 100640 | train 2.788 | val 2.683 | lr 2.03e-04 | 102,450 tok/s | 54.64 GB | 13.191B tok iter 100660 | train 2.611 | val 2.689 | lr 2.03e-04 | 102,450 tok/s | 54.64 GB | 13.194B tok iter 100680 | train 2.767 | val 2.675 | lr 2.03e-04 | 102,452 tok/s | 54.64 GB | 13.196B tok iter 100700 | train 2.669 | val 2.713 | lr 2.03e-04 | 102,453 tok/s | 54.64 GB | 13.199B tok iter 100720 | train 2.771 | val 2.692 | lr 2.02e-04 | 102,455 tok/s | 54.64 GB | 13.202B tok iter 100740 | train 2.862 | val 2.740 | lr 2.02e-04 | 102,455 tok/s | 54.64 GB | 13.204B tok iter 100760 | train 2.712 | val 2.710 | lr 2.02e-04 | 102,457 tok/s | 54.64 GB | 13.207B tok iter 100780 | train 2.649 | val 2.712 | lr 2.02e-04 | 102,459 tok/s | 54.64 GB | 13.210B tok iter 100800 | train 2.672 | val 2.733 | lr 2.02e-04 | 102,460 tok/s | 54.64 GB | 13.212B tok iter 100820 | train 2.709 | val 2.698 | lr 2.02e-04 | 102,461 tok/s | 54.64 GB | 13.215B tok iter 100840 | train 2.682 | val 2.687 | lr 2.02e-04 | 102,462 tok/s | 54.64 GB | 13.217B tok iter 100860 | train 2.844 | val 2.741 | lr 2.02e-04 | 102,464 tok/s | 54.64 GB | 13.220B tok iter 100880 | train 2.827 | val 2.781 | lr 2.02e-04 | 102,465 tok/s | 54.64 GB | 13.223B tok iter 100900 | train 2.732 | val 2.755 | lr 2.02e-04 | 102,466 tok/s | 54.64 GB | 13.225B tok iter 100920 | train 2.832 | val 2.686 | lr 2.01e-04 | 102,467 tok/s | 54.64 GB | 13.228B tok iter 100940 | train 2.823 | val 2.714 | lr 2.01e-04 | 102,469 tok/s | 54.64 GB | 13.231B tok iter 100960 | train 2.734 | val 2.740 | lr 2.01e-04 | 102,471 tok/s | 54.64 GB | 13.233B tok iter 100980 | train 2.793 | val 2.739 | lr 2.01e-04 | 102,471 tok/s | 54.64 GB | 13.236B tok checkpoint @ iter 100991 -> checkpoints/500m_latest.pt iter 101000 | train 2.715 | val 2.724 | lr 2.01e-04 | 102,462 tok/s | 54.64 GB | 13.238B tok checkpoint @ iter 101000 -> checkpoints/500m_latest.pt iter 101020 | train 2.892 | val 2.683 | lr 2.01e-04 | 102,443 tok/s | 54.64 GB | 13.241B tok iter 101040 | train 2.915 | val 2.662 | lr 2.01e-04 | 102,444 tok/s | 54.64 GB | 13.244B tok iter 101060 | train 2.926 | val 2.775 | lr 2.01e-04 | 102,445 tok/s | 54.64 GB | 13.246B tok iter 101080 | train 2.777 | val 2.758 | lr 2.01e-04 | 102,446 tok/s | 54.64 GB | 13.249B tok iter 101100 | train 2.770 | val 2.742 | lr 2.01e-04 | 102,448 tok/s | 54.64 GB | 13.252B tok iter 101120 | train 2.747 | val 2.733 | lr 2.00e-04 | 102,450 tok/s | 54.64 GB | 13.254B tok iter 101140 | train 2.890 | val 2.713 | lr 2.00e-04 | 102,450 tok/s | 54.64 GB | 13.257B tok iter 101160 | train 2.691 | val 2.739 | lr 2.00e-04 | 102,451 tok/s | 54.64 GB | 13.259B tok iter 101180 | train 2.694 | val 2.688 | lr 2.00e-04 | 102,453 tok/s | 54.64 GB | 13.262B tok iter 101200 | train 2.622 | val 2.710 | lr 2.00e-04 | 102,455 tok/s | 54.64 GB | 13.265B tok iter 101220 | train 2.720 | val 2.758 | lr 2.00e-04 | 102,455 tok/s | 54.64 GB | 13.267B tok iter 101240 | train 2.732 | val 2.770 | lr 2.00e-04 | 102,456 tok/s | 54.64 GB | 13.270B tok iter 101260 | train 2.680 | val 2.717 | lr 2.00e-04 | 102,458 tok/s | 54.64 GB | 13.272B tok iter 101280 | train 2.552 | val 2.691 | lr 2.00e-04 | 102,460 tok/s | 54.64 GB | 13.275B tok iter 101300 | train 2.650 | val 2.711 | lr 2.00e-04 | 102,460 tok/s | 54.64 GB | 13.278B tok iter 101320 | train 2.638 | val 2.783 | lr 1.99e-04 | 102,461 tok/s | 54.64 GB | 13.280B tok iter 101340 | train 2.745 | val 2.724 | lr 1.99e-04 | 102,463 tok/s | 54.64 GB | 13.283B tok iter 101360 | train 2.835 | val 2.703 | lr 1.99e-04 | 102,465 tok/s | 54.64 GB | 13.286B tok iter 101380 | train 2.725 | val 2.712 | lr 1.99e-04 | 102,465 tok/s | 54.64 GB | 13.288B tok iter 101400 | train 2.861 | val 2.707 | lr 1.99e-04 | 102,466 tok/s | 54.64 GB | 13.291B tok iter 101420 | train 2.741 | val 2.735 | lr 1.99e-04 | 102,468 tok/s | 54.64 GB | 13.293B tok iter 101440 | train 2.646 | val 2.693 | lr 1.99e-04 | 102,469 tok/s | 54.64 GB | 13.296B tok iter 101460 | train 2.754 | val 2.716 | lr 1.99e-04 | 102,470 tok/s | 54.64 GB | 13.299B tok iter 101480 | train 2.588 | val 2.741 | lr 1.99e-04 | 102,471 tok/s | 54.64 GB | 13.301B tok checkpoint @ iter 101490 -> checkpoints/500m_latest.pt iter 101500 | train 2.622 | val 2.736 | lr 1.99e-04 | 102,462 tok/s | 54.64 GB | 13.304B tok checkpoint @ iter 101500 -> checkpoints/500m_latest.pt iter 101520 | train 2.735 | val 2.734 | lr 1.98e-04 | 102,443 tok/s | 54.64 GB | 13.307B tok iter 101540 | train 2.686 | val 2.727 | lr 1.98e-04 | 102,443 tok/s | 54.64 GB | 13.309B tok iter 101560 | train 2.677 | val 2.732 | lr 1.98e-04 | 102,445 tok/s | 54.64 GB | 13.312B tok iter 101580 | train 2.630 | val 2.748 | lr 1.98e-04 | 102,447 tok/s | 54.64 GB | 13.314B tok iter 101600 | train 2.685 | val 2.757 | lr 1.98e-04 | 102,448 tok/s | 54.64 GB | 13.317B tok iter 101620 | train 2.610 | val 2.700 | lr 1.98e-04 | 102,449 tok/s | 54.64 GB | 13.320B tok iter 101640 | train 2.750 | val 2.748 | lr 1.98e-04 | 102,450 tok/s | 54.64 GB | 13.322B tok iter 101660 | train 2.677 | val 2.725 | lr 1.98e-04 | 102,452 tok/s | 54.64 GB | 13.325B tok iter 101680 | train 2.740 | val 2.711 | lr 1.98e-04 | 102,453 tok/s | 54.64 GB | 13.328B tok iter 101700 | train 2.591 | val 2.723 | lr 1.98e-04 | 102,454 tok/s | 54.64 GB | 13.330B tok iter 101720 | train 2.737 | val 2.751 | lr 1.98e-04 | 102,455 tok/s | 54.64 GB | 13.333B tok iter 101740 | train 2.806 | val 2.694 | lr 1.97e-04 | 102,457 tok/s | 54.64 GB | 13.335B tok iter 101760 | train 2.693 | val 2.725 | lr 1.97e-04 | 102,458 tok/s | 54.64 GB | 13.338B tok iter 101780 | train 2.741 | val 2.719 | lr 1.97e-04 | 102,459 tok/s | 54.64 GB | 13.341B tok iter 101800 | train 2.734 | val 2.773 | lr 1.97e-04 | 102,460 tok/s | 54.64 GB | 13.343B tok iter 101820 | train 2.733 | val 2.705 | lr 1.97e-04 | 102,462 tok/s | 54.64 GB | 13.346B tok iter 101840 | train 2.780 | val 2.704 | lr 1.97e-04 | 102,463 tok/s | 54.64 GB | 13.349B tok iter 101860 | train 2.707 | val 2.755 | lr 1.97e-04 | 102,464 tok/s | 54.64 GB | 13.351B tok iter 101880 | train 2.648 | val 2.764 | lr 1.97e-04 | 102,465 tok/s | 54.64 GB | 13.354B tok iter 101900 | train 2.782 | val 2.768 | lr 1.97e-04 | 102,467 tok/s | 54.64 GB | 13.356B tok iter 101920 | train 2.598 | val 2.680 | lr 1.97e-04 | 102,468 tok/s | 54.64 GB | 13.359B tok iter 101940 | train 2.818 | val 2.680 | lr 1.96e-04 | 102,469 tok/s | 54.64 GB | 13.362B tok iter 101960 | train 2.742 | val 2.717 | lr 1.96e-04 | 102,470 tok/s | 54.64 GB | 13.364B tok iter 101980 | train 2.764 | val 2.729 | lr 1.96e-04 | 102,472 tok/s | 54.64 GB | 13.367B tok checkpoint @ iter 101990 -> checkpoints/500m_latest.pt iter 102000 | train 2.673 | val 2.731 | lr 1.96e-04 | 102,463 tok/s | 54.64 GB | 13.369B tok checkpoint @ iter 102000 -> checkpoints/500m_latest.pt iter 102020 | train 2.741 | val 2.717 | lr 1.96e-04 | 102,442 tok/s | 54.64 GB | 13.372B tok iter 102040 | train 2.635 | val 2.761 | lr 1.96e-04 | 102,444 tok/s | 54.64 GB | 13.375B tok iter 102060 | train 2.672 | val 2.719 | lr 1.96e-04 | 102,445 tok/s | 54.64 GB | 13.377B tok iter 102080 | train 2.899 | val 2.663 | lr 1.96e-04 | 102,447 tok/s | 54.64 GB | 13.380B tok iter 102100 | train 2.808 | val 2.694 | lr 1.96e-04 | 102,447 tok/s | 54.64 GB | 13.383B tok iter 102120 | train 2.651 | val 2.728 | lr 1.96e-04 | 102,449 tok/s | 54.64 GB | 13.385B tok iter 102140 | train 2.773 | val 2.730 | lr 1.95e-04 | 102,450 tok/s | 54.64 GB | 13.388B tok iter 102160 | train 2.669 | val 2.744 | lr 1.95e-04 | 102,452 tok/s | 54.64 GB | 13.390B tok iter 102180 | train 2.673 | val 2.742 | lr 1.95e-04 | 102,452 tok/s | 54.64 GB | 13.393B tok iter 102200 | train 2.804 | val 2.684 | lr 1.95e-04 | 102,454 tok/s | 54.64 GB | 13.396B tok iter 102220 | train 2.741 | val 2.763 | lr 1.95e-04 | 102,455 tok/s | 54.64 GB | 13.398B tok iter 102240 | train 2.526 | val 2.728 | lr 1.95e-04 | 102,457 tok/s | 54.64 GB | 13.401B tok iter 102260 | train 2.648 | val 2.687 | lr 1.95e-04 | 102,457 tok/s | 54.64 GB | 13.404B tok iter 102280 | train 2.784 | val 2.717 | lr 1.95e-04 | 102,459 tok/s | 54.64 GB | 13.406B tok iter 102300 | train 2.656 | val 2.724 | lr 1.95e-04 | 102,460 tok/s | 54.64 GB | 13.409B tok iter 102320 | train 2.680 | val 2.751 | lr 1.95e-04 | 102,462 tok/s | 54.64 GB | 13.411B tok iter 102340 | train 2.733 | val 2.676 | lr 1.94e-04 | 102,462 tok/s | 54.64 GB | 13.414B tok iter 102360 | train 2.802 | val 2.709 | lr 1.94e-04 | 102,464 tok/s | 54.64 GB | 13.417B tok iter 102380 | train 2.665 | val 2.771 | lr 1.94e-04 | 102,465 tok/s | 54.64 GB | 13.419B tok iter 102400 | train 2.652 | val 2.751 | lr 1.94e-04 | 102,467 tok/s | 54.64 GB | 13.422B tok iter 102420 | train 2.722 | val 2.762 | lr 1.94e-04 | 102,467 tok/s | 54.64 GB | 13.425B tok iter 102440 | train 2.798 | val 2.793 | lr 1.94e-04 | 102,469 tok/s | 54.64 GB | 13.427B tok iter 102460 | train 2.709 | val 2.725 | lr 1.94e-04 | 102,470 tok/s | 54.64 GB | 13.430B tok iter 102480 | train 2.701 | val 2.708 | lr 1.94e-04 | 102,472 tok/s | 54.64 GB | 13.432B tok checkpoint @ iter 102490 -> checkpoints/500m_latest.pt iter 102500 | train 2.615 | val 2.782 | lr 1.94e-04 | 102,462 tok/s | 54.64 GB | 13.435B tok checkpoint @ iter 102500 -> checkpoints/500m_latest.pt iter 102520 | train 2.838 | val 2.719 | lr 1.94e-04 | 102,443 tok/s | 54.64 GB | 13.438B tok iter 102540 | train 2.695 | val 2.707 | lr 1.94e-04 | 102,445 tok/s | 54.64 GB | 13.440B tok iter 102560 | train 2.680 | val 2.682 | lr 1.93e-04 | 102,447 tok/s | 54.64 GB | 13.443B tok iter 102580 | train 2.762 | val 2.753 | lr 1.93e-04 | 102,447 tok/s | 54.64 GB | 13.445B tok iter 102600 | train 2.528 | val 2.764 | lr 1.93e-04 | 102,448 tok/s | 54.64 GB | 13.448B tok iter 102620 | train 2.753 | val 2.735 | lr 1.93e-04 | 102,450 tok/s | 54.64 GB | 13.451B tok iter 102640 | train 2.688 | val 2.751 | lr 1.93e-04 | 102,452 tok/s | 54.64 GB | 13.453B tok iter 102660 | train 2.745 | val 2.727 | lr 1.93e-04 | 102,452 tok/s | 54.64 GB | 13.456B tok iter 102680 | train 2.744 | val 2.698 | lr 1.93e-04 | 102,453 tok/s | 54.64 GB | 13.459B tok iter 102700 | train 2.831 | val 2.708 | lr 1.93e-04 | 102,455 tok/s | 54.64 GB | 13.461B tok iter 102720 | train 2.651 | val 2.720 | lr 1.93e-04 | 102,457 tok/s | 54.64 GB | 13.464B tok iter 102740 | train 2.554 | val 2.723 | lr 1.93e-04 | 102,457 tok/s | 54.64 GB | 13.466B tok iter 102760 | train 2.800 | val 2.689 | lr 1.92e-04 | 102,458 tok/s | 54.64 GB | 13.469B tok iter 102780 | train 2.682 | val 2.707 | lr 1.92e-04 | 102,460 tok/s | 54.64 GB | 13.472B tok iter 102800 | train 2.755 | val 2.699 | lr 1.92e-04 | 102,462 tok/s | 54.64 GB | 13.474B tok iter 102820 | train 2.671 | val 2.632 | lr 1.92e-04 | 102,462 tok/s | 54.64 GB | 13.477B tok iter 102840 | train 2.632 | val 2.744 | lr 1.92e-04 | 102,463 tok/s | 54.64 GB | 13.480B tok iter 102860 | train 2.739 | val 2.710 | lr 1.92e-04 | 102,465 tok/s | 54.64 GB | 13.482B tok iter 102880 | train 2.808 | val 2.667 | lr 1.92e-04 | 102,466 tok/s | 54.64 GB | 13.485B tok iter 102900 | train 2.726 | val 2.708 | lr 1.92e-04 | 102,467 tok/s | 54.64 GB | 13.487B tok iter 102920 | train 2.730 | val 2.697 | lr 1.92e-04 | 102,468 tok/s | 54.64 GB | 13.490B tok iter 102940 | train 2.649 | val 2.728 | lr 1.92e-04 | 102,470 tok/s | 54.64 GB | 13.493B tok iter 102960 | train 2.623 | val 2.746 | lr 1.91e-04 | 102,471 tok/s | 54.64 GB | 13.495B tok iter 102980 | train 2.736 | val 2.721 | lr 1.91e-04 | 102,472 tok/s | 54.64 GB | 13.498B tok checkpoint @ iter 102991 -> checkpoints/500m_latest.pt iter 103000 | train 2.836 | val 2.774 | lr 1.91e-04 | 102,463 tok/s | 54.64 GB | 13.501B tok checkpoint @ iter 103000 -> checkpoints/500m_latest.pt iter 103020 | train 2.663 | val 2.714 | lr 1.91e-04 | 102,445 tok/s | 54.64 GB | 13.503B tok iter 103040 | train 2.671 | val 2.697 | lr 1.91e-04 | 102,446 tok/s | 54.64 GB | 13.506B tok iter 103060 | train 2.692 | val 2.717 | lr 1.91e-04 | 102,447 tok/s | 54.64 GB | 13.508B tok iter 103080 | train 2.792 | val 2.690 | lr 1.91e-04 | 102,448 tok/s | 54.64 GB | 13.511B tok iter 103100 | train 2.752 | val 2.773 | lr 1.91e-04 | 102,450 tok/s | 54.64 GB | 13.514B tok iter 103120 | train 2.723 | val 2.719 | lr 1.91e-04 | 102,451 tok/s | 54.64 GB | 13.516B tok iter 103140 | train 2.644 | val 2.716 | lr 1.91e-04 | 102,451 tok/s | 54.64 GB | 13.519B tok iter 103160 | train 2.716 | val 2.672 | lr 1.91e-04 | 102,453 tok/s | 54.64 GB | 13.522B tok iter 103180 | train 2.809 | val 2.699 | lr 1.90e-04 | 102,455 tok/s | 54.64 GB | 13.524B tok iter 103200 | train 2.783 | val 2.729 | lr 1.90e-04 | 102,456 tok/s | 54.64 GB | 13.527B tok iter 103220 | train 2.564 | val 2.686 | lr 1.90e-04 | 102,456 tok/s | 54.64 GB | 13.529B tok iter 103240 | train 2.690 | val 2.765 | lr 1.90e-04 | 102,458 tok/s | 54.64 GB | 13.532B tok iter 103260 | train 2.696 | val 2.742 | lr 1.90e-04 | 102,459 tok/s | 54.64 GB | 13.535B tok iter 103280 | train 2.794 | val 2.727 | lr 1.90e-04 | 102,461 tok/s | 54.64 GB | 13.537B tok iter 103300 | train 2.803 | val 2.692 | lr 1.90e-04 | 102,461 tok/s | 54.64 GB | 13.540B tok iter 103320 | train 2.642 | val 2.740 | lr 1.90e-04 | 102,463 tok/s | 54.64 GB | 13.542B tok iter 103340 | train 2.652 | val 2.758 | lr 1.90e-04 | 102,464 tok/s | 54.64 GB | 13.545B tok iter 103360 | train 2.685 | val 2.701 | lr 1.90e-04 | 102,466 tok/s | 54.64 GB | 13.548B tok iter 103380 | train 2.713 | val 2.700 | lr 1.89e-04 | 102,466 tok/s | 54.64 GB | 13.550B tok iter 103400 | train 2.811 | val 2.640 | lr 1.89e-04 | 102,468 tok/s | 54.64 GB | 13.553B tok iter 103420 | train 2.675 | val 2.751 | lr 1.89e-04 | 102,469 tok/s | 54.64 GB | 13.556B tok iter 103440 | train 2.765 | val 2.688 | lr 1.89e-04 | 102,471 tok/s | 54.64 GB | 13.558B tok iter 103460 | train 2.806 | val 2.669 | lr 1.89e-04 | 102,471 tok/s | 54.64 GB | 13.561B tok iter 103480 | train 2.743 | val 2.688 | lr 1.89e-04 | 102,473 tok/s | 54.64 GB | 13.563B tok checkpoint @ iter 103490 -> checkpoints/500m_latest.pt iter 103500 | train 2.669 | val 2.738 | lr 1.89e-04 | 102,464 tok/s | 54.64 GB | 13.566B tok checkpoint @ iter 103500 -> checkpoints/500m_latest.pt iter 103520 | train 2.711 | val 2.801 | lr 1.89e-04 | 102,446 tok/s | 54.64 GB | 13.569B tok iter 103540 | train 2.661 | val 2.716 | lr 1.89e-04 | 102,447 tok/s | 54.64 GB | 13.571B tok iter 103560 | train 2.659 | val 2.704 | lr 1.89e-04 | 102,448 tok/s | 54.64 GB | 13.574B tok iter 103580 | train 2.719 | val 2.694 | lr 1.89e-04 | 102,450 tok/s | 54.64 GB | 13.577B tok iter 103600 | train 2.714 | val 2.759 | lr 1.88e-04 | 102,451 tok/s | 54.64 GB | 13.579B tok iter 103620 | train 2.572 | val 2.719 | lr 1.88e-04 | 102,452 tok/s | 54.64 GB | 13.582B tok iter 103640 | train 2.609 | val 2.720 | lr 1.88e-04 | 102,453 tok/s | 54.64 GB | 13.584B tok iter 103660 | train 2.732 | val 2.759 | lr 1.88e-04 | 102,455 tok/s | 54.64 GB | 13.587B tok iter 103680 | train 2.679 | val 2.737 | lr 1.88e-04 | 102,456 tok/s | 54.64 GB | 13.590B tok iter 103700 | train 2.654 | val 2.730 | lr 1.88e-04 | 102,457 tok/s | 54.64 GB | 13.592B tok iter 103720 | train 2.793 | val 2.766 | lr 1.88e-04 | 102,458 tok/s | 54.64 GB | 13.595B tok iter 103740 | train 2.637 | val 2.702 | lr 1.88e-04 | 102,460 tok/s | 54.64 GB | 13.598B tok iter 103760 | train 2.632 | val 2.713 | lr 1.88e-04 | 102,461 tok/s | 54.64 GB | 13.600B tok iter 103780 | train 2.778 | val 2.708 | lr 1.88e-04 | 102,461 tok/s | 54.64 GB | 13.603B tok iter 103800 | train 2.706 | val 2.723 | lr 1.87e-04 | 102,463 tok/s | 54.64 GB | 13.605B tok iter 103820 | train 2.659 | val 2.718 | lr 1.87e-04 | 102,465 tok/s | 54.64 GB | 13.608B tok iter 103840 | train 2.718 | val 2.738 | lr 1.87e-04 | 102,466 tok/s | 54.64 GB | 13.611B tok iter 103860 | train 2.682 | val 2.713 | lr 1.87e-04 | 102,466 tok/s | 54.64 GB | 13.613B tok iter 103880 | train 2.679 | val 2.731 | lr 1.87e-04 | 102,468 tok/s | 54.64 GB | 13.616B tok iter 103900 | train 2.801 | val 2.732 | lr 1.87e-04 | 102,469 tok/s | 54.64 GB | 13.619B tok iter 103920 | train 2.790 | val 2.681 | lr 1.87e-04 | 102,471 tok/s | 54.64 GB | 13.621B tok iter 103940 | train 2.805 | val 2.680 | lr 1.87e-04 | 102,471 tok/s | 54.64 GB | 13.624B tok iter 103960 | train 2.734 | val 2.695 | lr 1.87e-04 | 102,473 tok/s | 54.64 GB | 13.626B tok iter 103980 | train 2.620 | val 2.716 | lr 1.87e-04 | 102,474 tok/s | 54.64 GB | 13.629B tok checkpoint @ iter 103991 -> checkpoints/500m_latest.pt iter 104000 | train 2.827 | val 2.704 | lr 1.87e-04 | 102,466 tok/s | 54.64 GB | 13.632B tok checkpoint @ iter 104000 -> checkpoints/500m_latest.pt iter 104020 | train 2.586 | val 2.694 | lr 1.86e-04 | 102,447 tok/s | 54.64 GB | 13.634B tok iter 104040 | train 2.784 | val 2.735 | lr 1.86e-04 | 102,448 tok/s | 54.64 GB | 13.637B tok iter 104060 | train 2.677 | val 2.727 | lr 1.86e-04 | 102,450 tok/s | 54.64 GB | 13.639B tok iter 104080 | train 2.644 | val 2.685 | lr 1.86e-04 | 102,451 tok/s | 54.64 GB | 13.642B tok iter 104100 | train 2.644 | val 2.700 | lr 1.86e-04 | 102,452 tok/s | 54.64 GB | 13.645B tok iter 104120 | train 2.712 | val 2.713 | lr 1.86e-04 | 102,453 tok/s | 54.64 GB | 13.647B tok iter 104140 | train 2.758 | val 2.686 | lr 1.86e-04 | 102,455 tok/s | 54.64 GB | 13.650B tok iter 104160 | train 2.664 | val 2.631 | lr 1.86e-04 | 102,456 tok/s | 54.64 GB | 13.653B tok iter 104180 | train 2.771 | val 2.718 | lr 1.86e-04 | 102,457 tok/s | 54.64 GB | 13.655B tok iter 104200 | train 2.757 | val 2.644 | lr 1.86e-04 | 102,458 tok/s | 54.64 GB | 13.658B tok iter 104220 | train 2.761 | val 2.690 | lr 1.85e-04 | 102,460 tok/s | 54.64 GB | 13.660B tok iter 104240 | train 2.764 | val 2.708 | lr 1.85e-04 | 102,461 tok/s | 54.64 GB | 13.663B tok iter 104260 | train 2.615 | val 2.708 | lr 1.85e-04 | 102,461 tok/s | 54.64 GB | 13.666B tok iter 104280 | train 2.695 | val 2.704 | lr 1.85e-04 | 102,463 tok/s | 54.64 GB | 13.668B tok iter 104300 | train 2.771 | val 2.667 | lr 1.85e-04 | 102,465 tok/s | 54.64 GB | 13.671B tok iter 104320 | train 2.671 | val 2.706 | lr 1.85e-04 | 102,466 tok/s | 54.64 GB | 13.674B tok iter 104340 | train 2.598 | val 2.745 | lr 1.85e-04 | 102,466 tok/s | 54.64 GB | 13.676B tok iter 104360 | train 2.722 | val 2.749 | lr 1.85e-04 | 102,468 tok/s | 54.64 GB | 13.679B tok iter 104380 | train 2.725 | val 2.685 | lr 1.85e-04 | 102,469 tok/s | 54.64 GB | 13.681B tok iter 104400 | train 2.684 | val 2.713 | lr 1.85e-04 | 102,471 tok/s | 54.64 GB | 13.684B tok iter 104420 | train 2.690 | val 2.740 | lr 1.85e-04 | 102,471 tok/s | 54.64 GB | 13.687B tok iter 104440 | train 2.600 | val 2.702 | lr 1.84e-04 | 102,473 tok/s | 54.64 GB | 13.689B tok iter 104460 | train 2.740 | val 2.749 | lr 1.84e-04 | 102,474 tok/s | 54.64 GB | 13.692B tok iter 104480 | train 2.766 | val 2.713 | lr 1.84e-04 | 102,476 tok/s | 54.64 GB | 13.695B tok checkpoint @ iter 104491 -> checkpoints/500m_latest.pt iter 104500 | train 2.685 | val 2.667 | lr 1.84e-04 | 102,466 tok/s | 54.64 GB | 13.697B tok checkpoint @ iter 104500 -> checkpoints/500m_latest.pt iter 104520 | train 2.660 | val 2.748 | lr 1.84e-04 | 102,448 tok/s | 54.64 GB | 13.700B tok iter 104540 | train 2.656 | val 2.714 | lr 1.84e-04 | 102,450 tok/s | 54.64 GB | 13.702B tok iter 104560 | train 2.667 | val 2.693 | lr 1.84e-04 | 102,451 tok/s | 54.64 GB | 13.705B tok iter 104580 | train 2.803 | val 2.702 | lr 1.84e-04 | 102,451 tok/s | 54.64 GB | 13.708B tok iter 104600 | train 2.727 | val 2.679 | lr 1.84e-04 | 102,453 tok/s | 54.64 GB | 13.710B tok iter 104620 | train 2.717 | val 2.773 | lr 1.84e-04 | 102,455 tok/s | 54.64 GB | 13.713B tok iter 104640 | train 2.839 | val 2.755 | lr 1.83e-04 | 102,456 tok/s | 54.64 GB | 13.716B tok iter 104660 | train 2.648 | val 2.686 | lr 1.83e-04 | 102,456 tok/s | 54.64 GB | 13.718B tok iter 104680 | train 2.839 | val 2.712 | lr 1.83e-04 | 102,458 tok/s | 54.64 GB | 13.721B tok iter 104700 | train 2.674 | val 2.666 | lr 1.83e-04 | 102,459 tok/s | 54.64 GB | 13.723B tok iter 104720 | train 2.765 | val 2.727 | lr 1.83e-04 | 102,461 tok/s | 54.64 GB | 13.726B tok iter 104740 | train 2.600 | val 2.723 | lr 1.83e-04 | 102,461 tok/s | 54.64 GB | 13.729B tok iter 104760 | train 2.679 | val 2.685 | lr 1.83e-04 | 102,463 tok/s | 54.64 GB | 13.731B tok iter 104780 | train 2.574 | val 2.675 | lr 1.83e-04 | 102,464 tok/s | 54.64 GB | 13.734B tok iter 104800 | train 2.689 | val 2.692 | lr 1.83e-04 | 102,466 tok/s | 54.64 GB | 13.736B tok iter 104820 | train 2.768 | val 2.812 | lr 1.83e-04 | 102,466 tok/s | 54.64 GB | 13.739B tok iter 104840 | train 2.780 | val 2.761 | lr 1.83e-04 | 102,467 tok/s | 54.64 GB | 13.742B tok iter 104860 | train 2.799 | val 2.703 | lr 1.82e-04 | 102,469 tok/s | 54.64 GB | 13.744B tok iter 104880 | train 2.770 | val 2.671 | lr 1.82e-04 | 102,470 tok/s | 54.64 GB | 13.747B tok iter 104900 | train 2.741 | val 2.703 | lr 1.82e-04 | 102,471 tok/s | 54.64 GB | 13.750B tok iter 104920 | train 2.703 | val 2.737 | lr 1.82e-04 | 102,472 tok/s | 54.64 GB | 13.752B tok iter 104940 | train 2.699 | val 2.677 | lr 1.82e-04 | 102,474 tok/s | 54.64 GB | 13.755B tok iter 104960 | train 2.803 | val 2.724 | lr 1.82e-04 | 102,475 tok/s | 54.64 GB | 13.757B tok iter 104980 | train 2.697 | val 2.759 | lr 1.82e-04 | 102,476 tok/s | 54.64 GB | 13.760B tok checkpoint @ iter 104990 -> checkpoints/500m_latest.pt iter 105000 | train 2.636 | val 2.700 | lr 1.82e-04 | 102,468 tok/s | 54.64 GB | 13.763B tok checkpoint @ iter 105000 -> checkpoints/500m_latest.pt iter 105020 | train 2.671 | val 2.712 | lr 1.82e-04 | 102,449 tok/s | 54.64 GB | 13.765B tok iter 105040 | train 2.760 | val 2.648 | lr 1.82e-04 | 102,451 tok/s | 54.64 GB | 13.768B tok iter 105060 | train 2.499 | val 2.730 | lr 1.81e-04 | 102,451 tok/s | 54.64 GB | 13.771B tok iter 105080 | train 2.749 | val 2.717 | lr 1.81e-04 | 102,453 tok/s | 54.64 GB | 13.773B tok iter 105100 | train 2.623 | val 2.734 | lr 1.81e-04 | 102,454 tok/s | 54.64 GB | 13.776B tok iter 105120 | train 2.782 | val 2.685 | lr 1.81e-04 | 102,456 tok/s | 54.64 GB | 13.778B tok iter 105140 | train 2.567 | val 2.703 | lr 1.81e-04 | 102,456 tok/s | 54.64 GB | 13.781B tok iter 105160 | train 2.689 | val 2.706 | lr 1.81e-04 | 102,458 tok/s | 54.64 GB | 13.784B tok iter 105180 | train 2.693 | val 2.691 | lr 1.81e-04 | 102,459 tok/s | 54.64 GB | 13.786B tok iter 105200 | train 2.703 | val 2.739 | lr 1.81e-04 | 102,461 tok/s | 54.64 GB | 13.789B tok iter 105220 | train 2.646 | val 2.686 | lr 1.81e-04 | 102,461 tok/s | 54.64 GB | 13.792B tok iter 105240 | train 2.792 | val 2.706 | lr 1.81e-04 | 102,462 tok/s | 54.64 GB | 13.794B tok iter 105260 | train 2.835 | val 2.694 | lr 1.81e-04 | 102,464 tok/s | 54.64 GB | 13.797B tok iter 105280 | train 2.675 | val 2.724 | lr 1.80e-04 | 102,465 tok/s | 54.64 GB | 13.799B tok iter 105300 | train 2.832 | val 2.717 | lr 1.80e-04 | 102,466 tok/s | 54.64 GB | 13.802B tok iter 105320 | train 2.648 | val 2.711 | lr 1.80e-04 | 102,467 tok/s | 54.64 GB | 13.805B tok iter 105340 | train 2.699 | val 2.693 | lr 1.80e-04 | 102,469 tok/s | 54.64 GB | 13.807B tok iter 105360 | train 2.771 | val 2.682 | lr 1.80e-04 | 102,470 tok/s | 54.64 GB | 13.810B tok iter 105380 | train 2.798 | val 2.689 | lr 1.80e-04 | 102,471 tok/s | 54.64 GB | 13.812B tok iter 105400 | train 2.946 | val 2.655 | lr 1.80e-04 | 102,472 tok/s | 54.64 GB | 13.815B tok iter 105420 | train 2.732 | val 2.694 | lr 1.80e-04 | 102,474 tok/s | 54.64 GB | 13.818B tok iter 105440 | train 2.622 | val 2.689 | lr 1.80e-04 | 102,475 tok/s | 54.64 GB | 13.820B tok iter 105460 | train 2.754 | val 2.699 | lr 1.80e-04 | 102,475 tok/s | 54.64 GB | 13.823B tok iter 105480 | train 2.815 | val 2.689 | lr 1.80e-04 | 102,477 tok/s | 54.64 GB | 13.826B tok checkpoint @ iter 105491 -> checkpoints/500m_latest.pt iter 105500 | train 2.774 | val 2.690 | lr 1.79e-04 | 102,469 tok/s | 54.64 GB | 13.828B tok checkpoint @ iter 105500 -> checkpoints/500m_latest.pt iter 105520 | train 2.756 | val 2.715 | lr 1.79e-04 | 102,451 tok/s | 54.64 GB | 13.831B tok iter 105540 | train 2.696 | val 2.727 | lr 1.79e-04 | 102,451 tok/s | 54.64 GB | 13.833B tok iter 105560 | train 2.738 | val 2.759 | lr 1.79e-04 | 102,453 tok/s | 54.64 GB | 13.836B tok iter 105580 | train 2.690 | val 2.666 | lr 1.79e-04 | 102,454 tok/s | 54.64 GB | 13.839B tok iter 105600 | train 2.671 | val 2.717 | lr 1.79e-04 | 102,456 tok/s | 54.64 GB | 13.841B tok iter 105620 | train 2.776 | val 2.710 | lr 1.79e-04 | 102,456 tok/s | 54.64 GB | 13.844B tok iter 105640 | train 2.760 | val 2.745 | lr 1.79e-04 | 102,457 tok/s | 54.64 GB | 13.847B tok iter 105660 | train 2.702 | val 2.703 | lr 1.79e-04 | 102,459 tok/s | 54.64 GB | 13.849B tok iter 105680 | train 2.741 | val 2.738 | lr 1.79e-04 | 102,460 tok/s | 54.64 GB | 13.852B tok iter 105700 | train 2.702 | val 2.706 | lr 1.79e-04 | 102,461 tok/s | 54.64 GB | 13.854B tok iter 105720 | train 2.583 | val 2.719 | lr 1.78e-04 | 102,462 tok/s | 54.64 GB | 13.857B tok iter 105740 | train 2.780 | val 2.729 | lr 1.78e-04 | 102,464 tok/s | 54.64 GB | 13.860B tok iter 105760 | train 2.813 | val 2.707 | lr 1.78e-04 | 102,465 tok/s | 54.64 GB | 13.862B tok iter 105780 | train 2.693 | val 2.771 | lr 1.78e-04 | 102,466 tok/s | 54.64 GB | 13.865B tok iter 105800 | train 2.697 | val 2.685 | lr 1.78e-04 | 102,467 tok/s | 54.64 GB | 13.868B tok iter 105820 | train 2.775 | val 2.722 | lr 1.78e-04 | 102,469 tok/s | 54.64 GB | 13.870B tok iter 105840 | train 2.676 | val 2.717 | lr 1.78e-04 | 102,470 tok/s | 54.64 GB | 13.873B tok iter 105860 | train 2.694 | val 2.719 | lr 1.78e-04 | 102,470 tok/s | 54.64 GB | 13.875B tok iter 105880 | train 2.713 | val 2.652 | lr 1.78e-04 | 102,472 tok/s | 54.64 GB | 13.878B tok iter 105900 | train 2.680 | val 2.681 | lr 1.78e-04 | 102,473 tok/s | 54.64 GB | 13.881B tok iter 105920 | train 2.634 | val 2.668 | lr 1.77e-04 | 102,475 tok/s | 54.64 GB | 13.883B tok iter 105940 | train 2.777 | val 2.722 | lr 1.77e-04 | 102,475 tok/s | 54.64 GB | 13.886B tok iter 105960 | train 2.575 | val 2.673 | lr 1.77e-04 | 102,477 tok/s | 54.64 GB | 13.889B tok iter 105980 | train 2.854 | val 2.720 | lr 1.77e-04 | 102,478 tok/s | 54.64 GB | 13.891B tok checkpoint @ iter 105991 -> checkpoints/500m_latest.pt iter 106000 | train 2.685 | val 2.665 | lr 1.77e-04 | 102,470 tok/s | 54.64 GB | 13.894B tok checkpoint @ iter 106000 -> checkpoints/500m_latest.pt iter 106020 | train 2.815 | val 2.734 | lr 1.77e-04 | 102,451 tok/s | 54.64 GB | 13.896B tok iter 106040 | train 2.780 | val 2.719 | lr 1.77e-04 | 102,452 tok/s | 54.64 GB | 13.899B tok iter 106060 | train 2.567 | val 2.696 | lr 1.77e-04 | 102,454 tok/s | 54.64 GB | 13.902B tok iter 106080 | train 2.566 | val 2.676 | lr 1.77e-04 | 102,455 tok/s | 54.64 GB | 13.904B tok iter 106100 | train 2.707 | val 2.707 | lr 1.77e-04 | 102,456 tok/s | 54.64 GB | 13.907B tok iter 106120 | train 2.541 | val 2.680 | lr 1.77e-04 | 102,457 tok/s | 54.64 GB | 13.909B tok iter 106140 | train 2.757 | val 2.683 | lr 1.76e-04 | 102,459 tok/s | 54.64 GB | 13.912B tok iter 106160 | train 2.809 | val 2.754 | lr 1.76e-04 | 102,460 tok/s | 54.64 GB | 13.915B tok iter 106180 | train 2.670 | val 2.709 | lr 1.76e-04 | 102,460 tok/s | 54.64 GB | 13.917B tok iter 106200 | train 2.634 | val 2.707 | lr 1.76e-04 | 102,462 tok/s | 54.64 GB | 13.920B tok iter 106220 | train 2.579 | val 2.734 | lr 1.76e-04 | 102,463 tok/s | 54.64 GB | 13.923B tok iter 106240 | train 2.682 | val 2.700 | lr 1.76e-04 | 102,465 tok/s | 54.64 GB | 13.925B tok iter 106260 | train 2.681 | val 2.715 | lr 1.76e-04 | 102,465 tok/s | 54.64 GB | 13.928B tok iter 106280 | train 2.863 | val 2.709 | lr 1.76e-04 | 102,467 tok/s | 54.64 GB | 13.930B tok iter 106300 | train 2.646 | val 2.642 | lr 1.76e-04 | 102,468 tok/s | 54.64 GB | 13.933B tok iter 106320 | train 2.658 | val 2.682 | lr 1.76e-04 | 102,470 tok/s | 54.64 GB | 13.936B tok iter 106340 | train 2.639 | val 2.685 | lr 1.76e-04 | 102,470 tok/s | 54.64 GB | 13.938B tok iter 106360 | train 2.680 | val 2.702 | lr 1.75e-04 | 102,471 tok/s | 54.64 GB | 13.941B tok iter 106380 | train 2.647 | val 2.703 | lr 1.75e-04 | 102,473 tok/s | 54.64 GB | 13.944B tok iter 106400 | train 2.640 | val 2.705 | lr 1.75e-04 | 102,474 tok/s | 54.64 GB | 13.946B tok iter 106420 | train 2.504 | val 2.730 | lr 1.75e-04 | 102,475 tok/s | 54.64 GB | 13.949B tok iter 106440 | train 2.627 | val 2.648 | lr 1.75e-04 | 102,476 tok/s | 54.64 GB | 13.951B tok iter 106460 | train 2.696 | val 2.710 | lr 1.75e-04 | 102,478 tok/s | 54.64 GB | 13.954B tok iter 106480 | train 2.670 | val 2.713 | lr 1.75e-04 | 102,479 tok/s | 54.64 GB | 13.957B tok checkpoint @ iter 106491 -> checkpoints/500m_latest.pt iter 106500 | train 2.713 | val 2.701 | lr 1.75e-04 | 102,470 tok/s | 54.64 GB | 13.959B tok checkpoint @ iter 106500 -> checkpoints/500m_latest.pt iter 106520 | train 2.751 | val 2.694 | lr 1.75e-04 | 102,451 tok/s | 54.64 GB | 13.962B tok iter 106540 | train 2.657 | val 2.680 | lr 1.75e-04 | 102,453 tok/s | 54.64 GB | 13.965B tok iter 106560 | train 2.871 | val 2.746 | lr 1.75e-04 | 102,454 tok/s | 54.64 GB | 13.967B tok iter 106580 | train 2.667 | val 2.726 | lr 1.74e-04 | 102,454 tok/s | 54.64 GB | 13.970B tok iter 106600 | train 2.648 | val 2.697 | lr 1.74e-04 | 102,456 tok/s | 54.64 GB | 13.972B tok iter 106620 | train 2.737 | val 2.738 | lr 1.74e-04 | 102,457 tok/s | 54.64 GB | 13.975B tok iter 106640 | train 2.789 | val 2.693 | lr 1.74e-04 | 102,459 tok/s | 54.64 GB | 13.978B tok iter 106660 | train 2.623 | val 2.711 | lr 1.74e-04 | 102,459 tok/s | 54.64 GB | 13.980B tok iter 106680 | train 2.718 | val 2.706 | lr 1.74e-04 | 102,461 tok/s | 54.64 GB | 13.983B tok iter 106700 | train 2.742 | val 2.686 | lr 1.74e-04 | 102,462 tok/s | 54.64 GB | 13.986B tok iter 106720 | train 2.660 | val 2.648 | lr 1.74e-04 | 102,464 tok/s | 54.64 GB | 13.988B tok iter 106740 | train 2.456 | val 2.681 | lr 1.74e-04 | 102,464 tok/s | 54.64 GB | 13.991B tok iter 106760 | train 2.685 | val 2.701 | lr 1.74e-04 | 102,466 tok/s | 54.64 GB | 13.993B tok iter 106780 | train 2.809 | val 2.719 | lr 1.74e-04 | 102,467 tok/s | 54.64 GB | 13.996B tok iter 106800 | train 2.689 | val 2.677 | lr 1.73e-04 | 102,469 tok/s | 54.64 GB | 13.999B tok iter 106820 | train 2.680 | val 2.764 | lr 1.73e-04 | 102,469 tok/s | 54.64 GB | 14.001B tok iter 106840 | train 2.722 | val 2.723 | lr 1.73e-04 | 102,470 tok/s | 54.64 GB | 14.004B tok iter 106860 | train 2.665 | val 2.731 | lr 1.73e-04 | 102,472 tok/s | 54.64 GB | 14.006B tok iter 106880 | train 2.706 | val 2.698 | lr 1.73e-04 | 102,473 tok/s | 54.64 GB | 14.009B tok iter 106900 | train 2.672 | val 2.686 | lr 1.73e-04 | 102,474 tok/s | 54.64 GB | 14.012B tok iter 106920 | train 2.695 | val 2.674 | lr 1.73e-04 | 102,475 tok/s | 54.64 GB | 14.014B tok iter 106940 | train 2.645 | val 2.694 | lr 1.73e-04 | 102,477 tok/s | 54.64 GB | 14.017B tok iter 106960 | train 2.503 | val 2.715 | lr 1.73e-04 | 102,478 tok/s | 54.64 GB | 14.020B tok iter 106980 | train 2.623 | val 2.686 | lr 1.73e-04 | 102,479 tok/s | 54.64 GB | 14.022B tok checkpoint @ iter 106991 -> checkpoints/500m_latest.pt iter 107000 | train 2.654 | val 2.728 | lr 1.73e-04 | 102,470 tok/s | 54.64 GB | 14.025B tok checkpoint @ iter 107000 -> checkpoints/500m_latest.pt iter 107020 | train 2.727 | val 2.700 | lr 1.72e-04 | 102,452 tok/s | 54.64 GB | 14.027B tok iter 107040 | train 2.745 | val 2.690 | lr 1.72e-04 | 102,454 tok/s | 54.64 GB | 14.030B tok iter 107060 | train 2.547 | val 2.707 | lr 1.72e-04 | 102,454 tok/s | 54.64 GB | 14.033B tok iter 107080 | train 2.572 | val 2.696 | lr 1.72e-04 | 102,456 tok/s | 54.64 GB | 14.035B tok iter 107100 | train 2.681 | val 2.743 | lr 1.72e-04 | 102,457 tok/s | 54.64 GB | 14.038B tok iter 107120 | train 2.783 | val 2.753 | lr 1.72e-04 | 102,459 tok/s | 54.64 GB | 14.041B tok iter 107140 | train 2.751 | val 2.737 | lr 1.72e-04 | 102,459 tok/s | 54.64 GB | 14.043B tok iter 107160 | train 2.622 | val 2.722 | lr 1.72e-04 | 102,461 tok/s | 54.64 GB | 14.046B tok iter 107180 | train 2.680 | val 2.700 | lr 1.72e-04 | 102,462 tok/s | 54.64 GB | 14.048B tok iter 107200 | train 2.597 | val 2.716 | lr 1.72e-04 | 102,464 tok/s | 54.64 GB | 14.051B tok iter 107220 | train 2.690 | val 2.696 | lr 1.72e-04 | 102,464 tok/s | 54.64 GB | 14.054B tok iter 107240 | train 2.666 | val 2.711 | lr 1.71e-04 | 102,466 tok/s | 54.64 GB | 14.056B tok iter 107260 | train 2.853 | val 2.722 | lr 1.71e-04 | 102,467 tok/s | 54.64 GB | 14.059B tok iter 107280 | train 2.740 | val 2.676 | lr 1.71e-04 | 102,469 tok/s | 54.64 GB | 14.062B tok iter 107300 | train 2.731 | val 2.708 | lr 1.71e-04 | 102,469 tok/s | 54.64 GB | 14.064B tok iter 107320 | train 2.727 | val 2.712 | lr 1.71e-04 | 102,470 tok/s | 54.64 GB | 14.067B tok iter 107340 | train 2.797 | val 2.676 | lr 1.71e-04 | 102,472 tok/s | 54.64 GB | 14.069B tok iter 107360 | train 2.808 | val 2.721 | lr 1.71e-04 | 102,473 tok/s | 54.64 GB | 14.072B tok iter 107380 | train 2.790 | val 2.700 | lr 1.71e-04 | 102,474 tok/s | 54.64 GB | 14.075B tok iter 107400 | train 2.536 | val 2.702 | lr 1.71e-04 | 102,475 tok/s | 54.64 GB | 14.077B tok iter 107420 | train 2.828 | val 2.681 | lr 1.71e-04 | 102,477 tok/s | 54.64 GB | 14.080B tok iter 107440 | train 2.619 | val 2.729 | lr 1.71e-04 | 102,478 tok/s | 54.64 GB | 14.083B tok iter 107460 | train 2.633 | val 2.754 | lr 1.70e-04 | 102,478 tok/s | 54.64 GB | 14.085B tok iter 107480 | train 2.714 | val 2.694 | lr 1.70e-04 | 102,480 tok/s | 54.64 GB | 14.088B tok checkpoint @ iter 107491 -> checkpoints/500m_latest.pt iter 107500 | train 2.652 | val 2.751 | lr 1.70e-04 | 102,472 tok/s | 54.64 GB | 14.090B tok checkpoint @ iter 107500 -> checkpoints/500m_latest.pt iter 107520 | train 2.614 | val 2.692 | lr 1.70e-04 | 102,454 tok/s | 54.64 GB | 14.093B tok iter 107540 | train 2.686 | val 2.642 | lr 1.70e-04 | 102,454 tok/s | 54.64 GB | 14.096B tok iter 107560 | train 2.602 | val 2.761 | lr 1.70e-04 | 102,456 tok/s | 54.64 GB | 14.098B tok iter 107580 | train 2.827 | val 2.708 | lr 1.70e-04 | 102,457 tok/s | 54.64 GB | 14.101B tok iter 107600 | train 2.680 | val 2.715 | lr 1.70e-04 | 102,459 tok/s | 54.64 GB | 14.103B tok iter 107620 | train 2.711 | val 2.703 | lr 1.70e-04 | 102,459 tok/s | 54.64 GB | 14.106B tok iter 107640 | train 2.707 | val 2.677 | lr 1.70e-04 | 102,460 tok/s | 54.64 GB | 14.109B tok iter 107660 | train 2.663 | val 2.734 | lr 1.70e-04 | 102,462 tok/s | 54.64 GB | 14.111B tok iter 107680 | train 2.633 | val 2.708 | lr 1.69e-04 | 102,463 tok/s | 54.64 GB | 14.114B tok iter 107700 | train 2.731 | val 2.706 | lr 1.69e-04 | 102,464 tok/s | 54.64 GB | 14.117B tok iter 107720 | train 2.664 | val 2.693 | lr 1.69e-04 | 102,465 tok/s | 54.64 GB | 14.119B tok iter 107740 | train 2.775 | val 2.683 | lr 1.69e-04 | 102,467 tok/s | 54.64 GB | 14.122B tok iter 107760 | train 2.592 | val 2.688 | lr 1.69e-04 | 102,468 tok/s | 54.64 GB | 14.124B tok iter 107780 | train 2.879 | val 2.716 | lr 1.69e-04 | 102,469 tok/s | 54.64 GB | 14.127B tok iter 107800 | train 2.573 | val 2.727 | lr 1.69e-04 | 102,470 tok/s | 54.64 GB | 14.130B tok iter 107820 | train 2.659 | val 2.677 | lr 1.69e-04 | 102,472 tok/s | 54.64 GB | 14.132B tok iter 107840 | train 2.694 | val 2.699 | lr 1.69e-04 | 102,473 tok/s | 54.64 GB | 14.135B tok iter 107860 | train 2.721 | val 2.684 | lr 1.69e-04 | 102,474 tok/s | 54.64 GB | 14.138B tok iter 107880 | train 2.561 | val 2.697 | lr 1.69e-04 | 102,475 tok/s | 54.64 GB | 14.140B tok iter 107900 | train 2.662 | val 2.704 | lr 1.68e-04 | 102,476 tok/s | 54.64 GB | 14.143B tok iter 107920 | train 2.785 | val 2.673 | lr 1.68e-04 | 102,478 tok/s | 54.64 GB | 14.145B tok iter 107940 | train 2.649 | val 2.683 | lr 1.68e-04 | 102,478 tok/s | 54.64 GB | 14.148B tok iter 107960 | train 2.931 | val 2.703 | lr 1.68e-04 | 102,480 tok/s | 54.64 GB | 14.151B tok iter 107980 | train 2.557 | val 2.715 | lr 1.68e-04 | 102,481 tok/s | 54.64 GB | 14.153B tok checkpoint @ iter 107991 -> checkpoints/500m_latest.pt iter 108000 | train 2.596 | val 2.717 | lr 1.68e-04 | 102,473 tok/s | 54.64 GB | 14.156B tok checkpoint @ iter 108000 -> checkpoints/500m_latest.pt iter 108020 | train 2.677 | val 2.737 | lr 1.68e-04 | 102,454 tok/s | 54.64 GB | 14.159B tok iter 108040 | train 2.876 | val 2.712 | lr 1.68e-04 | 102,456 tok/s | 54.64 GB | 14.161B tok iter 108060 | train 2.667 | val 2.742 | lr 1.68e-04 | 102,457 tok/s | 54.64 GB | 14.164B tok iter 108080 | train 2.705 | val 2.714 | lr 1.68e-04 | 102,459 tok/s | 54.64 GB | 14.166B tok iter 108100 | train 2.780 | val 2.700 | lr 1.68e-04 | 102,459 tok/s | 54.64 GB | 14.169B tok iter 108120 | train 2.796 | val 2.693 | lr 1.67e-04 | 102,461 tok/s | 54.64 GB | 14.172B tok iter 108140 | train 2.812 | val 2.702 | lr 1.67e-04 | 102,462 tok/s | 54.64 GB | 14.174B tok iter 108160 | train 2.753 | val 2.659 | lr 1.67e-04 | 102,463 tok/s | 54.64 GB | 14.177B tok iter 108180 | train 2.718 | val 2.673 | lr 1.67e-04 | 102,464 tok/s | 54.64 GB | 14.180B tok iter 108200 | train 2.674 | val 2.725 | lr 1.67e-04 | 102,465 tok/s | 54.64 GB | 14.182B tok iter 108220 | train 2.568 | val 2.704 | lr 1.67e-04 | 102,467 tok/s | 54.64 GB | 14.185B tok iter 108240 | train 2.591 | val 2.725 | lr 1.67e-04 | 102,468 tok/s | 54.64 GB | 14.187B tok iter 108260 | train 2.778 | val 2.693 | lr 1.67e-04 | 102,469 tok/s | 54.64 GB | 14.190B tok iter 108280 | train 2.661 | val 2.706 | lr 1.67e-04 | 102,470 tok/s | 54.64 GB | 14.193B tok iter 108300 | train 2.669 | val 2.647 | lr 1.67e-04 | 102,472 tok/s | 54.64 GB | 14.195B tok iter 108320 | train 2.850 | val 2.668 | lr 1.67e-04 | 102,473 tok/s | 54.64 GB | 14.198B tok iter 108340 | train 2.505 | val 2.657 | lr 1.66e-04 | 102,473 tok/s | 54.64 GB | 14.200B tok iter 108360 | train 2.651 | val 2.707 | lr 1.66e-04 | 102,475 tok/s | 54.64 GB | 14.203B tok iter 108380 | train 2.809 | val 2.698 | lr 1.66e-04 | 102,476 tok/s | 54.64 GB | 14.206B tok iter 108400 | train 2.637 | val 2.734 | lr 1.66e-04 | 102,478 tok/s | 54.64 GB | 14.208B tok iter 108420 | train 2.685 | val 2.701 | lr 1.66e-04 | 102,478 tok/s | 54.64 GB | 14.211B tok iter 108440 | train 2.754 | val 2.687 | lr 1.66e-04 | 102,480 tok/s | 54.64 GB | 14.214B tok iter 108460 | train 2.663 | val 2.629 | lr 1.66e-04 | 102,481 tok/s | 54.64 GB | 14.216B tok iter 108480 | train 2.792 | val 2.706 | lr 1.66e-04 | 102,483 tok/s | 54.64 GB | 14.219B tok checkpoint @ iter 108491 -> checkpoints/500m_latest.pt iter 108500 | train 2.707 | val 2.705 | lr 1.66e-04 | 102,474 tok/s | 54.64 GB | 14.221B tok checkpoint @ iter 108500 -> checkpoints/500m_latest.pt iter 108520 | train 2.693 | val 2.668 | lr 1.66e-04 | 102,456 tok/s | 54.64 GB | 14.224B tok iter 108540 | train 2.684 | val 2.686 | lr 1.66e-04 | 102,458 tok/s | 54.64 GB | 14.227B tok iter 108560 | train 2.618 | val 2.687 | lr 1.65e-04 | 102,459 tok/s | 54.64 GB | 14.229B tok iter 108580 | train 2.712 | val 2.717 | lr 1.65e-04 | 102,460 tok/s | 54.64 GB | 14.232B tok iter 108600 | train 2.641 | val 2.671 | lr 1.65e-04 | 102,461 tok/s | 54.64 GB | 14.235B tok iter 108620 | train 2.655 | val 2.720 | lr 1.65e-04 | 102,463 tok/s | 54.64 GB | 14.237B tok iter 108640 | train 2.599 | val 2.689 | lr 1.65e-04 | 102,464 tok/s | 54.64 GB | 14.240B tok iter 108660 | train 2.628 | val 2.689 | lr 1.65e-04 | 102,464 tok/s | 54.64 GB | 14.242B tok iter 108680 | train 2.733 | val 2.682 | lr 1.65e-04 | 102,466 tok/s | 54.64 GB | 14.245B tok iter 108700 | train 2.775 | val 2.704 | lr 1.65e-04 | 102,467 tok/s | 54.64 GB | 14.248B tok iter 108720 | train 2.798 | val 2.695 | lr 1.65e-04 | 102,469 tok/s | 54.64 GB | 14.250B tok iter 108740 | train 2.829 | val 2.678 | lr 1.65e-04 | 102,469 tok/s | 54.64 GB | 14.253B tok iter 108760 | train 2.708 | val 2.646 | lr 1.65e-04 | 102,471 tok/s | 54.64 GB | 14.256B tok iter 108780 | train 2.691 | val 2.705 | lr 1.65e-04 | 102,472 tok/s | 54.64 GB | 14.258B tok iter 108800 | train 2.692 | val 2.729 | lr 1.64e-04 | 102,474 tok/s | 54.64 GB | 14.261B tok iter 108820 | train 2.744 | val 2.718 | lr 1.64e-04 | 102,474 tok/s | 54.64 GB | 14.263B tok iter 108840 | train 2.699 | val 2.719 | lr 1.64e-04 | 102,476 tok/s | 54.64 GB | 14.266B tok iter 108860 | train 2.823 | val 2.693 | lr 1.64e-04 | 102,477 tok/s | 54.64 GB | 14.269B tok iter 108880 | train 2.707 | val 2.705 | lr 1.64e-04 | 102,478 tok/s | 54.64 GB | 14.271B tok iter 108900 | train 2.638 | val 2.660 | lr 1.64e-04 | 102,479 tok/s | 54.64 GB | 14.274B tok iter 108920 | train 2.695 | val 2.691 | lr 1.64e-04 | 102,480 tok/s | 54.64 GB | 14.276B tok iter 108940 | train 2.720 | val 2.693 | lr 1.64e-04 | 102,482 tok/s | 54.64 GB | 14.279B tok iter 108960 | train 2.754 | val 2.698 | lr 1.64e-04 | 102,483 tok/s | 54.64 GB | 14.282B tok iter 108980 | train 2.667 | val 2.694 | lr 1.64e-04 | 102,484 tok/s | 54.64 GB | 14.284B tok checkpoint @ iter 108991 -> checkpoints/500m_latest.pt iter 109000 | train 2.663 | val 2.702 | lr 1.64e-04 | 102,476 tok/s | 54.64 GB | 14.287B tok checkpoint @ iter 109000 -> checkpoints/500m_latest.pt iter 109020 | train 2.747 | val 2.718 | lr 1.63e-04 | 102,459 tok/s | 54.64 GB | 14.290B tok iter 109040 | train 2.723 | val 2.722 | lr 1.63e-04 | 102,460 tok/s | 54.64 GB | 14.292B tok iter 109060 | train 2.664 | val 2.691 | lr 1.63e-04 | 102,460 tok/s | 54.64 GB | 14.295B tok iter 109080 | train 2.624 | val 2.696 | lr 1.63e-04 | 102,462 tok/s | 54.64 GB | 14.297B tok iter 109100 | train 2.636 | val 2.760 | lr 1.63e-04 | 102,463 tok/s | 54.64 GB | 14.300B tok iter 109120 | train 2.717 | val 2.697 | lr 1.63e-04 | 102,465 tok/s | 54.64 GB | 14.303B tok iter 109140 | train 2.696 | val 2.716 | lr 1.63e-04 | 102,465 tok/s | 54.64 GB | 14.305B tok iter 109160 | train 2.710 | val 2.776 | lr 1.63e-04 | 102,467 tok/s | 54.64 GB | 14.308B tok iter 109180 | train 2.674 | val 2.691 | lr 1.63e-04 | 102,468 tok/s | 54.64 GB | 14.311B tok iter 109200 | train 2.685 | val 2.728 | lr 1.63e-04 | 102,470 tok/s | 54.64 GB | 14.313B tok iter 109220 | train 2.603 | val 2.703 | lr 1.63e-04 | 102,470 tok/s | 54.64 GB | 14.316B tok iter 109240 | train 2.643 | val 2.690 | lr 1.62e-04 | 102,471 tok/s | 54.64 GB | 14.318B tok iter 109260 | train 2.703 | val 2.708 | lr 1.62e-04 | 102,473 tok/s | 54.64 GB | 14.321B tok iter 109280 | train 2.566 | val 2.704 | lr 1.62e-04 | 102,474 tok/s | 54.64 GB | 14.324B tok iter 109300 | train 2.681 | val 2.704 | lr 1.62e-04 | 102,475 tok/s | 54.64 GB | 14.326B tok iter 109320 | train 2.644 | val 2.707 | lr 1.62e-04 | 102,476 tok/s | 54.64 GB | 14.329B tok iter 109340 | train 2.740 | val 2.690 | lr 1.62e-04 | 102,478 tok/s | 54.64 GB | 14.332B tok iter 109360 | train 2.801 | val 2.705 | lr 1.62e-04 | 102,479 tok/s | 54.64 GB | 14.334B tok iter 109380 | train 2.889 | val 2.690 | lr 1.62e-04 | 102,480 tok/s | 54.64 GB | 14.337B tok iter 109400 | train 2.835 | val 2.741 | lr 1.62e-04 | 102,481 tok/s | 54.64 GB | 14.339B tok iter 109420 | train 2.639 | val 2.731 | lr 1.62e-04 | 102,482 tok/s | 54.64 GB | 14.342B tok iter 109440 | train 2.513 | val 2.691 | lr 1.62e-04 | 102,484 tok/s | 54.64 GB | 14.345B tok iter 109460 | train 2.557 | val 2.695 | lr 1.62e-04 | 102,484 tok/s | 54.64 GB | 14.347B tok iter 109480 | train 2.577 | val 2.678 | lr 1.61e-04 | 102,486 tok/s | 54.64 GB | 14.350B tok checkpoint @ iter 109492 -> checkpoints/500m_latest.pt iter 109500 | train 2.675 | val 2.691 | lr 1.61e-04 | 102,478 tok/s | 54.64 GB | 14.353B tok checkpoint @ iter 109500 -> checkpoints/500m_latest.pt iter 109520 | train 2.669 | val 2.701 | lr 1.61e-04 | 102,460 tok/s | 54.64 GB | 14.355B tok iter 109540 | train 2.575 | val 2.696 | lr 1.61e-04 | 102,460 tok/s | 54.64 GB | 14.358B tok iter 109560 | train 2.730 | val 2.728 | lr 1.61e-04 | 102,462 tok/s | 54.64 GB | 14.360B tok iter 109580 | train 2.640 | val 2.729 | lr 1.61e-04 | 102,463 tok/s | 54.64 GB | 14.363B tok iter 109600 | train 2.741 | val 2.719 | lr 1.61e-04 | 102,465 tok/s | 54.64 GB | 14.366B tok iter 109620 | train 2.683 | val 2.678 | lr 1.61e-04 | 102,465 tok/s | 54.64 GB | 14.368B tok iter 109640 | train 2.668 | val 2.738 | lr 1.61e-04 | 102,467 tok/s | 54.64 GB | 14.371B tok iter 109660 | train 2.730 | val 2.693 | lr 1.61e-04 | 102,468 tok/s | 54.64 GB | 14.373B tok iter 109680 | train 2.574 | val 2.717 | lr 1.61e-04 | 102,470 tok/s | 54.64 GB | 14.376B tok iter 109700 | train 2.719 | val 2.709 | lr 1.60e-04 | 102,470 tok/s | 54.64 GB | 14.379B tok iter 109720 | train 2.725 | val 2.670 | lr 1.60e-04 | 102,472 tok/s | 54.64 GB | 14.381B tok iter 109740 | train 2.617 | val 2.774 | lr 1.60e-04 | 102,473 tok/s | 54.64 GB | 14.384B tok iter 109760 | train 2.751 | val 2.704 | lr 1.60e-04 | 102,474 tok/s | 54.64 GB | 14.387B tok iter 109780 | train 2.631 | val 2.651 | lr 1.60e-04 | 102,475 tok/s | 54.64 GB | 14.389B tok iter 109800 | train 2.729 | val 2.667 | lr 1.60e-04 | 102,476 tok/s | 54.64 GB | 14.392B tok iter 109820 | train 2.615 | val 2.706 | lr 1.60e-04 | 102,478 tok/s | 54.64 GB | 14.394B tok iter 109840 | train 2.684 | val 2.754 | lr 1.60e-04 | 102,479 tok/s | 54.64 GB | 14.397B tok iter 109860 | train 2.683 | val 2.710 | lr 1.60e-04 | 102,479 tok/s | 54.64 GB | 14.400B tok iter 109880 | train 2.678 | val 2.717 | lr 1.60e-04 | 102,481 tok/s | 54.64 GB | 14.402B tok iter 109900 | train 2.796 | val 2.750 | lr 1.60e-04 | 102,482 tok/s | 54.64 GB | 14.405B tok iter 109920 | train 2.698 | val 2.685 | lr 1.60e-04 | 102,484 tok/s | 54.64 GB | 14.408B tok iter 109940 | train 2.601 | val 2.708 | lr 1.59e-04 | 102,484 tok/s | 54.64 GB | 14.410B tok iter 109960 | train 2.720 | val 2.748 | lr 1.59e-04 | 102,486 tok/s | 54.64 GB | 14.413B tok iter 109980 | train 2.829 | val 2.656 | lr 1.59e-04 | 102,487 tok/s | 54.64 GB | 14.415B tok checkpoint @ iter 109991 -> checkpoints/500m_latest.pt iter 110000 | train 2.565 | val 2.691 | lr 1.59e-04 | 102,479 tok/s | 54.64 GB | 14.418B tok checkpoint @ iter 110000 -> checkpoints/500m_latest.pt iter 110020 | train 2.687 | val 2.703 | lr 1.59e-04 | 102,461 tok/s | 54.64 GB | 14.421B tok iter 110040 | train 2.680 | val 2.696 | lr 1.59e-04 | 102,462 tok/s | 54.64 GB | 14.423B tok iter 110060 | train 2.680 | val 2.717 | lr 1.59e-04 | 102,464 tok/s | 54.64 GB | 14.426B tok iter 110080 | train 2.555 | val 2.696 | lr 1.59e-04 | 102,465 tok/s | 54.64 GB | 14.429B tok iter 110100 | train 2.831 | val 2.723 | lr 1.59e-04 | 102,466 tok/s | 54.64 GB | 14.431B tok iter 110120 | train 2.743 | val 2.685 | lr 1.59e-04 | 102,467 tok/s | 54.64 GB | 14.434B tok iter 110140 | train 2.694 | val 2.688 | lr 1.59e-04 | 102,469 tok/s | 54.64 GB | 14.436B tok iter 110160 | train 2.626 | val 2.704 | lr 1.58e-04 | 102,470 tok/s | 54.64 GB | 14.439B tok iter 110180 | train 2.801 | val 2.690 | lr 1.58e-04 | 102,470 tok/s | 54.64 GB | 14.442B tok iter 110200 | train 2.758 | val 2.662 | lr 1.58e-04 | 102,472 tok/s | 54.64 GB | 14.444B tok iter 110220 | train 2.912 | val 2.626 | lr 1.58e-04 | 102,473 tok/s | 54.64 GB | 14.447B tok iter 110240 | train 2.655 | val 2.708 | lr 1.58e-04 | 102,475 tok/s | 54.64 GB | 14.450B tok iter 110260 | train 2.755 | val 2.701 | lr 1.58e-04 | 102,475 tok/s | 54.64 GB | 14.452B tok iter 110280 | train 2.716 | val 2.685 | lr 1.58e-04 | 102,477 tok/s | 54.64 GB | 14.455B tok iter 110300 | train 2.694 | val 2.690 | lr 1.58e-04 | 102,478 tok/s | 54.64 GB | 14.457B tok iter 110320 | train 2.755 | val 2.691 | lr 1.58e-04 | 102,480 tok/s | 54.64 GB | 14.460B tok iter 110340 | train 2.621 | val 2.679 | lr 1.58e-04 | 102,480 tok/s | 54.64 GB | 14.463B tok iter 110360 | train 2.555 | val 2.674 | lr 1.58e-04 | 102,481 tok/s | 54.64 GB | 14.465B tok iter 110380 | train 2.578 | val 2.681 | lr 1.58e-04 | 102,483 tok/s | 54.64 GB | 14.468B tok iter 110400 | train 2.788 | val 2.658 | lr 1.57e-04 | 102,484 tok/s | 54.64 GB | 14.470B tok iter 110420 | train 2.749 | val 2.676 | lr 1.57e-04 | 102,485 tok/s | 54.64 GB | 14.473B tok iter 110440 | train 2.748 | val 2.675 | lr 1.57e-04 | 102,486 tok/s | 54.64 GB | 14.476B tok iter 110460 | train 2.707 | val 2.676 | lr 1.57e-04 | 102,487 tok/s | 54.64 GB | 14.478B tok iter 110480 | train 2.659 | val 2.712 | lr 1.57e-04 | 102,489 tok/s | 54.64 GB | 14.481B tok checkpoint @ iter 110491 -> checkpoints/500m_latest.pt iter 110500 | train 2.676 | val 2.693 | lr 1.57e-04 | 102,480 tok/s | 54.64 GB | 14.484B tok checkpoint @ iter 110500 -> checkpoints/500m_latest.pt iter 110520 | train 2.721 | val 2.688 | lr 1.57e-04 | 102,462 tok/s | 54.64 GB | 14.486B tok iter 110540 | train 2.588 | val 2.721 | lr 1.57e-04 | 102,464 tok/s | 54.64 GB | 14.489B tok iter 110560 | train 2.575 | val 2.693 | lr 1.57e-04 | 102,465 tok/s | 54.64 GB | 14.491B tok iter 110580 | train 2.694 | val 2.702 | lr 1.57e-04 | 102,466 tok/s | 54.64 GB | 14.494B tok iter 110600 | train 2.686 | val 2.710 | lr 1.57e-04 | 102,467 tok/s | 54.64 GB | 14.497B tok iter 110620 | train 2.735 | val 2.674 | lr 1.56e-04 | 102,469 tok/s | 54.64 GB | 14.499B tok iter 110640 | train 2.674 | val 2.676 | lr 1.56e-04 | 102,470 tok/s | 54.64 GB | 14.502B tok iter 110660 | train 2.563 | val 2.641 | lr 1.56e-04 | 102,470 tok/s | 54.64 GB | 14.505B tok iter 110680 | train 2.687 | val 2.705 | lr 1.56e-04 | 102,472 tok/s | 54.64 GB | 14.507B tok iter 110700 | train 2.727 | val 2.713 | lr 1.56e-04 | 102,473 tok/s | 54.64 GB | 14.510B tok iter 110720 | train 2.651 | val 2.738 | lr 1.56e-04 | 102,475 tok/s | 54.64 GB | 14.512B tok iter 110740 | train 2.641 | val 2.695 | lr 1.56e-04 | 102,475 tok/s | 54.64 GB | 14.515B tok iter 110760 | train 2.720 | val 2.669 | lr 1.56e-04 | 102,476 tok/s | 54.64 GB | 14.518B tok iter 110780 | train 2.709 | val 2.761 | lr 1.56e-04 | 102,478 tok/s | 54.64 GB | 14.520B tok iter 110800 | train 2.808 | val 2.746 | lr 1.56e-04 | 102,479 tok/s | 54.64 GB | 14.523B tok iter 110820 | train 2.707 | val 2.640 | lr 1.56e-04 | 102,480 tok/s | 54.64 GB | 14.526B tok iter 110840 | train 2.690 | val 2.683 | lr 1.56e-04 | 102,481 tok/s | 54.64 GB | 14.528B tok iter 110860 | train 2.767 | val 2.661 | lr 1.55e-04 | 102,483 tok/s | 54.64 GB | 14.531B tok iter 110880 | train 2.644 | val 2.701 | lr 1.55e-04 | 102,484 tok/s | 54.64 GB | 14.533B tok iter 110900 | train 2.684 | val 2.708 | lr 1.55e-04 | 102,484 tok/s | 54.64 GB | 14.536B tok iter 110920 | train 2.759 | val 2.702 | lr 1.55e-04 | 102,486 tok/s | 54.64 GB | 14.539B tok iter 110940 | train 2.853 | val 2.727 | lr 1.55e-04 | 102,487 tok/s | 54.64 GB | 14.541B tok iter 110960 | train 2.572 | val 2.715 | lr 1.55e-04 | 102,489 tok/s | 54.64 GB | 14.544B tok iter 110980 | train 2.733 | val 2.697 | lr 1.55e-04 | 102,489 tok/s | 54.64 GB | 14.547B tok checkpoint @ iter 110991 -> checkpoints/500m_latest.pt iter 111000 | train 2.789 | val 2.641 | lr 1.55e-04 | 102,481 tok/s | 54.64 GB | 14.549B tok checkpoint @ iter 111000 -> checkpoints/500m_latest.pt iter 111020 | train 2.785 | val 2.681 | lr 1.55e-04 | 102,464 tok/s | 54.64 GB | 14.552B tok iter 111040 | train 2.734 | val 2.707 | lr 1.55e-04 | 102,465 tok/s | 54.64 GB | 14.554B tok iter 111060 | train 2.639 | val 2.716 | lr 1.55e-04 | 102,465 tok/s | 54.64 GB | 14.557B tok iter 111080 | train 2.757 | val 2.690 | lr 1.55e-04 | 102,467 tok/s | 54.64 GB | 14.560B tok iter 111100 | train 2.807 | val 2.734 | lr 1.54e-04 | 102,468 tok/s | 54.64 GB | 14.562B tok iter 111120 | train 2.560 | val 2.693 | lr 1.54e-04 | 102,470 tok/s | 54.64 GB | 14.565B tok iter 111140 | train 2.546 | val 2.746 | lr 1.54e-04 | 102,470 tok/s | 54.64 GB | 14.567B tok iter 111160 | train 2.573 | val 2.716 | lr 1.54e-04 | 102,472 tok/s | 54.64 GB | 14.570B tok iter 111180 | train 2.718 | val 2.704 | lr 1.54e-04 | 102,473 tok/s | 54.64 GB | 14.573B tok iter 111200 | train 2.581 | val 2.707 | lr 1.54e-04 | 102,475 tok/s | 54.64 GB | 14.575B tok iter 111220 | train 2.737 | val 2.710 | lr 1.54e-04 | 102,475 tok/s | 54.64 GB | 14.578B tok iter 111240 | train 2.761 | val 2.677 | lr 1.54e-04 | 102,476 tok/s | 54.64 GB | 14.581B tok iter 111260 | train 2.757 | val 2.717 | lr 1.54e-04 | 102,478 tok/s | 54.64 GB | 14.583B tok iter 111280 | train 2.651 | val 2.678 | lr 1.54e-04 | 102,479 tok/s | 54.64 GB | 14.586B tok iter 111300 | train 2.784 | val 2.690 | lr 1.54e-04 | 102,480 tok/s | 54.64 GB | 14.588B tok iter 111320 | train 2.669 | val 2.694 | lr 1.53e-04 | 102,481 tok/s | 54.64 GB | 14.591B tok iter 111340 | train 2.577 | val 2.680 | lr 1.53e-04 | 102,482 tok/s | 54.64 GB | 14.594B tok iter 111360 | train 2.774 | val 2.687 | lr 1.53e-04 | 102,484 tok/s | 54.64 GB | 14.596B tok iter 111380 | train 2.724 | val 2.696 | lr 1.53e-04 | 102,484 tok/s | 54.64 GB | 14.599B tok iter 111400 | train 2.656 | val 2.695 | lr 1.53e-04 | 102,486 tok/s | 54.64 GB | 14.602B tok iter 111420 | train 2.588 | val 2.668 | lr 1.53e-04 | 102,487 tok/s | 54.64 GB | 14.604B tok iter 111440 | train 2.642 | val 2.662 | lr 1.53e-04 | 102,489 tok/s | 54.64 GB | 14.607B tok iter 111460 | train 2.548 | val 2.668 | lr 1.53e-04 | 102,489 tok/s | 54.64 GB | 14.609B tok iter 111480 | train 2.696 | val 2.692 | lr 1.53e-04 | 102,490 tok/s | 54.64 GB | 14.612B tok checkpoint @ iter 111491 -> checkpoints/500m_latest.pt iter 111500 | train 2.821 | val 2.681 | lr 1.53e-04 | 102,482 tok/s | 54.64 GB | 14.615B tok checkpoint @ iter 111500 -> checkpoints/500m_latest.pt iter 111520 | train 2.543 | val 2.723 | lr 1.53e-04 | 102,466 tok/s | 54.64 GB | 14.617B tok iter 111540 | train 2.639 | val 2.689 | lr 1.53e-04 | 102,466 tok/s | 54.64 GB | 14.620B tok iter 111560 | train 2.739 | val 2.707 | lr 1.52e-04 | 102,467 tok/s | 54.64 GB | 14.623B tok iter 111580 | train 2.751 | val 2.717 | lr 1.52e-04 | 102,469 tok/s | 54.64 GB | 14.625B tok iter 111600 | train 2.793 | val 2.659 | lr 1.52e-04 | 102,470 tok/s | 54.64 GB | 14.628B tok iter 111620 | train 2.655 | val 2.619 | lr 1.52e-04 | 102,471 tok/s | 54.64 GB | 14.630B tok iter 111640 | train 2.711 | val 2.704 | lr 1.52e-04 | 102,472 tok/s | 54.64 GB | 14.633B tok iter 111660 | train 2.742 | val 2.729 | lr 1.52e-04 | 102,473 tok/s | 54.64 GB | 14.636B tok iter 111680 | train 2.633 | val 2.675 | lr 1.52e-04 | 102,475 tok/s | 54.64 GB | 14.638B tok iter 111700 | train 2.656 | val 2.690 | lr 1.52e-04 | 102,475 tok/s | 54.64 GB | 14.641B tok iter 111720 | train 2.670 | val 2.728 | lr 1.52e-04 | 102,477 tok/s | 54.64 GB | 14.643B tok iter 111740 | train 2.682 | val 2.681 | lr 1.52e-04 | 102,478 tok/s | 54.64 GB | 14.646B tok iter 111760 | train 2.622 | val 2.705 | lr 1.52e-04 | 102,480 tok/s | 54.64 GB | 14.649B tok iter 111780 | train 2.691 | val 2.668 | lr 1.52e-04 | 102,480 tok/s | 54.64 GB | 14.651B tok iter 111800 | train 2.667 | val 2.701 | lr 1.51e-04 | 102,481 tok/s | 54.64 GB | 14.654B tok iter 111820 | train 2.741 | val 2.693 | lr 1.51e-04 | 102,483 tok/s | 54.64 GB | 14.657B tok iter 111840 | train 2.542 | val 2.673 | lr 1.51e-04 | 102,484 tok/s | 54.64 GB | 14.659B tok iter 111860 | train 2.690 | val 2.719 | lr 1.51e-04 | 102,485 tok/s | 54.64 GB | 14.662B tok iter 111880 | train 2.646 | val 2.697 | lr 1.51e-04 | 102,486 tok/s | 54.64 GB | 14.664B tok iter 111900 | train 2.671 | val 2.674 | lr 1.51e-04 | 102,487 tok/s | 54.64 GB | 14.667B tok iter 111920 | train 2.773 | val 2.716 | lr 1.51e-04 | 102,489 tok/s | 54.64 GB | 14.670B tok iter 111940 | train 2.649 | val 2.703 | lr 1.51e-04 | 102,489 tok/s | 54.64 GB | 14.672B tok iter 111960 | train 2.734 | val 2.720 | lr 1.51e-04 | 102,491 tok/s | 54.64 GB | 14.675B tok iter 111980 | train 2.704 | val 2.671 | lr 1.51e-04 | 102,492 tok/s | 54.64 GB | 14.678B tok checkpoint @ iter 111992 -> checkpoints/500m_latest.pt iter 112000 | train 2.638 | val 2.688 | lr 1.51e-04 | 102,484 tok/s | 54.64 GB | 14.680B tok checkpoint @ iter 112000 -> checkpoints/500m_latest.pt iter 112020 | train 2.602 | val 2.636 | lr 1.51e-04 | 102,466 tok/s | 54.64 GB | 14.683B tok iter 112040 | train 2.638 | val 2.689 | lr 1.50e-04 | 102,467 tok/s | 54.64 GB | 14.685B tok iter 112060 | train 2.717 | val 2.691 | lr 1.50e-04 | 102,468 tok/s | 54.64 GB | 14.688B tok iter 112080 | train 2.673 | val 2.668 | lr 1.50e-04 | 102,470 tok/s | 54.64 GB | 14.691B tok iter 112100 | train 2.684 | val 2.691 | lr 1.50e-04 | 102,470 tok/s | 54.64 GB | 14.693B tok iter 112120 | train 2.695 | val 2.675 | lr 1.50e-04 | 102,472 tok/s | 54.64 GB | 14.696B tok iter 112140 | train 2.632 | val 2.663 | lr 1.50e-04 | 102,473 tok/s | 54.64 GB | 14.699B tok iter 112160 | train 2.671 | val 2.677 | lr 1.50e-04 | 102,475 tok/s | 54.64 GB | 14.701B tok iter 112180 | train 2.753 | val 2.682 | lr 1.50e-04 | 102,475 tok/s | 54.64 GB | 14.704B tok iter 112200 | train 2.695 | val 2.692 | lr 1.50e-04 | 102,476 tok/s | 54.64 GB | 14.706B tok iter 112220 | train 2.754 | val 2.693 | lr 1.50e-04 | 102,478 tok/s | 54.64 GB | 14.709B tok iter 112240 | train 2.661 | val 2.691 | lr 1.50e-04 | 102,479 tok/s | 54.64 GB | 14.712B tok iter 112260 | train 2.739 | val 2.695 | lr 1.50e-04 | 102,480 tok/s | 54.64 GB | 14.714B tok iter 112280 | train 2.692 | val 2.701 | lr 1.49e-04 | 102,481 tok/s | 54.64 GB | 14.717B tok iter 112300 | train 2.629 | val 2.673 | lr 1.49e-04 | 102,482 tok/s | 54.64 GB | 14.720B tok iter 112320 | train 2.615 | val 2.700 | lr 1.49e-04 | 102,484 tok/s | 54.64 GB | 14.722B tok iter 112340 | train 2.789 | val 2.665 | lr 1.49e-04 | 102,484 tok/s | 54.64 GB | 14.725B tok iter 112360 | train 2.666 | val 2.680 | lr 1.49e-04 | 102,486 tok/s | 54.64 GB | 14.727B tok iter 112380 | train 2.648 | val 2.714 | lr 1.49e-04 | 102,487 tok/s | 54.64 GB | 14.730B tok iter 112400 | train 2.811 | val 2.731 | lr 1.49e-04 | 102,488 tok/s | 54.64 GB | 14.733B tok iter 112420 | train 2.748 | val 2.701 | lr 1.49e-04 | 102,489 tok/s | 54.64 GB | 14.735B tok iter 112440 | train 2.723 | val 2.713 | lr 1.49e-04 | 102,490 tok/s | 54.64 GB | 14.738B tok iter 112460 | train 2.631 | val 2.660 | lr 1.49e-04 | 102,492 tok/s | 54.64 GB | 14.740B tok iter 112480 | train 2.657 | val 2.677 | lr 1.49e-04 | 102,493 tok/s | 54.64 GB | 14.743B tok checkpoint @ iter 112491 -> checkpoints/500m_latest.pt iter 112500 | train 2.764 | val 2.726 | lr 1.49e-04 | 102,484 tok/s | 54.64 GB | 14.746B tok checkpoint @ iter 112500 -> checkpoints/500m_latest.pt iter 112520 | train 2.608 | val 2.746 | lr 1.48e-04 | 102,467 tok/s | 54.64 GB | 14.748B tok iter 112540 | train 2.507 | val 2.701 | lr 1.48e-04 | 102,468 tok/s | 54.64 GB | 14.751B tok iter 112560 | train 2.667 | val 2.673 | lr 1.48e-04 | 102,470 tok/s | 54.64 GB | 14.754B tok iter 112580 | train 2.760 | val 2.662 | lr 1.48e-04 | 102,470 tok/s | 54.64 GB | 14.756B tok iter 112600 | train 2.735 | val 2.709 | lr 1.48e-04 | 102,472 tok/s | 54.64 GB | 14.759B tok iter 112620 | train 2.682 | val 2.733 | lr 1.48e-04 | 102,473 tok/s | 54.64 GB | 14.761B tok iter 112640 | train 2.671 | val 2.728 | lr 1.48e-04 | 102,474 tok/s | 54.64 GB | 14.764B tok iter 112660 | train 2.701 | val 2.707 | lr 1.48e-04 | 102,475 tok/s | 54.64 GB | 14.767B tok iter 112680 | train 2.694 | val 2.647 | lr 1.48e-04 | 102,476 tok/s | 54.64 GB | 14.769B tok iter 112700 | train 2.835 | val 2.689 | lr 1.48e-04 | 102,478 tok/s | 54.64 GB | 14.772B tok iter 112720 | train 2.667 | val 2.655 | lr 1.48e-04 | 102,479 tok/s | 54.64 GB | 14.775B tok iter 112740 | train 2.659 | val 2.710 | lr 1.48e-04 | 102,479 tok/s | 54.64 GB | 14.777B tok iter 112760 | train 2.720 | val 2.716 | lr 1.47e-04 | 102,481 tok/s | 54.64 GB | 14.780B tok iter 112780 | train 2.670 | val 2.703 | lr 1.47e-04 | 102,482 tok/s | 54.64 GB | 14.782B tok iter 112800 | train 2.863 | val 2.703 | lr 1.47e-04 | 102,484 tok/s | 54.64 GB | 14.785B tok iter 112820 | train 2.757 | val 2.691 | lr 1.47e-04 | 102,484 tok/s | 54.64 GB | 14.788B tok iter 112840 | train 2.828 | val 2.687 | lr 1.47e-04 | 102,486 tok/s | 54.64 GB | 14.790B tok iter 112860 | train 2.663 | val 2.714 | lr 1.47e-04 | 102,487 tok/s | 54.64 GB | 14.793B tok iter 112880 | train 2.648 | val 2.737 | lr 1.47e-04 | 102,488 tok/s | 54.64 GB | 14.796B tok iter 112900 | train 2.825 | val 2.659 | lr 1.47e-04 | 102,489 tok/s | 54.64 GB | 14.798B tok iter 112920 | train 2.617 | val 2.736 | lr 1.47e-04 | 102,490 tok/s | 54.64 GB | 14.801B tok iter 112940 | train 2.565 | val 2.638 | lr 1.47e-04 | 102,492 tok/s | 54.64 GB | 14.803B tok iter 112960 | train 2.698 | val 2.669 | lr 1.47e-04 | 102,493 tok/s | 54.64 GB | 14.806B tok iter 112980 | train 2.655 | val 2.701 | lr 1.47e-04 | 102,493 tok/s | 54.64 GB | 14.809B tok checkpoint @ iter 112991 -> checkpoints/500m_latest.pt iter 113000 | train 2.765 | val 2.709 | lr 1.46e-04 | 102,485 tok/s | 54.64 GB | 14.811B tok checkpoint @ iter 113000 -> checkpoints/500m_latest.pt iter 113020 | train 2.701 | val 2.706 | lr 1.46e-04 | 102,467 tok/s | 54.64 GB | 14.814B tok iter 113040 | train 2.711 | val 2.718 | lr 1.46e-04 | 102,469 tok/s | 54.64 GB | 14.817B tok iter 113060 | train 2.718 | val 2.699 | lr 1.46e-04 | 102,469 tok/s | 54.64 GB | 14.819B tok iter 113080 | train 2.738 | val 2.712 | lr 1.46e-04 | 102,471 tok/s | 54.64 GB | 14.822B tok iter 113100 | train 2.596 | val 2.693 | lr 1.46e-04 | 102,472 tok/s | 54.64 GB | 14.824B tok iter 113120 | train 2.627 | val 2.707 | lr 1.46e-04 | 102,473 tok/s | 54.64 GB | 14.827B tok iter 113140 | train 2.669 | val 2.687 | lr 1.46e-04 | 102,474 tok/s | 54.64 GB | 14.830B tok iter 113160 | train 2.620 | val 2.696 | lr 1.46e-04 | 102,475 tok/s | 54.64 GB | 14.832B tok iter 113180 | train 2.745 | val 2.674 | lr 1.46e-04 | 102,477 tok/s | 54.64 GB | 14.835B tok iter 113200 | train 2.577 | val 2.676 | lr 1.46e-04 | 102,478 tok/s | 54.64 GB | 14.837B tok iter 113220 | train 2.587 | val 2.712 | lr 1.46e-04 | 102,479 tok/s | 54.64 GB | 14.840B tok iter 113240 | train 2.645 | val 2.674 | lr 1.45e-04 | 102,480 tok/s | 54.64 GB | 14.843B tok iter 113260 | train 2.669 | val 2.699 | lr 1.45e-04 | 102,481 tok/s | 54.64 GB | 14.845B tok iter 113280 | train 2.648 | val 2.720 | lr 1.45e-04 | 102,483 tok/s | 54.64 GB | 14.848B tok iter 113300 | train 2.795 | val 2.689 | lr 1.45e-04 | 102,483 tok/s | 54.64 GB | 14.851B tok iter 113320 | train 2.795 | val 2.677 | lr 1.45e-04 | 102,484 tok/s | 54.64 GB | 14.853B tok iter 113340 | train 2.631 | val 2.721 | lr 1.45e-04 | 102,486 tok/s | 54.64 GB | 14.856B tok iter 113360 | train 2.611 | val 2.663 | lr 1.45e-04 | 102,487 tok/s | 54.64 GB | 14.858B tok iter 113380 | train 2.777 | val 2.685 | lr 1.45e-04 | 102,488 tok/s | 54.64 GB | 14.861B tok iter 113400 | train 2.576 | val 2.691 | lr 1.45e-04 | 102,489 tok/s | 54.64 GB | 14.864B tok iter 113420 | train 2.641 | val 2.637 | lr 1.45e-04 | 102,490 tok/s | 54.64 GB | 14.866B tok iter 113440 | train 2.642 | val 2.676 | lr 1.45e-04 | 102,492 tok/s | 54.64 GB | 14.869B tok iter 113460 | train 2.647 | val 2.694 | lr 1.45e-04 | 102,492 tok/s | 54.64 GB | 14.872B tok iter 113480 | train 2.677 | val 2.625 | lr 1.44e-04 | 102,494 tok/s | 54.64 GB | 14.874B tok checkpoint @ iter 113491 -> checkpoints/500m_latest.pt iter 113500 | train 2.760 | val 2.682 | lr 1.44e-04 | 102,486 tok/s | 54.64 GB | 14.877B tok checkpoint @ iter 113500 -> checkpoints/500m_latest.pt iter 113520 | train 2.684 | val 2.667 | lr 1.44e-04 | 102,469 tok/s | 54.64 GB | 14.879B tok iter 113540 | train 2.491 | val 2.695 | lr 1.44e-04 | 102,469 tok/s | 54.64 GB | 14.882B tok iter 113560 | train 2.569 | val 2.704 | lr 1.44e-04 | 102,471 tok/s | 54.64 GB | 14.885B tok iter 113580 | train 2.652 | val 2.694 | lr 1.44e-04 | 102,472 tok/s | 54.64 GB | 14.887B tok iter 113600 | train 2.597 | val 2.686 | lr 1.44e-04 | 102,474 tok/s | 54.64 GB | 14.890B tok iter 113620 | train 2.571 | val 2.688 | lr 1.44e-04 | 102,474 tok/s | 54.64 GB | 14.893B tok iter 113640 | train 2.614 | val 2.690 | lr 1.44e-04 | 102,475 tok/s | 54.64 GB | 14.895B tok iter 113660 | train 2.664 | val 2.678 | lr 1.44e-04 | 102,477 tok/s | 54.64 GB | 14.898B tok iter 113680 | train 2.729 | val 2.702 | lr 1.44e-04 | 102,478 tok/s | 54.64 GB | 14.900B tok iter 113700 | train 2.835 | val 2.712 | lr 1.44e-04 | 102,478 tok/s | 54.64 GB | 14.903B tok iter 113720 | train 2.664 | val 2.723 | lr 1.44e-04 | 102,480 tok/s | 54.64 GB | 14.906B tok iter 113740 | train 2.657 | val 2.692 | lr 1.43e-04 | 102,481 tok/s | 54.64 GB | 14.908B tok iter 113760 | train 2.710 | val 2.730 | lr 1.43e-04 | 102,483 tok/s | 54.64 GB | 14.911B tok iter 113780 | train 2.760 | val 2.671 | lr 1.43e-04 | 102,483 tok/s | 54.64 GB | 14.914B tok iter 113800 | train 2.809 | val 2.679 | lr 1.43e-04 | 102,484 tok/s | 54.64 GB | 14.916B tok iter 113820 | train 2.683 | val 2.657 | lr 1.43e-04 | 102,486 tok/s | 54.64 GB | 14.919B tok iter 113840 | train 2.726 | val 2.676 | lr 1.43e-04 | 102,487 tok/s | 54.64 GB | 14.921B tok iter 113860 | train 2.826 | val 2.704 | lr 1.43e-04 | 102,488 tok/s | 54.64 GB | 14.924B tok iter 113880 | train 2.507 | val 2.660 | lr 1.43e-04 | 102,489 tok/s | 54.64 GB | 14.927B tok iter 113900 | train 2.643 | val 2.706 | lr 1.43e-04 | 102,490 tok/s | 54.64 GB | 14.929B tok iter 113920 | train 2.629 | val 2.671 | lr 1.43e-04 | 102,492 tok/s | 54.64 GB | 14.932B tok iter 113940 | train 2.652 | val 2.709 | lr 1.43e-04 | 102,492 tok/s | 54.64 GB | 14.934B tok iter 113960 | train 2.701 | val 2.666 | lr 1.43e-04 | 102,494 tok/s | 54.64 GB | 14.937B tok iter 113980 | train 2.692 | val 2.667 | lr 1.42e-04 | 102,495 tok/s | 54.64 GB | 14.940B tok checkpoint @ iter 113991 -> checkpoints/500m_latest.pt iter 114000 | train 2.700 | val 2.648 | lr 1.42e-04 | 102,487 tok/s | 54.64 GB | 14.942B tok checkpoint @ iter 114000 -> checkpoints/500m_latest.pt iter 114020 | train 2.620 | val 2.630 | lr 1.42e-04 | 102,469 tok/s | 54.64 GB | 14.945B tok iter 114040 | train 2.667 | val 2.638 | lr 1.42e-04 | 102,471 tok/s | 54.64 GB | 14.948B tok iter 114060 | train 2.487 | val 2.710 | lr 1.42e-04 | 102,472 tok/s | 54.64 GB | 14.950B tok iter 114080 | train 2.765 | val 2.672 | lr 1.42e-04 | 102,474 tok/s | 54.64 GB | 14.953B tok iter 114100 | train 2.631 | val 2.715 | lr 1.42e-04 | 102,474 tok/s | 54.64 GB | 14.955B tok iter 114120 | train 2.662 | val 2.700 | lr 1.42e-04 | 102,475 tok/s | 54.64 GB | 14.958B tok iter 114140 | train 2.736 | val 2.666 | lr 1.42e-04 | 102,477 tok/s | 54.64 GB | 14.961B tok iter 114160 | train 2.595 | val 2.648 | lr 1.42e-04 | 102,478 tok/s | 54.64 GB | 14.963B tok iter 114180 | train 2.705 | val 2.653 | lr 1.42e-04 | 102,478 tok/s | 54.64 GB | 14.966B tok iter 114200 | train 2.700 | val 2.717 | lr 1.42e-04 | 102,480 tok/s | 54.64 GB | 14.969B tok iter 114220 | train 2.672 | val 2.619 | lr 1.41e-04 | 102,481 tok/s | 54.64 GB | 14.971B tok iter 114240 | train 2.754 | val 2.722 | lr 1.41e-04 | 102,483 tok/s | 54.64 GB | 14.974B tok iter 114260 | train 2.633 | val 2.639 | lr 1.41e-04 | 102,483 tok/s | 54.64 GB | 14.976B tok iter 114280 | train 2.762 | val 2.720 | lr 1.41e-04 | 102,484 tok/s | 54.64 GB | 14.979B tok iter 114300 | train 2.661 | val 2.672 | lr 1.41e-04 | 102,486 tok/s | 54.64 GB | 14.982B tok iter 114320 | train 2.797 | val 2.671 | lr 1.41e-04 | 102,487 tok/s | 54.64 GB | 14.984B tok iter 114340 | train 2.755 | val 2.652 | lr 1.41e-04 | 102,488 tok/s | 54.64 GB | 14.987B tok iter 114360 | train 2.665 | val 2.661 | lr 1.41e-04 | 102,489 tok/s | 54.64 GB | 14.990B tok iter 114380 | train 2.832 | val 2.686 | lr 1.41e-04 | 102,490 tok/s | 54.64 GB | 14.992B tok iter 114400 | train 2.709 | val 2.673 | lr 1.41e-04 | 102,492 tok/s | 54.64 GB | 14.995B tok iter 114420 | train 2.616 | val 2.706 | lr 1.41e-04 | 102,492 tok/s | 54.64 GB | 14.997B tok iter 114440 | train 2.757 | val 2.716 | lr 1.41e-04 | 102,493 tok/s | 54.64 GB | 15.000B tok iter 114460 | train 2.728 | val 2.700 | lr 1.41e-04 | 102,495 tok/s | 54.64 GB | 15.003B tok iter 114480 | train 2.664 | val 2.710 | lr 1.40e-04 | 102,496 tok/s | 54.64 GB | 15.005B tok checkpoint @ iter 114491 -> checkpoints/500m_latest.pt iter 114500 | train 2.701 | val 2.656 | lr 1.40e-04 | 102,487 tok/s | 54.64 GB | 15.008B tok checkpoint @ iter 114500 -> checkpoints/500m_latest.pt iter 114520 | train 2.615 | val 2.691 | lr 1.40e-04 | 102,471 tok/s | 54.64 GB | 15.010B tok iter 114540 | train 2.670 | val 2.690 | lr 1.40e-04 | 102,472 tok/s | 54.64 GB | 15.013B tok iter 114560 | train 2.702 | val 2.673 | lr 1.40e-04 | 102,473 tok/s | 54.64 GB | 15.016B tok iter 114580 | train 2.665 | val 2.689 | lr 1.40e-04 | 102,474 tok/s | 54.64 GB | 15.018B tok iter 114600 | train 2.613 | val 2.630 | lr 1.40e-04 | 102,475 tok/s | 54.64 GB | 15.021B tok iter 114620 | train 2.615 | val 2.657 | lr 1.40e-04 | 102,477 tok/s | 54.64 GB | 15.024B tok iter 114640 | train 2.603 | val 2.717 | lr 1.40e-04 | 102,478 tok/s | 54.64 GB | 15.026B tok iter 114660 | train 2.856 | val 2.651 | lr 1.40e-04 | 102,478 tok/s | 54.64 GB | 15.029B tok iter 114680 | train 2.633 | val 2.637 | lr 1.40e-04 | 102,480 tok/s | 54.64 GB | 15.031B tok iter 114700 | train 2.726 | val 2.716 | lr 1.40e-04 | 102,481 tok/s | 54.64 GB | 15.034B tok iter 114720 | train 2.698 | val 2.650 | lr 1.39e-04 | 102,483 tok/s | 54.64 GB | 15.037B tok iter 114740 | train 2.591 | val 2.676 | lr 1.39e-04 | 102,483 tok/s | 54.64 GB | 15.039B tok iter 114760 | train 2.630 | val 2.644 | lr 1.39e-04 | 102,484 tok/s | 54.64 GB | 15.042B tok iter 114780 | train 2.661 | val 2.641 | lr 1.39e-04 | 102,486 tok/s | 54.64 GB | 15.045B tok iter 114800 | train 2.664 | val 2.652 | lr 1.39e-04 | 102,487 tok/s | 54.64 GB | 15.047B tok iter 114820 | train 2.741 | val 2.740 | lr 1.39e-04 | 102,487 tok/s | 54.64 GB | 15.050B tok iter 114840 | train 2.598 | val 2.726 | lr 1.39e-04 | 102,489 tok/s | 54.64 GB | 15.052B tok iter 114860 | train 2.858 | val 2.649 | lr 1.39e-04 | 102,490 tok/s | 54.64 GB | 15.055B tok iter 114880 | train 2.498 | val 2.649 | lr 1.39e-04 | 102,492 tok/s | 54.64 GB | 15.058B tok iter 114900 | train 2.656 | val 2.710 | lr 1.39e-04 | 102,492 tok/s | 54.64 GB | 15.060B tok iter 114920 | train 2.608 | val 2.648 | lr 1.39e-04 | 102,493 tok/s | 54.64 GB | 15.063B tok iter 114940 | train 2.779 | val 2.644 | lr 1.39e-04 | 102,495 tok/s | 54.64 GB | 15.066B tok iter 114960 | train 2.682 | val 2.649 | lr 1.39e-04 | 102,496 tok/s | 54.64 GB | 15.068B tok iter 114980 | train 2.691 | val 2.668 | lr 1.38e-04 | 102,496 tok/s | 54.64 GB | 15.071B tok checkpoint @ iter 114991 -> checkpoints/500m_latest.pt iter 115000 | train 2.656 | val 2.639 | lr 1.38e-04 | 102,488 tok/s | 54.64 GB | 15.073B tok checkpoint @ iter 115000 -> checkpoints/500m_latest.pt iter 115020 | train 2.776 | val 2.692 | lr 1.38e-04 | 102,472 tok/s | 54.64 GB | 15.076B tok iter 115040 | train 2.784 | val 2.669 | lr 1.38e-04 | 102,473 tok/s | 54.64 GB | 15.079B tok iter 115060 | train 2.714 | val 2.624 | lr 1.38e-04 | 102,474 tok/s | 54.64 GB | 15.081B tok iter 115080 | train 2.680 | val 2.644 | lr 1.38e-04 | 102,475 tok/s | 54.64 GB | 15.084B tok iter 115100 | train 2.684 | val 2.692 | lr 1.38e-04 | 102,476 tok/s | 54.64 GB | 15.087B tok iter 115120 | train 2.648 | val 2.645 | lr 1.38e-04 | 102,478 tok/s | 54.64 GB | 15.089B tok iter 115140 | train 2.738 | val 2.681 | lr 1.38e-04 | 102,478 tok/s | 54.64 GB | 15.092B tok iter 115160 | train 2.544 | val 2.724 | lr 1.38e-04 | 102,479 tok/s | 54.64 GB | 15.094B tok iter 115180 | train 2.723 | val 2.664 | lr 1.38e-04 | 102,481 tok/s | 54.64 GB | 15.097B tok iter 115200 | train 2.495 | val 2.695 | lr 1.38e-04 | 102,482 tok/s | 54.64 GB | 15.100B tok iter 115220 | train 2.743 | val 2.684 | lr 1.38e-04 | 102,483 tok/s | 54.64 GB | 15.102B tok iter 115240 | train 2.687 | val 2.672 | lr 1.37e-04 | 102,484 tok/s | 54.64 GB | 15.105B tok iter 115260 | train 2.627 | val 2.689 | lr 1.37e-04 | 102,485 tok/s | 54.64 GB | 15.107B tok iter 115280 | train 2.745 | val 2.686 | lr 1.37e-04 | 102,487 tok/s | 54.64 GB | 15.110B tok iter 115300 | train 2.804 | val 2.681 | lr 1.37e-04 | 102,487 tok/s | 54.64 GB | 15.113B tok iter 115320 | train 2.609 | val 2.694 | lr 1.37e-04 | 102,488 tok/s | 54.64 GB | 15.115B tok iter 115340 | train 2.690 | val 2.614 | lr 1.37e-04 | 102,490 tok/s | 54.64 GB | 15.118B tok iter 115360 | train 2.813 | val 2.704 | lr 1.37e-04 | 102,491 tok/s | 54.64 GB | 15.121B tok iter 115380 | train 2.593 | val 2.670 | lr 1.37e-04 | 102,491 tok/s | 54.64 GB | 15.123B tok iter 115400 | train 2.763 | val 2.679 | lr 1.37e-04 | 102,493 tok/s | 54.64 GB | 15.126B tok iter 115420 | train 2.642 | val 2.666 | lr 1.37e-04 | 102,494 tok/s | 54.64 GB | 15.128B tok iter 115440 | train 2.779 | val 2.668 | lr 1.37e-04 | 102,496 tok/s | 54.64 GB | 15.131B tok iter 115460 | train 2.713 | val 2.683 | lr 1.37e-04 | 102,496 tok/s | 54.64 GB | 15.134B tok iter 115480 | train 2.605 | val 2.669 | lr 1.36e-04 | 102,497 tok/s | 54.64 GB | 15.136B tok checkpoint @ iter 115491 -> checkpoints/500m_latest.pt iter 115500 | train 2.698 | val 2.679 | lr 1.36e-04 | 102,489 tok/s | 54.64 GB | 15.139B tok checkpoint @ iter 115500 -> checkpoints/500m_latest.pt iter 115520 | train 2.637 | val 2.668 | lr 1.36e-04 | 102,473 tok/s | 54.64 GB | 15.142B tok iter 115540 | train 2.544 | val 2.658 | lr 1.36e-04 | 102,473 tok/s | 54.64 GB | 15.144B tok iter 115560 | train 2.592 | val 2.702 | lr 1.36e-04 | 102,474 tok/s | 54.64 GB | 15.147B tok iter 115580 | train 2.628 | val 2.679 | lr 1.36e-04 | 102,476 tok/s | 54.64 GB | 15.149B tok iter 115600 | train 2.692 | val 2.669 | lr 1.36e-04 | 102,477 tok/s | 54.64 GB | 15.152B tok iter 115620 | train 2.778 | val 2.640 | lr 1.36e-04 | 102,477 tok/s | 54.64 GB | 15.155B tok iter 115640 | train 2.727 | val 2.685 | lr 1.36e-04 | 102,479 tok/s | 54.64 GB | 15.157B tok iter 115660 | train 2.560 | val 2.706 | lr 1.36e-04 | 102,480 tok/s | 54.64 GB | 15.160B tok iter 115680 | train 2.705 | val 2.663 | lr 1.36e-04 | 102,481 tok/s | 54.64 GB | 15.163B tok iter 115700 | train 2.723 | val 2.723 | lr 1.36e-04 | 102,482 tok/s | 54.64 GB | 15.165B tok iter 115720 | train 2.654 | val 2.719 | lr 1.36e-04 | 102,483 tok/s | 54.64 GB | 15.168B tok iter 115740 | train 2.817 | val 2.688 | lr 1.35e-04 | 102,484 tok/s | 54.64 GB | 15.170B tok iter 115760 | train 2.615 | val 2.686 | lr 1.35e-04 | 102,486 tok/s | 54.64 GB | 15.173B tok iter 115780 | train 2.722 | val 2.681 | lr 1.35e-04 | 102,486 tok/s | 54.64 GB | 15.176B tok iter 115800 | train 2.674 | val 2.692 | lr 1.35e-04 | 102,487 tok/s | 54.64 GB | 15.178B tok iter 115820 | train 2.688 | val 2.666 | lr 1.35e-04 | 102,489 tok/s | 54.64 GB | 15.181B tok iter 115840 | train 2.674 | val 2.721 | lr 1.35e-04 | 102,490 tok/s | 54.64 GB | 15.184B tok iter 115860 | train 2.693 | val 2.725 | lr 1.35e-04 | 102,490 tok/s | 54.64 GB | 15.186B tok iter 115880 | train 2.803 | val 2.657 | lr 1.35e-04 | 102,492 tok/s | 54.64 GB | 15.189B tok iter 115900 | train 2.702 | val 2.649 | lr 1.35e-04 | 102,493 tok/s | 54.64 GB | 15.191B tok iter 115920 | train 2.795 | val 2.669 | lr 1.35e-04 | 102,495 tok/s | 54.64 GB | 15.194B tok iter 115940 | train 2.758 | val 2.678 | lr 1.35e-04 | 102,495 tok/s | 54.64 GB | 15.197B tok iter 115960 | train 2.652 | val 2.683 | lr 1.35e-04 | 102,496 tok/s | 54.64 GB | 15.199B tok iter 115980 | train 2.643 | val 2.680 | lr 1.35e-04 | 102,498 tok/s | 54.64 GB | 15.202B tok checkpoint @ iter 115990 -> checkpoints/500m_latest.pt iter 116000 | train 2.620 | val 2.661 | lr 1.34e-04 | 102,490 tok/s | 54.64 GB | 15.204B tok checkpoint @ iter 116000 -> checkpoints/500m_latest.pt iter 116020 | train 2.672 | val 2.670 | lr 1.34e-04 | 102,472 tok/s | 54.64 GB | 15.207B tok iter 116040 | train 2.795 | val 2.650 | lr 1.34e-04 | 102,473 tok/s | 54.64 GB | 15.210B tok iter 116060 | train 2.696 | val 2.683 | lr 1.34e-04 | 102,475 tok/s | 54.64 GB | 15.212B tok iter 116080 | train 2.746 | val 2.660 | lr 1.34e-04 | 102,476 tok/s | 54.64 GB | 15.215B tok iter 116100 | train 2.733 | val 2.679 | lr 1.34e-04 | 102,476 tok/s | 54.64 GB | 15.218B tok iter 116120 | train 2.650 | val 2.671 | lr 1.34e-04 | 102,478 tok/s | 54.64 GB | 15.220B tok iter 116140 | train 2.668 | val 2.682 | lr 1.34e-04 | 102,479 tok/s | 54.64 GB | 15.223B tok iter 116160 | train 2.537 | val 2.686 | lr 1.34e-04 | 102,481 tok/s | 54.64 GB | 15.225B tok iter 116180 | train 2.753 | val 2.637 | lr 1.34e-04 | 102,481 tok/s | 54.64 GB | 15.228B tok iter 116200 | train 2.727 | val 2.658 | lr 1.34e-04 | 102,482 tok/s | 54.64 GB | 15.231B tok iter 116220 | train 2.656 | val 2.688 | lr 1.34e-04 | 102,484 tok/s | 54.64 GB | 15.233B tok iter 116240 | train 2.710 | val 2.702 | lr 1.34e-04 | 102,485 tok/s | 54.64 GB | 15.236B tok iter 116260 | train 2.691 | val 2.663 | lr 1.33e-04 | 102,485 tok/s | 54.64 GB | 15.239B tok iter 116280 | train 2.701 | val 2.678 | lr 1.33e-04 | 102,487 tok/s | 54.64 GB | 15.241B tok iter 116300 | train 2.642 | val 2.702 | lr 1.33e-04 | 102,488 tok/s | 54.64 GB | 15.244B tok iter 116320 | train 2.690 | val 2.664 | lr 1.33e-04 | 102,489 tok/s | 54.64 GB | 15.246B tok iter 116340 | train 2.616 | val 2.653 | lr 1.33e-04 | 102,490 tok/s | 54.64 GB | 15.249B tok iter 116360 | train 2.533 | val 2.681 | lr 1.33e-04 | 102,491 tok/s | 54.64 GB | 15.252B tok iter 116380 | train 2.693 | val 2.667 | lr 1.33e-04 | 102,493 tok/s | 54.64 GB | 15.254B tok iter 116400 | train 2.544 | val 2.696 | lr 1.33e-04 | 102,494 tok/s | 54.64 GB | 15.257B tok iter 116420 | train 2.606 | val 2.724 | lr 1.33e-04 | 102,494 tok/s | 54.64 GB | 15.260B tok iter 116440 | train 2.760 | val 2.713 | lr 1.33e-04 | 102,496 tok/s | 54.64 GB | 15.262B tok iter 116460 | train 2.624 | val 2.629 | lr 1.33e-04 | 102,497 tok/s | 54.64 GB | 15.265B tok iter 116480 | train 2.767 | val 2.713 | lr 1.33e-04 | 102,498 tok/s | 54.64 GB | 15.267B tok checkpoint @ iter 116491 -> checkpoints/500m_latest.pt iter 116500 | train 2.695 | val 2.710 | lr 1.33e-04 | 102,489 tok/s | 54.64 GB | 15.270B tok checkpoint @ iter 116500 -> checkpoints/500m_latest.pt iter 116520 | train 2.748 | val 2.667 | lr 1.32e-04 | 102,473 tok/s | 54.64 GB | 15.273B tok iter 116540 | train 2.687 | val 2.683 | lr 1.32e-04 | 102,474 tok/s | 54.64 GB | 15.275B tok iter 116560 | train 2.765 | val 2.643 | lr 1.32e-04 | 102,476 tok/s | 54.64 GB | 15.278B tok iter 116580 | train 2.666 | val 2.724 | lr 1.32e-04 | 102,476 tok/s | 54.64 GB | 15.281B tok iter 116600 | train 2.501 | val 2.668 | lr 1.32e-04 | 102,477 tok/s | 54.64 GB | 15.283B tok iter 116620 | train 2.643 | val 2.655 | lr 1.32e-04 | 102,479 tok/s | 54.64 GB | 15.286B tok iter 116640 | train 2.718 | val 2.631 | lr 1.32e-04 | 102,480 tok/s | 54.64 GB | 15.288B tok iter 116660 | train 2.654 | val 2.680 | lr 1.32e-04 | 102,480 tok/s | 54.64 GB | 15.291B tok iter 116680 | train 2.656 | val 2.678 | lr 1.32e-04 | 102,482 tok/s | 54.64 GB | 15.294B tok iter 116700 | train 2.730 | val 2.706 | lr 1.32e-04 | 102,483 tok/s | 54.64 GB | 15.296B tok iter 116720 | train 2.662 | val 2.691 | lr 1.32e-04 | 102,485 tok/s | 54.64 GB | 15.299B tok iter 116740 | train 2.584 | val 2.695 | lr 1.32e-04 | 102,485 tok/s | 54.64 GB | 15.301B tok iter 116760 | train 2.602 | val 2.679 | lr 1.32e-04 | 102,486 tok/s | 54.64 GB | 15.304B tok iter 116780 | train 2.722 | val 2.705 | lr 1.31e-04 | 102,488 tok/s | 54.64 GB | 15.307B tok iter 116800 | train 2.703 | val 2.691 | lr 1.31e-04 | 102,489 tok/s | 54.64 GB | 15.309B tok iter 116820 | train 2.710 | val 2.668 | lr 1.31e-04 | 102,489 tok/s | 54.64 GB | 15.312B tok iter 116840 | train 2.560 | val 2.700 | lr 1.31e-04 | 102,491 tok/s | 54.64 GB | 15.315B tok iter 116860 | train 2.609 | val 2.687 | lr 1.31e-04 | 102,492 tok/s | 54.64 GB | 15.317B tok iter 116880 | train 2.653 | val 2.627 | lr 1.31e-04 | 102,493 tok/s | 54.64 GB | 15.320B tok iter 116900 | train 2.798 | val 2.695 | lr 1.31e-04 | 102,494 tok/s | 54.64 GB | 15.322B tok iter 116920 | train 2.669 | val 2.674 | lr 1.31e-04 | 102,495 tok/s | 54.64 GB | 15.325B tok iter 116940 | train 2.649 | val 2.667 | lr 1.31e-04 | 102,496 tok/s | 54.64 GB | 15.328B tok iter 116960 | train 2.717 | val 2.661 | lr 1.31e-04 | 102,498 tok/s | 54.64 GB | 15.330B tok iter 116980 | train 2.678 | val 2.698 | lr 1.31e-04 | 102,498 tok/s | 54.64 GB | 15.333B tok checkpoint @ iter 116991 -> checkpoints/500m_latest.pt iter 117000 | train 2.619 | val 2.684 | lr 1.31e-04 | 102,490 tok/s | 54.64 GB | 15.336B tok checkpoint @ iter 117000 -> checkpoints/500m_latest.pt iter 117020 | train 2.753 | val 2.651 | lr 1.31e-04 | 102,474 tok/s | 54.64 GB | 15.338B tok iter 117040 | train 2.627 | val 2.658 | lr 1.30e-04 | 102,475 tok/s | 54.64 GB | 15.341B tok iter 117060 | train 2.734 | val 2.680 | lr 1.30e-04 | 102,476 tok/s | 54.64 GB | 15.343B tok iter 117080 | train 2.688 | val 2.697 | lr 1.30e-04 | 102,477 tok/s | 54.64 GB | 15.346B tok iter 117100 | train 2.627 | val 2.691 | lr 1.30e-04 | 102,478 tok/s | 54.64 GB | 15.349B tok iter 117120 | train 2.607 | val 2.682 | lr 1.30e-04 | 102,480 tok/s | 54.64 GB | 15.351B tok iter 117140 | train 2.596 | val 2.695 | lr 1.30e-04 | 102,480 tok/s | 54.64 GB | 15.354B tok iter 117160 | train 2.644 | val 2.684 | lr 1.30e-04 | 102,481 tok/s | 54.64 GB | 15.357B tok iter 117180 | train 2.689 | val 2.627 | lr 1.30e-04 | 102,483 tok/s | 54.64 GB | 15.359B tok iter 117200 | train 2.601 | val 2.678 | lr 1.30e-04 | 102,484 tok/s | 54.64 GB | 15.362B tok iter 117220 | train 2.576 | val 2.667 | lr 1.30e-04 | 102,484 tok/s | 54.64 GB | 15.364B tok iter 117240 | train 2.746 | val 2.715 | lr 1.30e-04 | 102,486 tok/s | 54.64 GB | 15.367B tok iter 117260 | train 2.675 | val 2.685 | lr 1.30e-04 | 102,487 tok/s | 54.64 GB | 15.370B tok iter 117280 | train 2.695 | val 2.676 | lr 1.30e-04 | 102,488 tok/s | 54.64 GB | 15.372B tok iter 117300 | train 2.678 | val 2.703 | lr 1.29e-04 | 102,489 tok/s | 54.64 GB | 15.375B tok iter 117320 | train 2.732 | val 2.685 | lr 1.29e-04 | 102,490 tok/s | 54.64 GB | 15.377B tok iter 117340 | train 2.678 | val 2.611 | lr 1.29e-04 | 102,491 tok/s | 54.64 GB | 15.380B tok iter 117360 | train 2.740 | val 2.679 | lr 1.29e-04 | 102,493 tok/s | 54.64 GB | 15.383B tok iter 117380 | train 2.808 | val 2.712 | lr 1.29e-04 | 102,493 tok/s | 54.64 GB | 15.385B tok iter 117400 | train 2.613 | val 2.695 | lr 1.29e-04 | 102,494 tok/s | 54.64 GB | 15.388B tok iter 117420 | train 2.468 | val 2.716 | lr 1.29e-04 | 102,496 tok/s | 54.64 GB | 15.391B tok iter 117440 | train 2.827 | val 2.669 | lr 1.29e-04 | 102,497 tok/s | 54.64 GB | 15.393B tok iter 117460 | train 2.650 | val 2.637 | lr 1.29e-04 | 102,497 tok/s | 54.64 GB | 15.396B tok iter 117480 | train 2.822 | val 2.650 | lr 1.29e-04 | 102,499 tok/s | 54.64 GB | 15.398B tok checkpoint @ iter 117491 -> checkpoints/500m_latest.pt iter 117500 | train 2.696 | val 2.669 | lr 1.29e-04 | 102,491 tok/s | 54.64 GB | 15.401B tok checkpoint @ iter 117500 -> checkpoints/500m_latest.pt iter 117520 | train 2.671 | val 2.701 | lr 1.29e-04 | 102,475 tok/s | 54.64 GB | 15.404B tok iter 117540 | train 2.777 | val 2.664 | lr 1.29e-04 | 102,475 tok/s | 54.64 GB | 15.406B tok iter 117560 | train 2.575 | val 2.652 | lr 1.29e-04 | 102,476 tok/s | 54.64 GB | 15.409B tok iter 117580 | train 2.751 | val 2.650 | lr 1.28e-04 | 102,478 tok/s | 54.64 GB | 15.412B tok iter 117600 | train 2.648 | val 2.687 | lr 1.28e-04 | 102,479 tok/s | 54.64 GB | 15.414B tok iter 117620 | train 2.648 | val 2.688 | lr 1.28e-04 | 102,479 tok/s | 54.64 GB | 15.417B tok iter 117640 | train 2.653 | val 2.673 | lr 1.28e-04 | 102,481 tok/s | 54.64 GB | 15.419B tok iter 117660 | train 2.618 | val 2.700 | lr 1.28e-04 | 102,482 tok/s | 54.64 GB | 15.422B tok iter 117680 | train 2.754 | val 2.641 | lr 1.28e-04 | 102,483 tok/s | 54.64 GB | 15.425B tok iter 117700 | train 2.625 | val 2.654 | lr 1.28e-04 | 102,484 tok/s | 54.64 GB | 15.427B tok iter 117720 | train 2.692 | val 2.696 | lr 1.28e-04 | 102,485 tok/s | 54.64 GB | 15.430B tok iter 117740 | train 2.725 | val 2.724 | lr 1.28e-04 | 102,486 tok/s | 54.64 GB | 15.433B tok iter 117760 | train 2.589 | val 2.619 | lr 1.28e-04 | 102,488 tok/s | 54.64 GB | 15.435B tok iter 117780 | train 2.671 | val 2.677 | lr 1.28e-04 | 102,488 tok/s | 54.64 GB | 15.438B tok iter 117800 | train 2.730 | val 2.660 | lr 1.28e-04 | 102,489 tok/s | 54.64 GB | 15.440B tok iter 117820 | train 2.808 | val 2.684 | lr 1.28e-04 | 102,491 tok/s | 54.64 GB | 15.443B tok iter 117840 | train 2.650 | val 2.650 | lr 1.27e-04 | 102,492 tok/s | 54.64 GB | 15.446B tok iter 117860 | train 2.705 | val 2.619 | lr 1.27e-04 | 102,492 tok/s | 54.64 GB | 15.448B tok iter 117880 | train 2.672 | val 2.685 | lr 1.27e-04 | 102,494 tok/s | 54.64 GB | 15.451B tok iter 117900 | train 2.695 | val 2.711 | lr 1.27e-04 | 102,495 tok/s | 54.64 GB | 15.454B tok iter 117920 | train 2.594 | val 2.657 | lr 1.27e-04 | 102,496 tok/s | 54.64 GB | 15.456B tok iter 117940 | train 2.706 | val 2.716 | lr 1.27e-04 | 102,497 tok/s | 54.64 GB | 15.459B tok iter 117960 | train 2.597 | val 2.713 | lr 1.27e-04 | 102,498 tok/s | 54.64 GB | 15.461B tok iter 117980 | train 2.772 | val 2.657 | lr 1.27e-04 | 102,499 tok/s | 54.64 GB | 15.464B tok checkpoint @ iter 117991 -> checkpoints/500m_latest.pt iter 118000 | train 2.615 | val 2.666 | lr 1.27e-04 | 102,491 tok/s | 54.64 GB | 15.467B tok checkpoint @ iter 118000 -> checkpoints/500m_latest.pt iter 118020 | train 2.602 | val 2.677 | lr 1.27e-04 | 102,474 tok/s | 54.64 GB | 15.469B tok iter 118040 | train 2.663 | val 2.673 | lr 1.27e-04 | 102,475 tok/s | 54.64 GB | 15.472B tok iter 118060 | train 2.617 | val 2.715 | lr 1.27e-04 | 102,476 tok/s | 54.64 GB | 15.474B tok iter 118080 | train 2.698 | val 2.685 | lr 1.27e-04 | 102,478 tok/s | 54.64 GB | 15.477B tok iter 118100 | train 2.632 | val 2.674 | lr 1.27e-04 | 102,478 tok/s | 54.64 GB | 15.480B tok iter 118120 | train 2.683 | val 2.616 | lr 1.26e-04 | 102,479 tok/s | 54.64 GB | 15.482B tok iter 118140 | train 2.685 | val 2.625 | lr 1.26e-04 | 102,481 tok/s | 54.64 GB | 15.485B tok iter 118160 | train 2.633 | val 2.688 | lr 1.26e-04 | 102,482 tok/s | 54.64 GB | 15.488B tok iter 118180 | train 2.564 | val 2.674 | lr 1.26e-04 | 102,482 tok/s | 54.64 GB | 15.490B tok iter 118200 | train 2.834 | val 2.659 | lr 1.26e-04 | 102,484 tok/s | 54.64 GB | 15.493B tok iter 118220 | train 2.732 | val 2.652 | lr 1.26e-04 | 102,485 tok/s | 54.64 GB | 15.495B tok iter 118240 | train 2.621 | val 2.654 | lr 1.26e-04 | 102,486 tok/s | 54.64 GB | 15.498B tok iter 118260 | train 2.729 | val 2.676 | lr 1.26e-04 | 102,487 tok/s | 54.64 GB | 15.501B tok iter 118280 | train 2.618 | val 2.683 | lr 1.26e-04 | 102,488 tok/s | 54.64 GB | 15.503B tok iter 118300 | train 2.694 | val 2.656 | lr 1.26e-04 | 102,489 tok/s | 54.64 GB | 15.506B tok iter 118320 | train 2.601 | val 2.666 | lr 1.26e-04 | 102,491 tok/s | 54.64 GB | 15.509B tok iter 118340 | train 2.657 | val 2.688 | lr 1.26e-04 | 102,491 tok/s | 54.64 GB | 15.511B tok iter 118360 | train 2.559 | val 2.665 | lr 1.26e-04 | 102,492 tok/s | 54.64 GB | 15.514B tok iter 118380 | train 2.640 | val 2.638 | lr 1.25e-04 | 102,494 tok/s | 54.64 GB | 15.516B tok iter 118400 | train 2.760 | val 2.722 | lr 1.25e-04 | 102,495 tok/s | 54.64 GB | 15.519B tok iter 118420 | train 2.678 | val 2.709 | lr 1.25e-04 | 102,495 tok/s | 54.64 GB | 15.522B tok iter 118440 | train 2.688 | val 2.676 | lr 1.25e-04 | 102,497 tok/s | 54.64 GB | 15.524B tok iter 118460 | train 2.582 | val 2.682 | lr 1.25e-04 | 102,498 tok/s | 54.64 GB | 15.527B tok iter 118480 | train 2.839 | val 2.685 | lr 1.25e-04 | 102,499 tok/s | 54.64 GB | 15.530B tok checkpoint @ iter 118490 -> checkpoints/500m_latest.pt iter 118500 | train 2.630 | val 2.724 | lr 1.25e-04 | 102,491 tok/s | 54.64 GB | 15.532B tok checkpoint @ iter 118500 -> checkpoints/500m_latest.pt iter 118520 | train 2.709 | val 2.667 | lr 1.25e-04 | 102,474 tok/s | 54.64 GB | 15.535B tok iter 118540 | train 2.648 | val 2.635 | lr 1.25e-04 | 102,475 tok/s | 54.64 GB | 15.537B tok iter 118560 | train 2.621 | val 2.658 | lr 1.25e-04 | 102,477 tok/s | 54.64 GB | 15.540B tok iter 118580 | train 2.612 | val 2.657 | lr 1.25e-04 | 102,477 tok/s | 54.64 GB | 15.543B tok iter 118600 | train 2.682 | val 2.674 | lr 1.25e-04 | 102,478 tok/s | 54.64 GB | 15.545B tok iter 118620 | train 2.604 | val 2.679 | lr 1.25e-04 | 102,480 tok/s | 54.64 GB | 15.548B tok iter 118640 | train 2.784 | val 2.679 | lr 1.25e-04 | 102,481 tok/s | 54.64 GB | 15.551B tok iter 118660 | train 2.679 | val 2.621 | lr 1.24e-04 | 102,481 tok/s | 54.64 GB | 15.553B tok iter 118680 | train 2.667 | val 2.693 | lr 1.24e-04 | 102,483 tok/s | 54.64 GB | 15.556B tok iter 118700 | train 2.673 | val 2.662 | lr 1.24e-04 | 102,484 tok/s | 54.64 GB | 15.558B tok iter 118720 | train 2.589 | val 2.672 | lr 1.24e-04 | 102,485 tok/s | 54.64 GB | 15.561B tok iter 118740 | train 2.708 | val 2.663 | lr 1.24e-04 | 102,486 tok/s | 54.64 GB | 15.564B tok iter 118760 | train 2.715 | val 2.706 | lr 1.24e-04 | 102,487 tok/s | 54.64 GB | 15.566B tok iter 118780 | train 2.831 | val 2.680 | lr 1.24e-04 | 102,488 tok/s | 54.64 GB | 15.569B tok iter 118800 | train 2.670 | val 2.694 | lr 1.24e-04 | 102,490 tok/s | 54.64 GB | 15.571B tok iter 118820 | train 2.672 | val 2.682 | lr 1.24e-04 | 102,490 tok/s | 54.64 GB | 15.574B tok iter 118840 | train 2.756 | val 2.657 | lr 1.24e-04 | 102,491 tok/s | 54.64 GB | 15.577B tok iter 118860 | train 2.558 | val 2.655 | lr 1.24e-04 | 102,493 tok/s | 54.64 GB | 15.579B tok iter 118880 | train 2.799 | val 2.641 | lr 1.24e-04 | 102,494 tok/s | 54.64 GB | 15.582B tok iter 118900 | train 2.589 | val 2.662 | lr 1.24e-04 | 102,494 tok/s | 54.64 GB | 15.585B tok iter 118920 | train 2.622 | val 2.688 | lr 1.24e-04 | 102,496 tok/s | 54.64 GB | 15.587B tok iter 118940 | train 2.595 | val 2.692 | lr 1.23e-04 | 102,497 tok/s | 54.64 GB | 15.590B tok iter 118960 | train 2.814 | val 2.669 | lr 1.23e-04 | 102,498 tok/s | 54.64 GB | 15.592B tok iter 118980 | train 2.716 | val 2.672 | lr 1.23e-04 | 102,499 tok/s | 54.64 GB | 15.595B tok checkpoint @ iter 118990 -> checkpoints/500m_latest.pt iter 119000 | train 2.680 | val 2.636 | lr 1.23e-04 | 102,491 tok/s | 54.64 GB | 15.598B tok checkpoint @ iter 119000 -> checkpoints/500m_latest.pt iter 119020 | train 2.615 | val 2.722 | lr 1.23e-04 | 102,475 tok/s | 54.64 GB | 15.600B tok iter 119040 | train 2.632 | val 2.713 | lr 1.23e-04 | 102,476 tok/s | 54.64 GB | 15.603B tok iter 119060 | train 2.716 | val 2.668 | lr 1.23e-04 | 102,476 tok/s | 54.64 GB | 15.606B tok iter 119080 | train 2.659 | val 2.656 | lr 1.23e-04 | 102,478 tok/s | 54.64 GB | 15.608B tok iter 119100 | train 2.609 | val 2.645 | lr 1.23e-04 | 102,479 tok/s | 54.64 GB | 15.611B tok iter 119120 | train 2.649 | val 2.726 | lr 1.23e-04 | 102,480 tok/s | 54.64 GB | 15.613B tok iter 119140 | train 2.797 | val 2.672 | lr 1.23e-04 | 102,480 tok/s | 54.64 GB | 15.616B tok iter 119160 | train 2.650 | val 2.679 | lr 1.23e-04 | 102,482 tok/s | 54.64 GB | 15.619B tok iter 119180 | train 2.673 | val 2.656 | lr 1.23e-04 | 102,483 tok/s | 54.64 GB | 15.621B tok iter 119200 | train 2.620 | val 2.664 | lr 1.23e-04 | 102,484 tok/s | 54.64 GB | 15.624B tok iter 119220 | train 2.624 | val 2.705 | lr 1.22e-04 | 102,485 tok/s | 54.64 GB | 15.627B tok iter 119240 | train 2.771 | val 2.676 | lr 1.22e-04 | 102,486 tok/s | 54.64 GB | 15.629B tok iter 119260 | train 2.526 | val 2.737 | lr 1.22e-04 | 102,487 tok/s | 54.64 GB | 15.632B tok iter 119280 | train 2.757 | val 2.673 | lr 1.22e-04 | 102,489 tok/s | 54.64 GB | 15.634B tok iter 119300 | train 2.752 | val 2.654 | lr 1.22e-04 | 102,489 tok/s | 54.64 GB | 15.637B tok iter 119320 | train 2.575 | val 2.701 | lr 1.22e-04 | 102,490 tok/s | 54.64 GB | 15.640B tok iter 119340 | train 2.696 | val 2.673 | lr 1.22e-04 | 102,492 tok/s | 54.64 GB | 15.642B tok iter 119360 | train 2.682 | val 2.686 | lr 1.22e-04 | 102,493 tok/s | 54.64 GB | 15.645B tok iter 119380 | train 2.740 | val 2.701 | lr 1.22e-04 | 102,493 tok/s | 54.64 GB | 15.648B tok iter 119400 | train 2.595 | val 2.675 | lr 1.22e-04 | 102,495 tok/s | 54.64 GB | 15.650B tok iter 119420 | train 2.650 | val 2.695 | lr 1.22e-04 | 102,496 tok/s | 54.64 GB | 15.653B tok iter 119440 | train 2.745 | val 2.684 | lr 1.22e-04 | 102,497 tok/s | 54.64 GB | 15.655B tok iter 119460 | train 2.603 | val 2.667 | lr 1.22e-04 | 102,498 tok/s | 54.64 GB | 15.658B tok iter 119480 | train 2.560 | val 2.686 | lr 1.22e-04 | 102,499 tok/s | 54.64 GB | 15.661B tok checkpoint @ iter 119490 -> checkpoints/500m_latest.pt iter 119500 | train 2.572 | val 2.692 | lr 1.21e-04 | 102,491 tok/s | 54.64 GB | 15.663B tok checkpoint @ iter 119500 -> checkpoints/500m_latest.pt iter 119520 | train 2.680 | val 2.641 | lr 1.21e-04 | 102,475 tok/s | 54.64 GB | 15.666B tok iter 119540 | train 2.603 | val 2.647 | lr 1.21e-04 | 102,475 tok/s | 54.64 GB | 15.668B tok iter 119560 | train 2.662 | val 2.635 | lr 1.21e-04 | 102,477 tok/s | 54.64 GB | 15.671B tok iter 119580 | train 2.699 | val 2.705 | lr 1.21e-04 | 102,478 tok/s | 54.64 GB | 15.674B tok iter 119600 | train 2.781 | val 2.682 | lr 1.21e-04 | 102,479 tok/s | 54.64 GB | 15.676B tok iter 119620 | train 2.695 | val 2.649 | lr 1.21e-04 | 102,480 tok/s | 54.64 GB | 15.679B tok iter 119640 | train 2.754 | val 2.703 | lr 1.21e-04 | 102,481 tok/s | 54.64 GB | 15.682B tok iter 119660 | train 2.684 | val 2.683 | lr 1.21e-04 | 102,482 tok/s | 54.64 GB | 15.684B tok iter 119680 | train 2.635 | val 2.622 | lr 1.21e-04 | 102,484 tok/s | 54.64 GB | 15.687B tok iter 119700 | train 2.763 | val 2.704 | lr 1.21e-04 | 102,484 tok/s | 54.64 GB | 15.689B tok iter 119720 | train 2.708 | val 2.656 | lr 1.21e-04 | 102,485 tok/s | 54.64 GB | 15.692B tok iter 119740 | train 2.686 | val 2.644 | lr 1.21e-04 | 102,487 tok/s | 54.64 GB | 15.695B tok iter 119760 | train 2.681 | val 2.699 | lr 1.21e-04 | 102,488 tok/s | 54.64 GB | 15.697B tok iter 119780 | train 2.580 | val 2.655 | lr 1.20e-04 | 102,488 tok/s | 54.64 GB | 15.700B tok iter 119800 | train 2.651 | val 2.671 | lr 1.20e-04 | 102,490 tok/s | 54.64 GB | 15.703B tok iter 119820 | train 2.723 | val 2.693 | lr 1.20e-04 | 102,491 tok/s | 54.64 GB | 15.705B tok iter 119840 | train 2.654 | val 2.669 | lr 1.20e-04 | 102,492 tok/s | 54.64 GB | 15.708B tok iter 119860 | train 2.509 | val 2.719 | lr 1.20e-04 | 102,493 tok/s | 54.64 GB | 15.710B tok iter 119880 | train 2.702 | val 2.695 | lr 1.20e-04 | 102,494 tok/s | 54.64 GB | 15.713B tok iter 119900 | train 2.616 | val 2.666 | lr 1.20e-04 | 102,495 tok/s | 54.64 GB | 15.716B tok iter 119920 | train 2.673 | val 2.668 | lr 1.20e-04 | 102,497 tok/s | 54.64 GB | 15.718B tok iter 119940 | train 2.741 | val 2.659 | lr 1.20e-04 | 102,497 tok/s | 54.64 GB | 15.721B tok iter 119960 | train 2.653 | val 2.664 | lr 1.20e-04 | 102,498 tok/s | 54.64 GB | 15.724B tok iter 119980 | train 2.637 | val 2.660 | lr 1.20e-04 | 102,499 tok/s | 54.64 GB | 15.726B tok checkpoint @ iter 119991 -> checkpoints/500m_latest.pt iter 120000 | train 2.560 | val 2.649 | lr 1.20e-04 | 102,492 tok/s | 54.64 GB | 15.729B tok checkpoint @ iter 120000 -> checkpoints/500m_latest.pt iter 120020 | train 2.699 | val 2.659 | lr 1.20e-04 | 102,474 tok/s | 54.64 GB | 15.731B tok iter 120040 | train 2.701 | val 2.630 | lr 1.20e-04 | 102,476 tok/s | 54.64 GB | 15.734B tok iter 120060 | train 2.621 | val 2.756 | lr 1.19e-04 | 102,477 tok/s | 54.64 GB | 15.737B tok iter 120080 | train 2.686 | val 2.685 | lr 1.19e-04 | 102,478 tok/s | 54.64 GB | 15.739B tok iter 120100 | train 2.691 | val 2.639 | lr 1.19e-04 | 102,479 tok/s | 54.64 GB | 15.742B tok iter 120120 | train 2.713 | val 2.665 | lr 1.19e-04 | 102,480 tok/s | 54.64 GB | 15.744B tok iter 120140 | train 2.696 | val 2.662 | lr 1.19e-04 | 102,481 tok/s | 54.64 GB | 15.747B tok iter 120160 | train 2.719 | val 2.735 | lr 1.19e-04 | 102,482 tok/s | 54.64 GB | 15.750B tok iter 120180 | train 2.687 | val 2.665 | lr 1.19e-04 | 102,483 tok/s | 54.64 GB | 15.752B tok iter 120200 | train 2.528 | val 2.712 | lr 1.19e-04 | 102,484 tok/s | 54.64 GB | 15.755B tok iter 120220 | train 2.589 | val 2.682 | lr 1.19e-04 | 102,485 tok/s | 54.64 GB | 15.758B tok iter 120240 | train 2.557 | val 2.654 | lr 1.19e-04 | 102,487 tok/s | 54.64 GB | 15.760B tok iter 120260 | train 2.699 | val 2.673 | lr 1.19e-04 | 102,487 tok/s | 54.64 GB | 15.763B tok iter 120280 | train 2.698 | val 2.661 | lr 1.19e-04 | 102,488 tok/s | 54.64 GB | 15.765B tok iter 120300 | train 2.629 | val 2.671 | lr 1.19e-04 | 102,490 tok/s | 54.64 GB | 15.768B tok iter 120320 | train 2.706 | val 2.646 | lr 1.19e-04 | 102,491 tok/s | 54.64 GB | 15.771B tok iter 120340 | train 2.677 | val 2.659 | lr 1.18e-04 | 102,491 tok/s | 54.64 GB | 15.773B tok iter 120360 | train 2.757 | val 2.683 | lr 1.18e-04 | 102,493 tok/s | 54.64 GB | 15.776B tok iter 120380 | train 2.629 | val 2.703 | lr 1.18e-04 | 102,494 tok/s | 54.64 GB | 15.779B tok iter 120400 | train 2.675 | val 2.693 | lr 1.18e-04 | 102,495 tok/s | 54.64 GB | 15.781B tok iter 120420 | train 2.614 | val 2.642 | lr 1.18e-04 | 102,496 tok/s | 54.64 GB | 15.784B tok iter 120440 | train 2.754 | val 2.662 | lr 1.18e-04 | 102,497 tok/s | 54.64 GB | 15.786B tok iter 120460 | train 2.708 | val 2.684 | lr 1.18e-04 | 102,498 tok/s | 54.64 GB | 15.789B tok iter 120480 | train 2.682 | val 2.670 | lr 1.18e-04 | 102,500 tok/s | 54.64 GB | 15.792B tok checkpoint @ iter 120490 -> checkpoints/500m_latest.pt iter 120500 | train 2.703 | val 2.659 | lr 1.18e-04 | 102,491 tok/s | 54.64 GB | 15.794B tok checkpoint @ iter 120500 -> checkpoints/500m_latest.pt iter 120520 | train 2.583 | val 2.736 | lr 1.18e-04 | 102,475 tok/s | 54.64 GB | 15.797B tok iter 120540 | train 2.833 | val 2.701 | lr 1.18e-04 | 102,476 tok/s | 54.64 GB | 15.800B tok iter 120560 | train 2.700 | val 2.662 | lr 1.18e-04 | 102,478 tok/s | 54.64 GB | 15.802B tok iter 120580 | train 2.411 | val 2.645 | lr 1.18e-04 | 102,478 tok/s | 54.64 GB | 15.805B tok iter 120600 | train 2.675 | val 2.661 | lr 1.18e-04 | 102,479 tok/s | 54.64 GB | 15.807B tok iter 120620 | train 2.616 | val 2.664 | lr 1.18e-04 | 102,481 tok/s | 54.64 GB | 15.810B tok iter 120640 | train 2.547 | val 2.633 | lr 1.17e-04 | 102,482 tok/s | 54.64 GB | 15.813B tok iter 120660 | train 2.589 | val 2.644 | lr 1.17e-04 | 102,482 tok/s | 54.64 GB | 15.815B tok iter 120680 | train 2.765 | val 2.616 | lr 1.17e-04 | 102,484 tok/s | 54.64 GB | 15.818B tok iter 120700 | train 2.565 | val 2.677 | lr 1.17e-04 | 102,485 tok/s | 54.64 GB | 15.821B tok iter 120720 | train 2.675 | val 2.602 | lr 1.17e-04 | 102,486 tok/s | 54.64 GB | 15.823B tok iter 120740 | train 2.717 | val 2.656 | lr 1.17e-04 | 102,487 tok/s | 54.64 GB | 15.826B tok iter 120760 | train 2.558 | val 2.635 | lr 1.17e-04 | 102,488 tok/s | 54.64 GB | 15.828B tok iter 120780 | train 2.587 | val 2.626 | lr 1.17e-04 | 102,489 tok/s | 54.64 GB | 15.831B tok iter 120800 | train 2.556 | val 2.619 | lr 1.17e-04 | 102,490 tok/s | 54.64 GB | 15.834B tok iter 120820 | train 2.714 | val 2.706 | lr 1.17e-04 | 102,491 tok/s | 54.64 GB | 15.836B tok iter 120840 | train 2.677 | val 2.668 | lr 1.17e-04 | 102,492 tok/s | 54.64 GB | 15.839B tok iter 120860 | train 2.684 | val 2.661 | lr 1.17e-04 | 102,493 tok/s | 54.64 GB | 15.841B tok iter 120880 | train 2.689 | val 2.699 | lr 1.17e-04 | 102,495 tok/s | 54.64 GB | 15.844B tok iter 120900 | train 2.600 | val 2.666 | lr 1.17e-04 | 102,495 tok/s | 54.64 GB | 15.847B tok iter 120920 | train 2.645 | val 2.670 | lr 1.16e-04 | 102,496 tok/s | 54.64 GB | 15.849B tok iter 120940 | train 2.689 | val 2.621 | lr 1.16e-04 | 102,498 tok/s | 54.64 GB | 15.852B tok iter 120960 | train 2.744 | val 2.662 | lr 1.16e-04 | 102,499 tok/s | 54.64 GB | 15.855B tok iter 120980 | train 2.642 | val 2.672 | lr 1.16e-04 | 102,499 tok/s | 54.64 GB | 15.857B tok checkpoint @ iter 120990 -> checkpoints/500m_latest.pt iter 121000 | train 2.581 | val 2.684 | lr 1.16e-04 | 102,492 tok/s | 54.64 GB | 15.860B tok checkpoint @ iter 121000 -> checkpoints/500m_latest.pt iter 121020 | train 2.608 | val 2.695 | lr 1.16e-04 | 102,475 tok/s | 54.64 GB | 15.862B tok iter 121040 | train 2.673 | val 2.706 | lr 1.16e-04 | 102,477 tok/s | 54.64 GB | 15.865B tok iter 121060 | train 2.523 | val 2.683 | lr 1.16e-04 | 102,477 tok/s | 54.64 GB | 15.868B tok iter 121080 | train 2.628 | val 2.683 | lr 1.16e-04 | 102,478 tok/s | 54.64 GB | 15.870B tok iter 121100 | train 2.569 | val 2.640 | lr 1.16e-04 | 102,480 tok/s | 54.64 GB | 15.873B tok iter 121120 | train 2.634 | val 2.646 | lr 1.16e-04 | 102,481 tok/s | 54.64 GB | 15.876B tok iter 121140 | train 2.813 | val 2.660 | lr 1.16e-04 | 102,481 tok/s | 54.64 GB | 15.878B tok iter 121160 | train 2.731 | val 2.669 | lr 1.16e-04 | 102,483 tok/s | 54.64 GB | 15.881B tok iter 121180 | train 2.565 | val 2.661 | lr 1.16e-04 | 102,484 tok/s | 54.64 GB | 15.883B tok iter 121200 | train 2.644 | val 2.656 | lr 1.16e-04 | 102,485 tok/s | 54.64 GB | 15.886B tok iter 121220 | train 2.699 | val 2.667 | lr 1.15e-04 | 102,486 tok/s | 54.64 GB | 15.889B tok iter 121240 | train 2.533 | val 2.701 | lr 1.15e-04 | 102,487 tok/s | 54.64 GB | 15.891B tok iter 121260 | train 2.629 | val 2.640 | lr 1.15e-04 | 102,488 tok/s | 54.64 GB | 15.894B tok iter 121280 | train 2.641 | val 2.665 | lr 1.15e-04 | 102,490 tok/s | 54.64 GB | 15.897B tok iter 121300 | train 2.697 | val 2.652 | lr 1.15e-04 | 102,490 tok/s | 54.64 GB | 15.899B tok iter 121320 | train 2.821 | val 2.665 | lr 1.15e-04 | 102,491 tok/s | 54.64 GB | 15.902B tok iter 121340 | train 2.538 | val 2.683 | lr 1.15e-04 | 102,493 tok/s | 54.64 GB | 15.904B tok iter 121360 | train 2.667 | val 2.683 | lr 1.15e-04 | 102,494 tok/s | 54.64 GB | 15.907B tok iter 121380 | train 2.693 | val 2.700 | lr 1.15e-04 | 102,494 tok/s | 54.64 GB | 15.910B tok iter 121400 | train 2.754 | val 2.651 | lr 1.15e-04 | 102,495 tok/s | 54.64 GB | 15.912B tok iter 121420 | train 2.669 | val 2.674 | lr 1.15e-04 | 102,497 tok/s | 54.64 GB | 15.915B tok iter 121440 | train 2.531 | val 2.699 | lr 1.15e-04 | 102,498 tok/s | 54.64 GB | 15.918B tok iter 121460 | train 2.786 | val 2.595 | lr 1.15e-04 | 102,498 tok/s | 54.64 GB | 15.920B tok iter 121480 | train 2.694 | val 2.671 | lr 1.15e-04 | 102,500 tok/s | 54.64 GB | 15.923B tok checkpoint @ iter 121491 -> checkpoints/500m_latest.pt iter 121500 | train 2.534 | val 2.648 | lr 1.14e-04 | 102,492 tok/s | 54.64 GB | 15.925B tok checkpoint @ iter 121500 -> checkpoints/500m_latest.pt iter 121520 | train 2.719 | val 2.694 | lr 1.14e-04 | 102,475 tok/s | 54.64 GB | 15.928B tok iter 121540 | train 2.637 | val 2.720 | lr 1.14e-04 | 102,475 tok/s | 54.64 GB | 15.931B tok iter 121560 | train 2.738 | val 2.665 | lr 1.14e-04 | 102,476 tok/s | 54.64 GB | 15.933B tok iter 121580 | train 2.610 | val 2.685 | lr 1.14e-04 | 102,478 tok/s | 54.64 GB | 15.936B tok iter 121600 | train 2.737 | val 2.673 | lr 1.14e-04 | 102,479 tok/s | 54.64 GB | 15.938B tok iter 121620 | train 2.631 | val 2.685 | lr 1.14e-04 | 102,479 tok/s | 54.64 GB | 15.941B tok iter 121640 | train 2.677 | val 2.661 | lr 1.14e-04 | 102,480 tok/s | 54.64 GB | 15.944B tok iter 121660 | train 2.770 | val 2.644 | lr 1.14e-04 | 102,482 tok/s | 54.64 GB | 15.946B tok iter 121680 | train 2.798 | val 2.687 | lr 1.14e-04 | 102,483 tok/s | 54.64 GB | 15.949B tok iter 121700 | train 2.761 | val 2.659 | lr 1.14e-04 | 102,483 tok/s | 54.64 GB | 15.952B tok iter 121720 | train 2.804 | val 2.682 | lr 1.14e-04 | 102,485 tok/s | 54.64 GB | 15.954B tok iter 121740 | train 2.644 | val 2.639 | lr 1.14e-04 | 102,486 tok/s | 54.64 GB | 15.957B tok iter 121760 | train 2.658 | val 2.624 | lr 1.14e-04 | 102,487 tok/s | 54.64 GB | 15.959B tok iter 121780 | train 2.716 | val 2.638 | lr 1.14e-04 | 102,488 tok/s | 54.64 GB | 15.962B tok iter 121800 | train 2.649 | val 2.662 | lr 1.13e-04 | 102,489 tok/s | 54.64 GB | 15.965B tok iter 121820 | train 2.636 | val 2.697 | lr 1.13e-04 | 102,490 tok/s | 54.64 GB | 15.967B tok iter 121840 | train 2.764 | val 2.672 | lr 1.13e-04 | 102,492 tok/s | 54.64 GB | 15.970B tok iter 121860 | train 2.629 | val 2.696 | lr 1.13e-04 | 102,492 tok/s | 54.64 GB | 15.973B tok iter 121880 | train 2.674 | val 2.658 | lr 1.13e-04 | 102,493 tok/s | 54.64 GB | 15.975B tok iter 121900 | train 2.802 | val 2.649 | lr 1.13e-04 | 102,495 tok/s | 54.64 GB | 15.978B tok iter 121920 | train 2.694 | val 2.668 | lr 1.13e-04 | 102,496 tok/s | 54.64 GB | 15.980B tok iter 121940 | train 2.617 | val 2.678 | lr 1.13e-04 | 102,496 tok/s | 54.64 GB | 15.983B tok iter 121960 | train 2.612 | val 2.661 | lr 1.13e-04 | 102,497 tok/s | 54.64 GB | 15.986B tok iter 121980 | train 2.520 | val 2.692 | lr 1.13e-04 | 102,499 tok/s | 54.64 GB | 15.988B tok checkpoint @ iter 121990 -> checkpoints/500m_latest.pt iter 122000 | train 2.684 | val 2.645 | lr 1.13e-04 | 102,492 tok/s | 54.64 GB | 15.991B tok checkpoint @ iter 122000 -> checkpoints/500m_latest.pt iter 122020 | train 2.635 | val 2.662 | lr 1.13e-04 | 102,474 tok/s | 54.64 GB | 15.994B tok iter 122040 | train 2.515 | val 2.632 | lr 1.13e-04 | 102,475 tok/s | 54.64 GB | 15.996B tok iter 122060 | train 2.672 | val 2.601 | lr 1.13e-04 | 102,476 tok/s | 54.64 GB | 15.999B tok iter 122080 | train 2.644 | val 2.687 | lr 1.13e-04 | 102,478 tok/s | 54.64 GB | 16.001B tok iter 122100 | train 2.695 | val 2.698 | lr 1.12e-04 | 102,478 tok/s | 54.64 GB | 16.004B tok iter 122120 | train 2.796 | val 2.683 | lr 1.12e-04 | 102,479 tok/s | 54.64 GB | 16.007B tok iter 122140 | train 2.690 | val 2.643 | lr 1.12e-04 | 102,481 tok/s | 54.64 GB | 16.009B tok iter 122160 | train 2.539 | val 2.695 | lr 1.12e-04 | 102,482 tok/s | 54.64 GB | 16.012B tok iter 122180 | train 2.704 | val 2.703 | lr 1.12e-04 | 102,482 tok/s | 54.64 GB | 16.015B tok iter 122200 | train 2.623 | val 2.665 | lr 1.12e-04 | 102,484 tok/s | 54.64 GB | 16.017B tok iter 122220 | train 2.744 | val 2.632 | lr 1.12e-04 | 102,485 tok/s | 54.64 GB | 16.020B tok iter 122240 | train 2.727 | val 2.649 | lr 1.12e-04 | 102,486 tok/s | 54.64 GB | 16.022B tok iter 122260 | train 2.642 | val 2.661 | lr 1.12e-04 | 102,486 tok/s | 54.64 GB | 16.025B tok iter 122280 | train 2.643 | val 2.706 | lr 1.12e-04 | 102,488 tok/s | 54.64 GB | 16.028B tok iter 122300 | train 2.787 | val 2.622 | lr 1.12e-04 | 102,489 tok/s | 54.64 GB | 16.030B tok iter 122320 | train 2.634 | val 2.664 | lr 1.12e-04 | 102,490 tok/s | 54.64 GB | 16.033B tok iter 122340 | train 2.572 | val 2.722 | lr 1.12e-04 | 102,491 tok/s | 54.64 GB | 16.035B tok iter 122360 | train 2.591 | val 2.709 | lr 1.12e-04 | 102,492 tok/s | 54.64 GB | 16.038B tok iter 122380 | train 2.605 | val 2.677 | lr 1.12e-04 | 102,493 tok/s | 54.64 GB | 16.041B tok iter 122400 | train 2.501 | val 2.655 | lr 1.11e-04 | 102,495 tok/s | 54.64 GB | 16.043B tok iter 122420 | train 2.692 | val 2.658 | lr 1.11e-04 | 102,495 tok/s | 54.64 GB | 16.046B tok iter 122440 | train 2.702 | val 2.645 | lr 1.11e-04 | 102,496 tok/s | 54.64 GB | 16.049B tok iter 122460 | train 2.721 | val 2.684 | lr 1.11e-04 | 102,497 tok/s | 54.64 GB | 16.051B tok iter 122480 | train 2.770 | val 2.684 | lr 1.11e-04 | 102,499 tok/s | 54.64 GB | 16.054B tok checkpoint @ iter 122490 -> checkpoints/500m_latest.pt iter 122500 | train 2.740 | val 2.680 | lr 1.11e-04 | 102,490 tok/s | 54.64 GB | 16.056B tok checkpoint @ iter 122500 -> checkpoints/500m_latest.pt iter 122520 | train 2.780 | val 2.693 | lr 1.11e-04 | 102,474 tok/s | 54.64 GB | 16.059B tok iter 122540 | train 2.607 | val 2.688 | lr 1.11e-04 | 102,476 tok/s | 54.64 GB | 16.062B tok iter 122560 | train 2.624 | val 2.640 | lr 1.11e-04 | 102,477 tok/s | 54.64 GB | 16.064B tok iter 122580 | train 2.619 | val 2.701 | lr 1.11e-04 | 102,477 tok/s | 54.64 GB | 16.067B tok iter 122600 | train 2.691 | val 2.632 | lr 1.11e-04 | 102,479 tok/s | 54.64 GB | 16.070B tok iter 122620 | train 2.582 | val 2.677 | lr 1.11e-04 | 102,480 tok/s | 54.64 GB | 16.072B tok iter 122640 | train 2.482 | val 2.638 | lr 1.11e-04 | 102,481 tok/s | 54.64 GB | 16.075B tok iter 122660 | train 2.648 | val 2.671 | lr 1.11e-04 | 102,482 tok/s | 54.64 GB | 16.077B tok iter 122680 | train 2.632 | val 2.647 | lr 1.11e-04 | 102,483 tok/s | 54.64 GB | 16.080B tok iter 122700 | train 2.670 | val 2.696 | lr 1.10e-04 | 102,484 tok/s | 54.64 GB | 16.083B tok iter 122720 | train 2.665 | val 2.649 | lr 1.10e-04 | 102,485 tok/s | 54.64 GB | 16.085B tok iter 122740 | train 2.696 | val 2.707 | lr 1.10e-04 | 102,486 tok/s | 54.64 GB | 16.088B tok iter 122760 | train 2.585 | val 2.661 | lr 1.10e-04 | 102,487 tok/s | 54.64 GB | 16.091B tok iter 122780 | train 2.795 | val 2.631 | lr 1.10e-04 | 102,488 tok/s | 54.64 GB | 16.093B tok iter 122800 | train 2.667 | val 2.641 | lr 1.10e-04 | 102,490 tok/s | 54.64 GB | 16.096B tok iter 122820 | train 2.652 | val 2.660 | lr 1.10e-04 | 102,490 tok/s | 54.64 GB | 16.098B tok iter 122840 | train 2.752 | val 2.699 | lr 1.10e-04 | 102,491 tok/s | 54.64 GB | 16.101B tok iter 122860 | train 2.698 | val 2.654 | lr 1.10e-04 | 102,492 tok/s | 54.64 GB | 16.104B tok iter 122880 | train 2.642 | val 2.657 | lr 1.10e-04 | 102,494 tok/s | 54.64 GB | 16.106B tok iter 122900 | train 2.670 | val 2.652 | lr 1.10e-04 | 102,494 tok/s | 54.64 GB | 16.109B tok iter 122920 | train 2.580 | val 2.677 | lr 1.10e-04 | 102,495 tok/s | 54.64 GB | 16.112B tok iter 122940 | train 2.686 | val 2.688 | lr 1.10e-04 | 102,497 tok/s | 54.64 GB | 16.114B tok iter 122960 | train 2.624 | val 2.661 | lr 1.10e-04 | 102,498 tok/s | 54.64 GB | 16.117B tok iter 122980 | train 2.621 | val 2.680 | lr 1.10e-04 | 102,498 tok/s | 54.64 GB | 16.119B tok checkpoint @ iter 122991 -> checkpoints/500m_latest.pt iter 123000 | train 2.619 | val 2.629 | lr 1.10e-04 | 102,491 tok/s | 54.64 GB | 16.122B tok checkpoint @ iter 123000 -> checkpoints/500m_latest.pt iter 123020 | train 2.652 | val 2.673 | lr 1.09e-04 | 102,474 tok/s | 54.64 GB | 16.125B tok iter 123040 | train 2.620 | val 2.645 | lr 1.09e-04 | 102,476 tok/s | 54.64 GB | 16.127B tok iter 123060 | train 2.622 | val 2.675 | lr 1.09e-04 | 102,476 tok/s | 54.64 GB | 16.130B tok iter 123080 | train 2.677 | val 2.666 | lr 1.09e-04 | 102,477 tok/s | 54.64 GB | 16.132B tok iter 123100 | train 2.658 | val 2.657 | lr 1.09e-04 | 102,478 tok/s | 54.64 GB | 16.135B tok iter 123120 | train 2.542 | val 2.652 | lr 1.09e-04 | 102,480 tok/s | 54.64 GB | 16.138B tok iter 123140 | train 2.737 | val 2.698 | lr 1.09e-04 | 102,480 tok/s | 54.64 GB | 16.140B tok iter 123160 | train 2.599 | val 2.667 | lr 1.09e-04 | 102,481 tok/s | 54.64 GB | 16.143B tok iter 123180 | train 2.625 | val 2.672 | lr 1.09e-04 | 102,483 tok/s | 54.64 GB | 16.146B tok iter 123200 | train 2.757 | val 2.660 | lr 1.09e-04 | 102,484 tok/s | 54.64 GB | 16.148B tok iter 123220 | train 2.645 | val 2.689 | lr 1.09e-04 | 102,484 tok/s | 54.64 GB | 16.151B tok iter 123240 | train 2.583 | val 2.640 | lr 1.09e-04 | 102,486 tok/s | 54.64 GB | 16.153B tok iter 123260 | train 2.589 | val 2.607 | lr 1.09e-04 | 102,487 tok/s | 54.64 GB | 16.156B tok iter 123280 | train 2.633 | val 2.647 | lr 1.09e-04 | 102,488 tok/s | 54.64 GB | 16.159B tok iter 123300 | train 2.804 | val 2.648 | lr 1.09e-04 | 102,488 tok/s | 54.64 GB | 16.161B tok iter 123320 | train 2.739 | val 2.650 | lr 1.08e-04 | 102,490 tok/s | 54.64 GB | 16.164B tok iter 123340 | train 2.511 | val 2.674 | lr 1.08e-04 | 102,491 tok/s | 54.64 GB | 16.167B tok iter 123360 | train 2.696 | val 2.682 | lr 1.08e-04 | 102,492 tok/s | 54.64 GB | 16.169B tok iter 123380 | train 2.728 | val 2.687 | lr 1.08e-04 | 102,492 tok/s | 54.64 GB | 16.172B tok iter 123400 | train 2.670 | val 2.650 | lr 1.08e-04 | 102,494 tok/s | 54.64 GB | 16.174B tok iter 123420 | train 2.570 | val 2.631 | lr 1.08e-04 | 102,495 tok/s | 54.64 GB | 16.177B tok iter 123440 | train 2.571 | val 2.684 | lr 1.08e-04 | 102,496 tok/s | 54.64 GB | 16.180B tok iter 123460 | train 2.611 | val 2.650 | lr 1.08e-04 | 102,497 tok/s | 54.64 GB | 16.182B tok iter 123480 | train 2.531 | val 2.653 | lr 1.08e-04 | 102,498 tok/s | 54.64 GB | 16.185B tok checkpoint @ iter 123490 -> checkpoints/500m_latest.pt iter 123500 | train 2.698 | val 2.679 | lr 1.08e-04 | 102,490 tok/s | 54.64 GB | 16.188B tok checkpoint @ iter 123500 -> checkpoints/500m_latest.pt iter 123520 | train 2.647 | val 2.672 | lr 1.08e-04 | 102,474 tok/s | 54.64 GB | 16.190B tok iter 123540 | train 2.732 | val 2.666 | lr 1.08e-04 | 102,475 tok/s | 54.64 GB | 16.193B tok iter 123560 | train 2.799 | val 2.622 | lr 1.08e-04 | 102,476 tok/s | 54.64 GB | 16.195B tok iter 123580 | train 2.548 | val 2.660 | lr 1.08e-04 | 102,477 tok/s | 54.64 GB | 16.198B tok iter 123600 | train 2.517 | val 2.693 | lr 1.08e-04 | 102,478 tok/s | 54.64 GB | 16.201B tok iter 123620 | train 2.575 | val 2.703 | lr 1.08e-04 | 102,479 tok/s | 54.64 GB | 16.203B tok iter 123640 | train 2.640 | val 2.650 | lr 1.07e-04 | 102,480 tok/s | 54.64 GB | 16.206B tok iter 123660 | train 2.642 | val 2.688 | lr 1.07e-04 | 102,481 tok/s | 54.64 GB | 16.208B tok iter 123680 | train 2.607 | val 2.619 | lr 1.07e-04 | 102,483 tok/s | 54.64 GB | 16.211B tok iter 123700 | train 2.684 | val 2.630 | lr 1.07e-04 | 102,483 tok/s | 54.64 GB | 16.214B tok iter 123720 | train 2.647 | val 2.651 | lr 1.07e-04 | 102,484 tok/s | 54.64 GB | 16.216B tok iter 123740 | train 2.712 | val 2.644 | lr 1.07e-04 | 102,485 tok/s | 54.64 GB | 16.219B tok iter 123760 | train 2.735 | val 2.632 | lr 1.07e-04 | 102,487 tok/s | 54.64 GB | 16.222B tok iter 123780 | train 2.612 | val 2.653 | lr 1.07e-04 | 102,487 tok/s | 54.64 GB | 16.224B tok iter 123800 | train 2.728 | val 2.718 | lr 1.07e-04 | 102,488 tok/s | 54.64 GB | 16.227B tok iter 123820 | train 2.683 | val 2.693 | lr 1.07e-04 | 102,490 tok/s | 54.64 GB | 16.229B tok iter 123840 | train 2.628 | val 2.660 | lr 1.07e-04 | 102,491 tok/s | 54.64 GB | 16.232B tok iter 123860 | train 2.679 | val 2.640 | lr 1.07e-04 | 102,491 tok/s | 54.64 GB | 16.235B tok iter 123880 | train 2.602 | val 2.667 | lr 1.07e-04 | 102,492 tok/s | 54.64 GB | 16.237B tok iter 123900 | train 2.608 | val 2.638 | lr 1.07e-04 | 102,494 tok/s | 54.64 GB | 16.240B tok iter 123920 | train 2.516 | val 2.622 | lr 1.07e-04 | 102,495 tok/s | 54.64 GB | 16.243B tok iter 123940 | train 2.795 | val 2.694 | lr 1.07e-04 | 102,495 tok/s | 54.64 GB | 16.245B tok iter 123960 | train 2.659 | val 2.660 | lr 1.06e-04 | 102,497 tok/s | 54.64 GB | 16.248B tok iter 123980 | train 2.777 | val 2.637 | lr 1.06e-04 | 102,498 tok/s | 54.64 GB | 16.250B tok checkpoint @ iter 123990 -> checkpoints/500m_latest.pt iter 124000 | train 2.554 | val 2.660 | lr 1.06e-04 | 102,490 tok/s | 54.64 GB | 16.253B tok checkpoint @ iter 124000 -> checkpoints/500m_latest.pt iter 124020 | train 2.679 | val 2.661 | lr 1.06e-04 | 102,473 tok/s | 54.64 GB | 16.256B tok iter 124040 | train 2.730 | val 2.637 | lr 1.06e-04 | 102,475 tok/s | 54.64 GB | 16.258B tok iter 124060 | train 2.674 | val 2.671 | lr 1.06e-04 | 102,476 tok/s | 54.64 GB | 16.261B tok iter 124080 | train 2.754 | val 2.655 | lr 1.06e-04 | 102,477 tok/s | 54.64 GB | 16.264B tok iter 124100 | train 2.691 | val 2.676 | lr 1.06e-04 | 102,477 tok/s | 54.64 GB | 16.266B tok iter 124120 | train 2.513 | val 2.645 | lr 1.06e-04 | 102,479 tok/s | 54.64 GB | 16.269B tok iter 124140 | train 2.607 | val 2.647 | lr 1.06e-04 | 102,480 tok/s | 54.64 GB | 16.271B tok iter 124160 | train 2.536 | val 2.683 | lr 1.06e-04 | 102,481 tok/s | 54.64 GB | 16.274B tok iter 124180 | train 2.699 | val 2.702 | lr 1.06e-04 | 102,481 tok/s | 54.64 GB | 16.277B tok iter 124200 | train 2.530 | val 2.688 | lr 1.06e-04 | 102,483 tok/s | 54.64 GB | 16.279B tok iter 124220 | train 2.567 | val 2.670 | lr 1.06e-04 | 102,484 tok/s | 54.64 GB | 16.282B tok iter 124240 | train 2.677 | val 2.691 | lr 1.06e-04 | 102,485 tok/s | 54.64 GB | 16.285B tok iter 124260 | train 2.521 | val 2.661 | lr 1.06e-04 | 102,486 tok/s | 54.64 GB | 16.287B tok iter 124280 | train 2.668 | val 2.703 | lr 1.05e-04 | 102,487 tok/s | 54.64 GB | 16.290B tok iter 124300 | train 2.675 | val 2.660 | lr 1.05e-04 | 102,488 tok/s | 54.64 GB | 16.292B tok iter 124320 | train 2.621 | val 2.650 | lr 1.05e-04 | 102,489 tok/s | 54.64 GB | 16.295B tok iter 124340 | train 2.636 | val 2.631 | lr 1.05e-04 | 102,490 tok/s | 54.64 GB | 16.298B tok iter 124360 | train 2.707 | val 2.634 | lr 1.05e-04 | 102,491 tok/s | 54.64 GB | 16.300B tok iter 124380 | train 2.666 | val 2.654 | lr 1.05e-04 | 102,492 tok/s | 54.64 GB | 16.303B tok iter 124400 | train 2.637 | val 2.669 | lr 1.05e-04 | 102,494 tok/s | 54.64 GB | 16.305B tok iter 124420 | train 2.634 | val 2.664 | lr 1.05e-04 | 102,494 tok/s | 54.64 GB | 16.308B tok iter 124440 | train 2.522 | val 2.708 | lr 1.05e-04 | 102,495 tok/s | 54.64 GB | 16.311B tok iter 124460 | train 2.599 | val 2.640 | lr 1.05e-04 | 102,496 tok/s | 54.64 GB | 16.313B tok iter 124480 | train 2.668 | val 2.655 | lr 1.05e-04 | 102,498 tok/s | 54.64 GB | 16.316B tok checkpoint @ iter 124490 -> checkpoints/500m_latest.pt iter 124500 | train 2.645 | val 2.669 | lr 1.05e-04 | 102,490 tok/s | 54.64 GB | 16.319B tok checkpoint @ iter 124500 -> checkpoints/500m_latest.pt iter 124520 | train 2.617 | val 2.658 | lr 1.05e-04 | 102,474 tok/s | 54.64 GB | 16.321B tok iter 124540 | train 2.752 | val 2.679 | lr 1.05e-04 | 102,476 tok/s | 54.64 GB | 16.324B tok iter 124560 | train 2.729 | val 2.639 | lr 1.05e-04 | 102,477 tok/s | 54.64 GB | 16.326B tok iter 124580 | train 2.486 | val 2.638 | lr 1.05e-04 | 102,477 tok/s | 54.64 GB | 16.329B tok iter 124600 | train 2.640 | val 2.712 | lr 1.04e-04 | 102,479 tok/s | 54.64 GB | 16.332B tok iter 124620 | train 2.601 | val 2.655 | lr 1.04e-04 | 102,480 tok/s | 54.64 GB | 16.334B tok iter 124640 | train 2.503 | val 2.614 | lr 1.04e-04 | 102,481 tok/s | 54.64 GB | 16.337B tok iter 124660 | train 2.540 | val 2.644 | lr 1.04e-04 | 102,481 tok/s | 54.64 GB | 16.340B tok iter 124680 | train 2.617 | val 2.673 | lr 1.04e-04 | 102,483 tok/s | 54.64 GB | 16.342B tok iter 124700 | train 2.604 | val 2.683 | lr 1.04e-04 | 102,484 tok/s | 54.64 GB | 16.345B tok iter 124720 | train 2.716 | val 2.680 | lr 1.04e-04 | 102,485 tok/s | 54.64 GB | 16.347B tok iter 124740 | train 2.663 | val 2.671 | lr 1.04e-04 | 102,485 tok/s | 54.64 GB | 16.350B tok iter 124760 | train 2.660 | val 2.655 | lr 1.04e-04 | 102,487 tok/s | 54.64 GB | 16.353B tok iter 124780 | train 2.576 | val 2.654 | lr 1.04e-04 | 102,488 tok/s | 54.64 GB | 16.355B tok iter 124800 | train 2.815 | val 2.674 | lr 1.04e-04 | 102,489 tok/s | 54.64 GB | 16.358B tok iter 124820 | train 2.671 | val 2.672 | lr 1.04e-04 | 102,490 tok/s | 54.64 GB | 16.361B tok iter 124840 | train 2.668 | val 2.653 | lr 1.04e-04 | 102,491 tok/s | 54.64 GB | 16.363B tok iter 124860 | train 2.580 | val 2.693 | lr 1.04e-04 | 102,492 tok/s | 54.64 GB | 16.366B tok iter 124880 | train 2.617 | val 2.649 | lr 1.04e-04 | 102,493 tok/s | 54.64 GB | 16.368B tok iter 124900 | train 2.597 | val 2.658 | lr 1.04e-04 | 102,494 tok/s | 54.64 GB | 16.371B tok iter 124920 | train 2.556 | val 2.679 | lr 1.03e-04 | 102,495 tok/s | 54.64 GB | 16.374B tok iter 124940 | train 2.666 | val 2.651 | lr 1.03e-04 | 102,496 tok/s | 54.64 GB | 16.376B tok iter 124960 | train 2.670 | val 2.643 | lr 1.03e-04 | 102,497 tok/s | 54.64 GB | 16.379B tok iter 124980 | train 2.465 | val 2.631 | lr 1.03e-04 | 102,498 tok/s | 54.64 GB | 16.382B tok checkpoint @ iter 124991 -> checkpoints/500m_latest.pt iter 125000 | train 2.470 | val 2.621 | lr 1.03e-04 | 102,490 tok/s | 54.64 GB | 16.384B tok checkpoint @ iter 125000 -> checkpoints/500m_latest.pt iter 125020 | train 2.659 | val 2.672 | lr 1.03e-04 | 102,475 tok/s | 54.64 GB | 16.387B tok iter 125040 | train 2.663 | val 2.650 | lr 1.03e-04 | 102,477 tok/s | 54.64 GB | 16.389B tok iter 125060 | train 2.791 | val 2.694 | lr 1.03e-04 | 102,477 tok/s | 54.64 GB | 16.392B tok iter 125080 | train 2.540 | val 2.693 | lr 1.03e-04 | 102,478 tok/s | 54.64 GB | 16.395B tok iter 125100 | train 2.801 | val 2.658 | lr 1.03e-04 | 102,479 tok/s | 54.64 GB | 16.397B tok iter 125120 | train 2.665 | val 2.670 | lr 1.03e-04 | 102,481 tok/s | 54.64 GB | 16.400B tok iter 125140 | train 2.775 | val 2.617 | lr 1.03e-04 | 102,481 tok/s | 54.64 GB | 16.402B tok iter 125160 | train 2.514 | val 2.629 | lr 1.03e-04 | 102,482 tok/s | 54.64 GB | 16.405B tok iter 125180 | train 2.507 | val 2.691 | lr 1.03e-04 | 102,483 tok/s | 54.64 GB | 16.408B tok iter 125200 | train 2.610 | val 2.686 | lr 1.03e-04 | 102,485 tok/s | 54.64 GB | 16.410B tok iter 125220 | train 2.685 | val 2.661 | lr 1.03e-04 | 102,485 tok/s | 54.64 GB | 16.413B tok iter 125240 | train 2.589 | val 2.631 | lr 1.03e-04 | 102,486 tok/s | 54.64 GB | 16.416B tok iter 125260 | train 2.575 | val 2.640 | lr 1.02e-04 | 102,488 tok/s | 54.64 GB | 16.418B tok iter 125280 | train 2.670 | val 2.665 | lr 1.02e-04 | 102,489 tok/s | 54.64 GB | 16.421B tok iter 125300 | train 2.693 | val 2.681 | lr 1.02e-04 | 102,489 tok/s | 54.64 GB | 16.423B tok iter 125320 | train 2.657 | val 2.684 | lr 1.02e-04 | 102,490 tok/s | 54.64 GB | 16.426B tok iter 125340 | train 2.650 | val 2.602 | lr 1.02e-04 | 102,492 tok/s | 54.64 GB | 16.429B tok iter 125360 | train 2.573 | val 2.677 | lr 1.02e-04 | 102,493 tok/s | 54.64 GB | 16.431B tok iter 125380 | train 2.512 | val 2.671 | lr 1.02e-04 | 102,493 tok/s | 54.64 GB | 16.434B tok iter 125400 | train 2.679 | val 2.635 | lr 1.02e-04 | 102,494 tok/s | 54.64 GB | 16.437B tok iter 125420 | train 2.588 | val 2.693 | lr 1.02e-04 | 102,496 tok/s | 54.64 GB | 16.439B tok iter 125440 | train 2.683 | val 2.654 | lr 1.02e-04 | 102,497 tok/s | 54.64 GB | 16.442B tok iter 125460 | train 2.623 | val 2.662 | lr 1.02e-04 | 102,497 tok/s | 54.64 GB | 16.444B tok iter 125480 | train 2.693 | val 2.638 | lr 1.02e-04 | 102,498 tok/s | 54.64 GB | 16.447B tok checkpoint @ iter 125490 -> checkpoints/500m_latest.pt iter 125500 | train 2.704 | val 2.647 | lr 1.02e-04 | 102,491 tok/s | 54.64 GB | 16.450B tok checkpoint @ iter 125500 -> checkpoints/500m_latest.pt iter 125520 | train 2.617 | val 2.679 | lr 1.02e-04 | 102,477 tok/s | 54.64 GB | 16.452B tok iter 125540 | train 2.711 | val 2.588 | lr 1.02e-04 | 102,477 tok/s | 54.64 GB | 16.455B tok iter 125560 | train 2.710 | val 2.634 | lr 1.02e-04 | 102,478 tok/s | 54.64 GB | 16.458B tok iter 125580 | train 2.595 | val 2.628 | lr 1.01e-04 | 102,480 tok/s | 54.64 GB | 16.460B tok iter 125600 | train 2.727 | val 2.645 | lr 1.01e-04 | 102,481 tok/s | 54.64 GB | 16.463B tok iter 125620 | train 2.673 | val 2.679 | lr 1.01e-04 | 102,481 tok/s | 54.64 GB | 16.465B tok iter 125640 | train 2.705 | val 2.683 | lr 1.01e-04 | 102,482 tok/s | 54.64 GB | 16.468B tok iter 125660 | train 2.626 | val 2.627 | lr 1.01e-04 | 102,484 tok/s | 54.64 GB | 16.471B tok iter 125680 | train 2.704 | val 2.649 | lr 1.01e-04 | 102,485 tok/s | 54.64 GB | 16.473B tok iter 125700 | train 2.464 | val 2.707 | lr 1.01e-04 | 102,485 tok/s | 54.64 GB | 16.476B tok iter 125720 | train 2.512 | val 2.634 | lr 1.01e-04 | 102,486 tok/s | 54.64 GB | 16.479B tok iter 125740 | train 2.715 | val 2.608 | lr 1.01e-04 | 102,488 tok/s | 54.64 GB | 16.481B tok iter 125760 | train 2.701 | val 2.694 | lr 1.01e-04 | 102,489 tok/s | 54.64 GB | 16.484B tok iter 125780 | train 2.645 | val 2.666 | lr 1.01e-04 | 102,489 tok/s | 54.64 GB | 16.486B tok iter 125800 | train 2.577 | val 2.646 | lr 1.01e-04 | 102,490 tok/s | 54.64 GB | 16.489B tok iter 125820 | train 2.543 | val 2.692 | lr 1.01e-04 | 102,492 tok/s | 54.64 GB | 16.492B tok iter 125840 | train 2.568 | val 2.630 | lr 1.01e-04 | 102,493 tok/s | 54.64 GB | 16.494B tok iter 125860 | train 2.564 | val 2.647 | lr 1.01e-04 | 102,493 tok/s | 54.64 GB | 16.497B tok iter 125880 | train 2.529 | val 2.628 | lr 1.01e-04 | 102,494 tok/s | 54.64 GB | 16.499B tok iter 125900 | train 2.661 | val 2.633 | lr 1.01e-04 | 102,496 tok/s | 54.64 GB | 16.502B tok iter 125920 | train 2.509 | val 2.643 | lr 1.00e-04 | 102,497 tok/s | 54.64 GB | 16.505B tok iter 125940 | train 2.747 | val 2.637 | lr 1.00e-04 | 102,497 tok/s | 54.64 GB | 16.507B tok iter 125960 | train 2.665 | val 2.686 | lr 1.00e-04 | 102,498 tok/s | 54.64 GB | 16.510B tok iter 125980 | train 2.704 | val 2.671 | lr 1.00e-04 | 102,500 tok/s | 54.64 GB | 16.513B tok checkpoint @ iter 125991 -> checkpoints/500m_latest.pt iter 126000 | train 2.662 | val 2.612 | lr 1.00e-04 | 102,493 tok/s | 54.64 GB | 16.515B tok checkpoint @ iter 126000 -> checkpoints/500m_latest.pt iter 126020 | train 2.630 | val 2.676 | lr 1.00e-04 | 102,477 tok/s | 54.64 GB | 16.518B tok iter 126040 | train 2.518 | val 2.655 | lr 1.00e-04 | 102,478 tok/s | 54.64 GB | 16.520B tok iter 126060 | train 2.627 | val 2.639 | lr 1.00e-04 | 102,479 tok/s | 54.64 GB | 16.523B tok iter 126080 | train 2.637 | val 2.630 | lr 1.00e-04 | 102,481 tok/s | 54.64 GB | 16.526B tok iter 126100 | train 2.713 | val 2.628 | lr 9.99e-05 | 102,481 tok/s | 54.64 GB | 16.528B tok iter 126120 | train 2.606 | val 2.658 | lr 9.99e-05 | 102,482 tok/s | 54.64 GB | 16.531B tok iter 126140 | train 2.525 | val 2.707 | lr 9.98e-05 | 102,483 tok/s | 54.64 GB | 16.534B tok iter 126160 | train 2.715 | val 2.674 | lr 9.98e-05 | 102,485 tok/s | 54.64 GB | 16.536B tok iter 126180 | train 2.686 | val 2.641 | lr 9.97e-05 | 102,485 tok/s | 54.64 GB | 16.539B tok iter 126200 | train 2.676 | val 2.661 | lr 9.96e-05 | 102,486 tok/s | 54.64 GB | 16.541B tok iter 126220 | train 2.503 | val 2.637 | lr 9.96e-05 | 102,487 tok/s | 54.64 GB | 16.544B tok iter 126240 | train 2.626 | val 2.684 | lr 9.95e-05 | 102,489 tok/s | 54.64 GB | 16.547B tok iter 126260 | train 2.466 | val 2.698 | lr 9.95e-05 | 102,489 tok/s | 54.64 GB | 16.549B tok iter 126280 | train 2.705 | val 2.658 | lr 9.94e-05 | 102,490 tok/s | 54.64 GB | 16.552B tok iter 126300 | train 2.579 | val 2.631 | lr 9.94e-05 | 102,491 tok/s | 54.64 GB | 16.555B tok iter 126320 | train 2.659 | val 2.620 | lr 9.93e-05 | 102,493 tok/s | 54.64 GB | 16.557B tok iter 126340 | train 2.677 | val 2.667 | lr 9.92e-05 | 102,493 tok/s | 54.64 GB | 16.560B tok iter 126360 | train 2.589 | val 2.682 | lr 9.92e-05 | 102,494 tok/s | 54.64 GB | 16.562B tok iter 126380 | train 2.684 | val 2.680 | lr 9.91e-05 | 102,495 tok/s | 54.64 GB | 16.565B tok iter 126400 | train 2.681 | val 2.632 | lr 9.91e-05 | 102,497 tok/s | 54.64 GB | 16.568B tok iter 126420 | train 2.673 | val 2.629 | lr 9.90e-05 | 102,497 tok/s | 54.64 GB | 16.570B tok iter 126440 | train 2.661 | val 2.674 | lr 9.89e-05 | 102,498 tok/s | 54.64 GB | 16.573B tok iter 126460 | train 2.725 | val 2.608 | lr 9.89e-05 | 102,499 tok/s | 54.64 GB | 16.575B tok iter 126480 | train 2.761 | val 2.684 | lr 9.88e-05 | 102,501 tok/s | 54.64 GB | 16.578B tok checkpoint @ iter 126491 -> checkpoints/500m_latest.pt iter 126500 | train 2.746 | val 2.662 | lr 9.88e-05 | 102,493 tok/s | 54.64 GB | 16.581B tok checkpoint @ iter 126500 -> checkpoints/500m_latest.pt iter 126520 | train 2.655 | val 2.669 | lr 9.87e-05 | 102,478 tok/s | 54.64 GB | 16.583B tok iter 126540 | train 2.802 | val 2.646 | lr 9.87e-05 | 102,479 tok/s | 54.64 GB | 16.586B tok iter 126560 | train 2.797 | val 2.653 | lr 9.86e-05 | 102,481 tok/s | 54.64 GB | 16.589B tok iter 126580 | train 2.686 | val 2.639 | lr 9.85e-05 | 102,481 tok/s | 54.64 GB | 16.591B tok iter 126600 | train 2.620 | val 2.645 | lr 9.85e-05 | 102,482 tok/s | 54.64 GB | 16.594B tok iter 126620 | train 2.644 | val 2.668 | lr 9.84e-05 | 102,483 tok/s | 54.64 GB | 16.596B tok iter 126640 | train 2.687 | val 2.605 | lr 9.84e-05 | 102,485 tok/s | 54.64 GB | 16.599B tok iter 126660 | train 2.659 | val 2.635 | lr 9.83e-05 | 102,485 tok/s | 54.64 GB | 16.602B tok iter 126680 | train 2.670 | val 2.613 | lr 9.83e-05 | 102,486 tok/s | 54.64 GB | 16.604B tok iter 126700 | train 2.729 | val 2.615 | lr 9.82e-05 | 102,487 tok/s | 54.64 GB | 16.607B tok iter 126720 | train 2.718 | val 2.669 | lr 9.81e-05 | 102,488 tok/s | 54.64 GB | 16.610B tok iter 126740 | train 2.614 | val 2.709 | lr 9.81e-05 | 102,489 tok/s | 54.64 GB | 16.612B tok iter 126760 | train 2.642 | val 2.595 | lr 9.80e-05 | 102,490 tok/s | 54.64 GB | 16.615B tok iter 126780 | train 2.616 | val 2.684 | lr 9.80e-05 | 102,491 tok/s | 54.64 GB | 16.617B tok iter 126800 | train 2.681 | val 2.671 | lr 9.79e-05 | 102,492 tok/s | 54.64 GB | 16.620B tok iter 126820 | train 2.739 | val 2.653 | lr 9.78e-05 | 102,493 tok/s | 54.64 GB | 16.623B tok iter 126840 | train 2.664 | val 2.689 | lr 9.78e-05 | 102,494 tok/s | 54.64 GB | 16.625B tok iter 126860 | train 2.735 | val 2.702 | lr 9.77e-05 | 102,495 tok/s | 54.64 GB | 16.628B tok iter 126880 | train 2.752 | val 2.673 | lr 9.77e-05 | 102,496 tok/s | 54.64 GB | 16.631B tok iter 126900 | train 2.791 | val 2.608 | lr 9.76e-05 | 102,497 tok/s | 54.64 GB | 16.633B tok iter 126920 | train 2.722 | val 2.629 | lr 9.76e-05 | 102,498 tok/s | 54.64 GB | 16.636B tok iter 126940 | train 2.522 | val 2.648 | lr 9.75e-05 | 102,499 tok/s | 54.64 GB | 16.638B tok iter 126960 | train 2.752 | val 2.616 | lr 9.74e-05 | 102,500 tok/s | 54.64 GB | 16.641B tok iter 126980 | train 2.601 | val 2.665 | lr 9.74e-05 | 102,501 tok/s | 54.64 GB | 16.644B tok checkpoint @ iter 126990 -> checkpoints/500m_latest.pt iter 127000 | train 2.569 | val 2.633 | lr 9.73e-05 | 102,494 tok/s | 54.64 GB | 16.646B tok checkpoint @ iter 127000 -> checkpoints/500m_latest.pt iter 127020 | train 2.534 | val 2.627 | lr 9.73e-05 | 102,479 tok/s | 54.64 GB | 16.649B tok iter 127040 | train 2.859 | val 2.662 | lr 9.72e-05 | 102,480 tok/s | 54.64 GB | 16.652B tok iter 127060 | train 2.518 | val 2.662 | lr 9.72e-05 | 102,481 tok/s | 54.64 GB | 16.654B tok iter 127080 | train 2.659 | val 2.622 | lr 9.71e-05 | 102,482 tok/s | 54.64 GB | 16.657B tok iter 127100 | train 2.644 | val 2.645 | lr 9.70e-05 | 102,483 tok/s | 54.64 GB | 16.659B tok iter 127120 | train 2.631 | val 2.600 | lr 9.70e-05 | 102,484 tok/s | 54.64 GB | 16.662B tok iter 127140 | train 2.555 | val 2.683 | lr 9.69e-05 | 102,485 tok/s | 54.64 GB | 16.665B tok iter 127160 | train 2.636 | val 2.646 | lr 9.69e-05 | 102,486 tok/s | 54.64 GB | 16.667B tok iter 127180 | train 2.777 | val 2.652 | lr 9.68e-05 | 102,487 tok/s | 54.64 GB | 16.670B tok iter 127200 | train 2.608 | val 2.653 | lr 9.68e-05 | 102,488 tok/s | 54.64 GB | 16.672B tok iter 127220 | train 2.532 | val 2.665 | lr 9.67e-05 | 102,489 tok/s | 54.64 GB | 16.675B tok iter 127240 | train 2.672 | val 2.659 | lr 9.67e-05 | 102,490 tok/s | 54.64 GB | 16.678B tok iter 127260 | train 2.612 | val 2.650 | lr 9.66e-05 | 102,491 tok/s | 54.64 GB | 16.680B tok iter 127280 | train 2.639 | val 2.625 | lr 9.65e-05 | 102,492 tok/s | 54.64 GB | 16.683B tok iter 127300 | train 2.641 | val 2.641 | lr 9.65e-05 | 102,492 tok/s | 54.64 GB | 16.686B tok iter 127320 | train 2.531 | val 2.650 | lr 9.64e-05 | 102,494 tok/s | 54.64 GB | 16.688B tok iter 127340 | train 2.724 | val 2.692 | lr 9.64e-05 | 102,495 tok/s | 54.64 GB | 16.691B tok iter 127360 | train 2.724 | val 2.648 | lr 9.63e-05 | 102,496 tok/s | 54.64 GB | 16.693B tok iter 127380 | train 2.804 | val 2.697 | lr 9.63e-05 | 102,496 tok/s | 54.64 GB | 16.696B tok iter 127400 | train 2.714 | val 2.671 | lr 9.62e-05 | 102,498 tok/s | 54.64 GB | 16.699B tok iter 127420 | train 2.591 | val 2.706 | lr 9.61e-05 | 102,499 tok/s | 54.64 GB | 16.701B tok iter 127440 | train 2.701 | val 2.657 | lr 9.61e-05 | 102,500 tok/s | 54.64 GB | 16.704B tok iter 127460 | train 2.618 | val 2.634 | lr 9.60e-05 | 102,500 tok/s | 54.64 GB | 16.707B tok iter 127480 | train 2.638 | val 2.654 | lr 9.60e-05 | 102,502 tok/s | 54.64 GB | 16.709B tok checkpoint @ iter 127491 -> checkpoints/500m_latest.pt iter 127500 | train 2.707 | val 2.657 | lr 9.59e-05 | 102,495 tok/s | 54.64 GB | 16.712B tok checkpoint @ iter 127500 -> checkpoints/500m_latest.pt iter 127520 | train 2.717 | val 2.669 | lr 9.59e-05 | 102,480 tok/s | 54.64 GB | 16.714B tok iter 127540 | train 2.714 | val 2.648 | lr 9.58e-05 | 102,480 tok/s | 54.64 GB | 16.717B tok iter 127560 | train 2.638 | val 2.606 | lr 9.58e-05 | 102,481 tok/s | 54.64 GB | 16.720B tok iter 127580 | train 2.611 | val 2.642 | lr 9.57e-05 | 102,483 tok/s | 54.64 GB | 16.722B tok iter 127600 | train 2.621 | val 2.665 | lr 9.56e-05 | 102,484 tok/s | 54.64 GB | 16.725B tok iter 127620 | train 2.566 | val 2.689 | lr 9.56e-05 | 102,484 tok/s | 54.64 GB | 16.728B tok iter 127640 | train 2.470 | val 2.640 | lr 9.55e-05 | 102,485 tok/s | 54.64 GB | 16.730B tok iter 127660 | train 2.633 | val 2.602 | lr 9.55e-05 | 102,487 tok/s | 54.64 GB | 16.733B tok iter 127680 | train 2.549 | val 2.645 | lr 9.54e-05 | 102,488 tok/s | 54.64 GB | 16.735B tok iter 127700 | train 2.756 | val 2.633 | lr 9.54e-05 | 102,488 tok/s | 54.64 GB | 16.738B tok iter 127720 | train 2.669 | val 2.607 | lr 9.53e-05 | 102,489 tok/s | 54.64 GB | 16.741B tok iter 127740 | train 2.858 | val 2.664 | lr 9.53e-05 | 102,491 tok/s | 54.64 GB | 16.743B tok iter 127760 | train 2.566 | val 2.668 | lr 9.52e-05 | 102,492 tok/s | 54.64 GB | 16.746B tok iter 127780 | train 2.567 | val 2.639 | lr 9.51e-05 | 102,492 tok/s | 54.64 GB | 16.749B tok iter 127800 | train 2.670 | val 2.662 | lr 9.51e-05 | 102,493 tok/s | 54.64 GB | 16.751B tok iter 127820 | train 2.766 | val 2.581 | lr 9.50e-05 | 102,495 tok/s | 54.64 GB | 16.754B tok iter 127840 | train 2.627 | val 2.682 | lr 9.50e-05 | 102,496 tok/s | 54.64 GB | 16.756B tok iter 127860 | train 2.642 | val 2.612 | lr 9.49e-05 | 102,496 tok/s | 54.64 GB | 16.759B tok iter 127880 | train 2.519 | val 2.610 | lr 9.49e-05 | 102,497 tok/s | 54.64 GB | 16.762B tok iter 127900 | train 2.518 | val 2.647 | lr 9.48e-05 | 102,499 tok/s | 54.64 GB | 16.764B tok iter 127920 | train 2.736 | val 2.628 | lr 9.48e-05 | 102,500 tok/s | 54.64 GB | 16.767B tok iter 127940 | train 2.584 | val 2.664 | lr 9.47e-05 | 102,500 tok/s | 54.64 GB | 16.769B tok iter 127960 | train 2.603 | val 2.652 | lr 9.46e-05 | 102,501 tok/s | 54.64 GB | 16.772B tok iter 127980 | train 2.712 | val 2.625 | lr 9.46e-05 | 102,503 tok/s | 54.64 GB | 16.775B tok checkpoint @ iter 127991 -> checkpoints/500m_latest.pt iter 128000 | train 2.554 | val 2.670 | lr 9.45e-05 | 102,496 tok/s | 54.64 GB | 16.777B tok checkpoint @ iter 128000 -> checkpoints/500m_latest.pt iter 128020 | train 2.719 | val 2.684 | lr 9.45e-05 | 102,480 tok/s | 54.64 GB | 16.780B tok iter 128040 | train 2.670 | val 2.644 | lr 9.44e-05 | 102,481 tok/s | 54.64 GB | 16.783B tok iter 128060 | train 2.585 | val 2.640 | lr 9.44e-05 | 102,483 tok/s | 54.64 GB | 16.785B tok iter 128080 | train 2.686 | val 2.641 | lr 9.43e-05 | 102,484 tok/s | 54.64 GB | 16.788B tok iter 128100 | train 2.547 | val 2.672 | lr 9.43e-05 | 102,484 tok/s | 54.64 GB | 16.790B tok iter 128120 | train 2.674 | val 2.638 | lr 9.42e-05 | 102,485 tok/s | 54.64 GB | 16.793B tok iter 128140 | train 2.677 | val 2.660 | lr 9.42e-05 | 102,487 tok/s | 54.64 GB | 16.796B tok iter 128160 | train 2.617 | val 2.680 | lr 9.41e-05 | 102,488 tok/s | 54.64 GB | 16.798B tok iter 128180 | train 2.779 | val 2.592 | lr 9.40e-05 | 102,488 tok/s | 54.64 GB | 16.801B tok iter 128200 | train 2.702 | val 2.649 | lr 9.40e-05 | 102,489 tok/s | 54.64 GB | 16.804B tok iter 128220 | train 2.603 | val 2.648 | lr 9.39e-05 | 102,491 tok/s | 54.64 GB | 16.806B tok iter 128240 | train 2.592 | val 2.683 | lr 9.39e-05 | 102,492 tok/s | 54.64 GB | 16.809B tok iter 128260 | train 2.551 | val 2.690 | lr 9.38e-05 | 102,492 tok/s | 54.64 GB | 16.811B tok iter 128280 | train 2.695 | val 2.646 | lr 9.38e-05 | 102,493 tok/s | 54.64 GB | 16.814B tok iter 128300 | train 2.745 | val 2.641 | lr 9.37e-05 | 102,495 tok/s | 54.64 GB | 16.817B tok iter 128320 | train 2.602 | val 2.629 | lr 9.37e-05 | 102,496 tok/s | 54.64 GB | 16.819B tok iter 128340 | train 2.585 | val 2.621 | lr 9.36e-05 | 102,496 tok/s | 54.64 GB | 16.822B tok iter 128360 | train 2.451 | val 2.646 | lr 9.36e-05 | 102,497 tok/s | 54.64 GB | 16.825B tok iter 128380 | train 2.618 | val 2.629 | lr 9.35e-05 | 102,498 tok/s | 54.64 GB | 16.827B tok iter 128400 | train 2.749 | val 2.694 | lr 9.34e-05 | 102,500 tok/s | 54.64 GB | 16.830B tok iter 128420 | train 2.747 | val 2.665 | lr 9.34e-05 | 102,500 tok/s | 54.64 GB | 16.832B tok iter 128440 | train 2.631 | val 2.646 | lr 9.33e-05 | 102,501 tok/s | 54.64 GB | 16.835B tok iter 128460 | train 2.655 | val 2.683 | lr 9.33e-05 | 102,502 tok/s | 54.64 GB | 16.838B tok iter 128480 | train 2.579 | val 2.665 | lr 9.32e-05 | 102,504 tok/s | 54.64 GB | 16.840B tok checkpoint @ iter 128491 -> checkpoints/500m_latest.pt iter 128500 | train 2.610 | val 2.700 | lr 9.32e-05 | 102,496 tok/s | 54.64 GB | 16.843B tok checkpoint @ iter 128500 -> checkpoints/500m_latest.pt iter 128520 | train 2.605 | val 2.660 | lr 9.31e-05 | 102,481 tok/s | 54.64 GB | 16.846B tok iter 128540 | train 2.630 | val 2.639 | lr 9.31e-05 | 102,482 tok/s | 54.64 GB | 16.848B tok iter 128560 | train 2.691 | val 2.605 | lr 9.30e-05 | 102,483 tok/s | 54.64 GB | 16.851B tok iter 128580 | train 2.509 | val 2.602 | lr 9.30e-05 | 102,484 tok/s | 54.64 GB | 16.853B tok iter 128600 | train 2.740 | val 2.603 | lr 9.29e-05 | 102,485 tok/s | 54.64 GB | 16.856B tok iter 128620 | train 2.489 | val 2.676 | lr 9.29e-05 | 102,486 tok/s | 54.64 GB | 16.859B tok iter 128640 | train 2.631 | val 2.667 | lr 9.28e-05 | 102,487 tok/s | 54.64 GB | 16.861B tok iter 128660 | train 2.668 | val 2.671 | lr 9.27e-05 | 102,488 tok/s | 54.64 GB | 16.864B tok iter 128680 | train 2.604 | val 2.657 | lr 9.27e-05 | 102,489 tok/s | 54.64 GB | 16.866B tok iter 128700 | train 2.674 | val 2.660 | lr 9.26e-05 | 102,490 tok/s | 54.64 GB | 16.869B tok iter 128720 | train 2.634 | val 2.654 | lr 9.26e-05 | 102,491 tok/s | 54.64 GB | 16.872B tok iter 128740 | train 2.580 | val 2.630 | lr 9.25e-05 | 102,492 tok/s | 54.64 GB | 16.874B tok iter 128760 | train 2.656 | val 2.669 | lr 9.25e-05 | 102,493 tok/s | 54.64 GB | 16.877B tok iter 128780 | train 2.658 | val 2.672 | lr 9.24e-05 | 102,494 tok/s | 54.64 GB | 16.880B tok iter 128800 | train 2.567 | val 2.631 | lr 9.24e-05 | 102,495 tok/s | 54.64 GB | 16.882B tok iter 128820 | train 2.714 | val 2.649 | lr 9.23e-05 | 102,495 tok/s | 54.64 GB | 16.885B tok iter 128840 | train 2.744 | val 2.646 | lr 9.23e-05 | 102,497 tok/s | 54.64 GB | 16.887B tok iter 128860 | train 2.788 | val 2.646 | lr 9.22e-05 | 102,498 tok/s | 54.64 GB | 16.890B tok iter 128880 | train 2.733 | val 2.630 | lr 9.22e-05 | 102,499 tok/s | 54.64 GB | 16.893B tok iter 128900 | train 2.549 | val 2.670 | lr 9.21e-05 | 102,499 tok/s | 54.64 GB | 16.895B tok iter 128920 | train 2.592 | val 2.635 | lr 9.21e-05 | 102,501 tok/s | 54.64 GB | 16.898B tok iter 128940 | train 2.568 | val 2.709 | lr 9.20e-05 | 102,502 tok/s | 54.64 GB | 16.901B tok iter 128960 | train 2.599 | val 2.688 | lr 9.19e-05 | 102,503 tok/s | 54.64 GB | 16.903B tok iter 128980 | train 2.736 | val 2.654 | lr 9.19e-05 | 102,503 tok/s | 54.64 GB | 16.906B tok checkpoint @ iter 128990 -> checkpoints/500m_latest.pt iter 129000 | train 2.639 | val 2.625 | lr 9.18e-05 | 102,497 tok/s | 54.64 GB | 16.908B tok checkpoint @ iter 129000 -> checkpoints/500m_latest.pt iter 129020 | train 2.767 | val 2.637 | lr 9.18e-05 | 102,482 tok/s | 54.64 GB | 16.911B tok iter 129040 | train 2.716 | val 2.663 | lr 9.17e-05 | 102,483 tok/s | 54.64 GB | 16.914B tok iter 129060 | train 2.666 | val 2.621 | lr 9.17e-05 | 102,484 tok/s | 54.64 GB | 16.916B tok iter 129080 | train 2.605 | val 2.645 | lr 9.16e-05 | 102,485 tok/s | 54.64 GB | 16.919B tok iter 129100 | train 2.787 | val 2.640 | lr 9.16e-05 | 102,486 tok/s | 54.64 GB | 16.922B tok iter 129120 | train 2.647 | val 2.660 | lr 9.15e-05 | 102,487 tok/s | 54.64 GB | 16.924B tok iter 129140 | train 2.671 | val 2.643 | lr 9.15e-05 | 102,488 tok/s | 54.64 GB | 16.927B tok iter 129160 | train 2.420 | val 2.621 | lr 9.14e-05 | 102,489 tok/s | 54.64 GB | 16.929B tok iter 129180 | train 2.860 | val 2.641 | lr 9.14e-05 | 102,490 tok/s | 54.64 GB | 16.932B tok iter 129200 | train 2.672 | val 2.619 | lr 9.13e-05 | 102,491 tok/s | 54.64 GB | 16.935B tok iter 129220 | train 2.636 | val 2.629 | lr 9.13e-05 | 102,491 tok/s | 54.64 GB | 16.937B tok iter 129240 | train 2.658 | val 2.634 | lr 9.12e-05 | 102,493 tok/s | 54.64 GB | 16.940B tok iter 129260 | train 2.649 | val 2.608 | lr 9.12e-05 | 102,494 tok/s | 54.64 GB | 16.942B tok iter 129280 | train 2.540 | val 2.594 | lr 9.11e-05 | 102,495 tok/s | 54.64 GB | 16.945B tok iter 129300 | train 2.559 | val 2.640 | lr 9.10e-05 | 102,495 tok/s | 54.64 GB | 16.948B tok iter 129320 | train 2.608 | val 2.655 | lr 9.10e-05 | 102,497 tok/s | 54.64 GB | 16.950B tok iter 129340 | train 2.663 | val 2.628 | lr 9.09e-05 | 102,498 tok/s | 54.64 GB | 16.953B tok iter 129360 | train 2.597 | val 2.698 | lr 9.09e-05 | 102,499 tok/s | 54.64 GB | 16.956B tok iter 129380 | train 2.704 | val 2.671 | lr 9.08e-05 | 102,499 tok/s | 54.64 GB | 16.958B tok iter 129400 | train 2.616 | val 2.694 | lr 9.08e-05 | 102,500 tok/s | 54.64 GB | 16.961B tok iter 129420 | train 2.630 | val 2.656 | lr 9.07e-05 | 102,502 tok/s | 54.64 GB | 16.963B tok iter 129440 | train 2.605 | val 2.613 | lr 9.07e-05 | 102,503 tok/s | 54.64 GB | 16.966B tok iter 129460 | train 2.534 | val 2.688 | lr 9.06e-05 | 102,503 tok/s | 54.64 GB | 16.969B tok iter 129480 | train 2.547 | val 2.675 | lr 9.06e-05 | 102,504 tok/s | 54.64 GB | 16.971B tok checkpoint @ iter 129491 -> checkpoints/500m_latest.pt iter 129500 | train 2.558 | val 2.658 | lr 9.05e-05 | 102,497 tok/s | 54.64 GB | 16.974B tok checkpoint @ iter 129500 -> checkpoints/500m_latest.pt iter 129520 | train 2.748 | val 2.645 | lr 9.05e-05 | 102,483 tok/s | 54.64 GB | 16.977B tok iter 129540 | train 2.759 | val 2.676 | lr 9.04e-05 | 102,483 tok/s | 54.64 GB | 16.979B tok iter 129560 | train 2.646 | val 2.621 | lr 9.04e-05 | 102,484 tok/s | 54.64 GB | 16.982B tok iter 129580 | train 2.823 | val 2.661 | lr 9.03e-05 | 102,485 tok/s | 54.64 GB | 16.984B tok iter 129600 | train 2.579 | val 2.626 | lr 9.03e-05 | 102,487 tok/s | 54.64 GB | 16.987B tok iter 129620 | train 2.608 | val 2.616 | lr 9.02e-05 | 102,487 tok/s | 54.64 GB | 16.990B tok iter 129640 | train 2.738 | val 2.651 | lr 9.02e-05 | 102,488 tok/s | 54.64 GB | 16.992B tok iter 129660 | train 2.625 | val 2.637 | lr 9.01e-05 | 102,489 tok/s | 54.64 GB | 16.995B tok iter 129680 | train 2.703 | val 2.649 | lr 9.01e-05 | 102,490 tok/s | 54.64 GB | 16.998B tok iter 129700 | train 2.639 | val 2.669 | lr 9.00e-05 | 102,491 tok/s | 54.64 GB | 17.000B tok iter 129720 | train 2.649 | val 2.636 | lr 9.00e-05 | 102,492 tok/s | 54.64 GB | 17.003B tok iter 129740 | train 2.715 | val 2.610 | lr 8.99e-05 | 102,493 tok/s | 54.64 GB | 17.005B tok iter 129760 | train 2.627 | val 2.636 | lr 8.99e-05 | 102,494 tok/s | 54.64 GB | 17.008B tok iter 129780 | train 2.728 | val 2.627 | lr 8.98e-05 | 102,495 tok/s | 54.64 GB | 17.011B tok iter 129800 | train 2.586 | val 2.660 | lr 8.98e-05 | 102,496 tok/s | 54.64 GB | 17.013B tok iter 129820 | train 2.698 | val 2.682 | lr 8.97e-05 | 102,497 tok/s | 54.64 GB | 17.016B tok iter 129840 | train 2.587 | val 2.671 | lr 8.97e-05 | 102,498 tok/s | 54.64 GB | 17.019B tok iter 129860 | train 2.639 | val 2.703 | lr 8.96e-05 | 102,498 tok/s | 54.64 GB | 17.021B tok iter 129880 | train 2.795 | val 2.613 | lr 8.96e-05 | 102,500 tok/s | 54.64 GB | 17.024B tok iter 129900 | train 2.750 | val 2.627 | lr 8.95e-05 | 102,501 tok/s | 54.64 GB | 17.026B tok iter 129920 | train 2.707 | val 2.626 | lr 8.94e-05 | 102,502 tok/s | 54.64 GB | 17.029B tok iter 129940 | train 2.638 | val 2.675 | lr 8.94e-05 | 102,502 tok/s | 54.64 GB | 17.032B tok iter 129960 | train 2.653 | val 2.612 | lr 8.93e-05 | 102,504 tok/s | 54.64 GB | 17.034B tok iter 129980 | train 2.606 | val 2.638 | lr 8.93e-05 | 102,505 tok/s | 54.64 GB | 17.037B tok checkpoint @ iter 129991 -> checkpoints/500m_latest.pt iter 130000 | train 2.777 | val 2.639 | lr 8.92e-05 | 102,498 tok/s | 54.64 GB | 17.039B tok checkpoint @ iter 130000 -> checkpoints/500m_latest.pt iter 130020 | train 2.605 | val 2.599 | lr 8.92e-05 | 102,482 tok/s | 54.64 GB | 17.042B tok iter 130040 | train 2.651 | val 2.645 | lr 8.91e-05 | 102,484 tok/s | 54.64 GB | 17.045B tok iter 130060 | train 2.571 | val 2.674 | lr 8.91e-05 | 102,485 tok/s | 54.64 GB | 17.047B tok iter 130080 | train 2.572 | val 2.650 | lr 8.90e-05 | 102,486 tok/s | 54.64 GB | 17.050B tok iter 130100 | train 2.699 | val 2.604 | lr 8.90e-05 | 102,486 tok/s | 54.64 GB | 17.053B tok iter 130120 | train 2.519 | val 2.630 | lr 8.89e-05 | 102,487 tok/s | 54.64 GB | 17.055B tok iter 130140 | train 2.630 | val 2.657 | lr 8.89e-05 | 102,489 tok/s | 54.64 GB | 17.058B tok iter 130160 | train 2.508 | val 2.643 | lr 8.88e-05 | 102,490 tok/s | 54.64 GB | 17.060B tok iter 130180 | train 2.567 | val 2.645 | lr 8.88e-05 | 102,490 tok/s | 54.64 GB | 17.063B tok iter 130200 | train 2.526 | val 2.652 | lr 8.87e-05 | 102,491 tok/s | 54.64 GB | 17.066B tok iter 130220 | train 2.532 | val 2.682 | lr 8.87e-05 | 102,493 tok/s | 54.64 GB | 17.068B tok iter 130240 | train 2.724 | val 2.646 | lr 8.86e-05 | 102,494 tok/s | 54.64 GB | 17.071B tok iter 130260 | train 2.577 | val 2.685 | lr 8.86e-05 | 102,494 tok/s | 54.64 GB | 17.074B tok iter 130280 | train 2.531 | val 2.621 | lr 8.85e-05 | 102,495 tok/s | 54.64 GB | 17.076B tok iter 130300 | train 2.577 | val 2.643 | lr 8.85e-05 | 102,496 tok/s | 54.64 GB | 17.079B tok iter 130320 | train 2.707 | val 2.655 | lr 8.84e-05 | 102,498 tok/s | 54.64 GB | 17.081B tok iter 130340 | train 2.627 | val 2.644 | lr 8.84e-05 | 102,498 tok/s | 54.64 GB | 17.084B tok iter 130360 | train 2.727 | val 2.615 | lr 8.83e-05 | 102,499 tok/s | 54.64 GB | 17.087B tok iter 130380 | train 2.665 | val 2.669 | lr 8.83e-05 | 102,500 tok/s | 54.64 GB | 17.089B tok iter 130400 | train 2.692 | val 2.675 | lr 8.82e-05 | 102,502 tok/s | 54.64 GB | 17.092B tok iter 130420 | train 2.628 | val 2.647 | lr 8.82e-05 | 102,502 tok/s | 54.64 GB | 17.095B tok iter 130440 | train 2.720 | val 2.639 | lr 8.81e-05 | 102,503 tok/s | 54.64 GB | 17.097B tok iter 130460 | train 2.763 | val 2.651 | lr 8.81e-05 | 102,504 tok/s | 54.64 GB | 17.100B tok iter 130480 | train 2.468 | val 2.646 | lr 8.80e-05 | 102,505 tok/s | 54.64 GB | 17.102B tok checkpoint @ iter 130491 -> checkpoints/500m_latest.pt iter 130500 | train 2.478 | val 2.695 | lr 8.80e-05 | 102,498 tok/s | 54.64 GB | 17.105B tok checkpoint @ iter 130500 -> checkpoints/500m_latest.pt iter 130520 | train 2.605 | val 2.640 | lr 8.79e-05 | 102,483 tok/s | 54.64 GB | 17.108B tok iter 130540 | train 2.570 | val 2.638 | lr 8.79e-05 | 102,484 tok/s | 54.64 GB | 17.110B tok iter 130560 | train 2.770 | val 2.628 | lr 8.78e-05 | 102,485 tok/s | 54.64 GB | 17.113B tok iter 130580 | train 2.473 | val 2.645 | lr 8.78e-05 | 102,486 tok/s | 54.64 GB | 17.116B tok iter 130600 | train 2.691 | val 2.636 | lr 8.77e-05 | 102,487 tok/s | 54.64 GB | 17.118B tok iter 130620 | train 2.661 | val 2.645 | lr 8.77e-05 | 102,488 tok/s | 54.64 GB | 17.121B tok iter 130640 | train 2.699 | val 2.599 | lr 8.76e-05 | 102,489 tok/s | 54.64 GB | 17.123B tok iter 130660 | train 2.550 | val 2.625 | lr 8.76e-05 | 102,489 tok/s | 54.64 GB | 17.126B tok iter 130680 | train 2.688 | val 2.674 | lr 8.75e-05 | 102,491 tok/s | 54.64 GB | 17.129B tok iter 130700 | train 2.653 | val 2.656 | lr 8.75e-05 | 102,492 tok/s | 54.64 GB | 17.131B tok iter 130720 | train 2.615 | val 2.667 | lr 8.74e-05 | 102,493 tok/s | 54.64 GB | 17.134B tok iter 130740 | train 2.790 | val 2.605 | lr 8.74e-05 | 102,493 tok/s | 54.64 GB | 17.136B tok iter 130760 | train 2.722 | val 2.684 | lr 8.73e-05 | 102,495 tok/s | 54.64 GB | 17.139B tok iter 130780 | train 2.756 | val 2.673 | lr 8.73e-05 | 102,496 tok/s | 54.64 GB | 17.142B tok iter 130800 | train 2.566 | val 2.619 | lr 8.72e-05 | 102,497 tok/s | 54.64 GB | 17.144B tok iter 130820 | train 2.522 | val 2.652 | lr 8.72e-05 | 102,497 tok/s | 54.64 GB | 17.147B tok iter 130840 | train 2.645 | val 2.660 | lr 8.71e-05 | 102,498 tok/s | 54.64 GB | 17.150B tok iter 130860 | train 2.847 | val 2.644 | lr 8.71e-05 | 102,500 tok/s | 54.64 GB | 17.152B tok iter 130880 | train 2.570 | val 2.721 | lr 8.70e-05 | 102,501 tok/s | 54.64 GB | 17.155B tok iter 130900 | train 2.682 | val 2.651 | lr 8.70e-05 | 102,501 tok/s | 54.64 GB | 17.157B tok iter 130920 | train 2.611 | val 2.657 | lr 8.70e-05 | 102,502 tok/s | 54.64 GB | 17.160B tok iter 130940 | train 2.710 | val 2.641 | lr 8.69e-05 | 102,503 tok/s | 54.64 GB | 17.163B tok iter 130960 | train 2.556 | val 2.643 | lr 8.69e-05 | 102,505 tok/s | 54.64 GB | 17.165B tok iter 130980 | train 2.767 | val 2.676 | lr 8.68e-05 | 102,505 tok/s | 54.64 GB | 17.168B tok checkpoint @ iter 130990 -> checkpoints/500m_latest.pt iter 131000 | train 2.667 | val 2.672 | lr 8.68e-05 | 102,498 tok/s | 54.64 GB | 17.171B tok checkpoint @ iter 131000 -> checkpoints/500m_latest.pt iter 131020 | train 2.581 | val 2.672 | lr 8.67e-05 | 102,484 tok/s | 54.64 GB | 17.173B tok iter 131040 | train 2.648 | val 2.683 | lr 8.67e-05 | 102,485 tok/s | 54.64 GB | 17.176B tok iter 131060 | train 2.532 | val 2.649 | lr 8.66e-05 | 102,485 tok/s | 54.64 GB | 17.178B tok iter 131080 | train 2.709 | val 2.642 | lr 8.66e-05 | 102,487 tok/s | 54.64 GB | 17.181B tok iter 131100 | train 2.598 | val 2.591 | lr 8.65e-05 | 102,488 tok/s | 54.64 GB | 17.184B tok iter 131120 | train 2.657 | val 2.641 | lr 8.65e-05 | 102,489 tok/s | 54.64 GB | 17.186B tok iter 131140 | train 2.714 | val 2.632 | lr 8.64e-05 | 102,489 tok/s | 54.64 GB | 17.189B tok iter 131160 | train 2.647 | val 2.642 | lr 8.64e-05 | 102,490 tok/s | 54.64 GB | 17.192B tok iter 131180 | train 2.626 | val 2.634 | lr 8.63e-05 | 102,492 tok/s | 54.64 GB | 17.194B tok iter 131200 | train 2.599 | val 2.634 | lr 8.63e-05 | 102,493 tok/s | 54.64 GB | 17.197B tok iter 131220 | train 2.605 | val 2.636 | lr 8.62e-05 | 102,493 tok/s | 54.64 GB | 17.199B tok iter 131240 | train 2.673 | val 2.654 | lr 8.62e-05 | 102,494 tok/s | 54.64 GB | 17.202B tok iter 131260 | train 2.797 | val 2.663 | lr 8.61e-05 | 102,495 tok/s | 54.64 GB | 17.205B tok iter 131280 | train 2.623 | val 2.648 | lr 8.61e-05 | 102,497 tok/s | 54.64 GB | 17.207B tok iter 131300 | train 2.661 | val 2.628 | lr 8.60e-05 | 102,497 tok/s | 54.64 GB | 17.210B tok iter 131320 | train 2.637 | val 2.625 | lr 8.60e-05 | 102,498 tok/s | 54.64 GB | 17.213B tok iter 131340 | train 2.623 | val 2.661 | lr 8.59e-05 | 102,499 tok/s | 54.64 GB | 17.215B tok iter 131360 | train 2.683 | val 2.687 | lr 8.59e-05 | 102,501 tok/s | 54.64 GB | 17.218B tok iter 131380 | train 2.585 | val 2.676 | lr 8.58e-05 | 102,501 tok/s | 54.64 GB | 17.220B tok iter 131400 | train 2.720 | val 2.644 | lr 8.58e-05 | 102,502 tok/s | 54.64 GB | 17.223B tok iter 131420 | train 2.622 | val 2.647 | lr 8.57e-05 | 102,503 tok/s | 54.64 GB | 17.226B tok iter 131440 | train 2.846 | val 2.607 | lr 8.57e-05 | 102,504 tok/s | 54.64 GB | 17.228B tok iter 131460 | train 2.643 | val 2.652 | lr 8.56e-05 | 102,505 tok/s | 54.64 GB | 17.231B tok iter 131480 | train 2.659 | val 2.630 | lr 8.56e-05 | 102,506 tok/s | 54.64 GB | 17.233B tok checkpoint @ iter 131491 -> checkpoints/500m_latest.pt iter 131500 | train 2.630 | val 2.584 | lr 8.55e-05 | 102,500 tok/s | 54.64 GB | 17.236B tok checkpoint @ iter 131500 -> checkpoints/500m_latest.pt iter 131520 | train 2.826 | val 2.670 | lr 8.55e-05 | 102,486 tok/s | 54.64 GB | 17.239B tok iter 131540 | train 2.639 | val 2.663 | lr 8.55e-05 | 102,486 tok/s | 54.64 GB | 17.241B tok iter 131560 | train 2.644 | val 2.638 | lr 8.54e-05 | 102,487 tok/s | 54.64 GB | 17.244B tok iter 131580 | train 2.466 | val 2.612 | lr 8.54e-05 | 102,488 tok/s | 54.64 GB | 17.247B tok iter 131600 | train 2.716 | val 2.629 | lr 8.53e-05 | 102,490 tok/s | 54.64 GB | 17.249B tok iter 131620 | train 2.787 | val 2.629 | lr 8.53e-05 | 102,490 tok/s | 54.64 GB | 17.252B tok iter 131640 | train 2.736 | val 2.619 | lr 8.52e-05 | 102,491 tok/s | 54.64 GB | 17.254B tok iter 131660 | train 2.728 | val 2.686 | lr 8.52e-05 | 102,492 tok/s | 54.64 GB | 17.257B tok iter 131680 | train 2.656 | val 2.637 | lr 8.51e-05 | 102,494 tok/s | 54.64 GB | 17.260B tok iter 131700 | train 2.655 | val 2.629 | lr 8.51e-05 | 102,494 tok/s | 54.64 GB | 17.262B tok iter 131720 | train 2.640 | val 2.692 | lr 8.50e-05 | 102,495 tok/s | 54.64 GB | 17.265B tok iter 131740 | train 2.486 | val 2.631 | lr 8.50e-05 | 102,496 tok/s | 54.64 GB | 17.268B tok iter 131760 | train 2.658 | val 2.661 | lr 8.49e-05 | 102,497 tok/s | 54.64 GB | 17.270B tok iter 131780 | train 2.696 | val 2.608 | lr 8.49e-05 | 102,497 tok/s | 54.64 GB | 17.273B tok iter 131800 | train 2.470 | val 2.629 | lr 8.48e-05 | 102,499 tok/s | 54.64 GB | 17.275B tok iter 131820 | train 2.808 | val 2.660 | lr 8.48e-05 | 102,500 tok/s | 54.64 GB | 17.278B tok iter 131840 | train 2.551 | val 2.676 | lr 8.47e-05 | 102,501 tok/s | 54.64 GB | 17.281B tok iter 131860 | train 2.581 | val 2.642 | lr 8.47e-05 | 102,501 tok/s | 54.64 GB | 17.283B tok iter 131880 | train 2.661 | val 2.654 | lr 8.47e-05 | 102,503 tok/s | 54.64 GB | 17.286B tok iter 131900 | train 2.657 | val 2.637 | lr 8.46e-05 | 102,504 tok/s | 54.64 GB | 17.289B tok iter 131920 | train 2.768 | val 2.682 | lr 8.46e-05 | 102,505 tok/s | 54.64 GB | 17.291B tok iter 131940 | train 2.536 | val 2.618 | lr 8.45e-05 | 102,505 tok/s | 54.64 GB | 17.294B tok iter 131960 | train 2.722 | val 2.626 | lr 8.45e-05 | 102,506 tok/s | 54.64 GB | 17.296B tok iter 131980 | train 2.628 | val 2.610 | lr 8.44e-05 | 102,508 tok/s | 54.64 GB | 17.299B tok checkpoint @ iter 131991 -> checkpoints/500m_latest.pt iter 132000 | train 2.647 | val 2.636 | lr 8.44e-05 | 102,501 tok/s | 54.64 GB | 17.302B tok checkpoint @ iter 132000 -> checkpoints/500m_latest.pt iter 132020 | train 2.693 | val 2.625 | lr 8.43e-05 | 102,486 tok/s | 54.64 GB | 17.304B tok iter 132040 | train 2.585 | val 2.602 | lr 8.43e-05 | 102,488 tok/s | 54.64 GB | 17.307B tok iter 132060 | train 2.536 | val 2.650 | lr 8.42e-05 | 102,489 tok/s | 54.64 GB | 17.309B tok iter 132080 | train 2.561 | val 2.654 | lr 8.42e-05 | 102,490 tok/s | 54.64 GB | 17.312B tok iter 132100 | train 2.719 | val 2.639 | lr 8.41e-05 | 102,490 tok/s | 54.64 GB | 17.315B tok iter 132120 | train 2.655 | val 2.657 | lr 8.41e-05 | 102,492 tok/s | 54.64 GB | 17.317B tok iter 132140 | train 2.798 | val 2.663 | lr 8.40e-05 | 102,493 tok/s | 54.64 GB | 17.320B tok iter 132160 | train 2.537 | val 2.629 | lr 8.40e-05 | 102,494 tok/s | 54.64 GB | 17.323B tok iter 132180 | train 2.627 | val 2.622 | lr 8.40e-05 | 102,494 tok/s | 54.64 GB | 17.325B tok iter 132200 | train 2.645 | val 2.679 | lr 8.39e-05 | 102,495 tok/s | 54.64 GB | 17.328B tok iter 132220 | train 2.581 | val 2.709 | lr 8.39e-05 | 102,497 tok/s | 54.64 GB | 17.330B tok iter 132240 | train 2.564 | val 2.579 | lr 8.38e-05 | 102,498 tok/s | 54.64 GB | 17.333B tok iter 132260 | train 2.681 | val 2.669 | lr 8.38e-05 | 102,498 tok/s | 54.64 GB | 17.336B tok iter 132280 | train 2.589 | val 2.666 | lr 8.37e-05 | 102,499 tok/s | 54.64 GB | 17.338B tok iter 132300 | train 2.663 | val 2.665 | lr 8.37e-05 | 102,500 tok/s | 54.64 GB | 17.341B tok iter 132320 | train 2.639 | val 2.632 | lr 8.36e-05 | 102,502 tok/s | 54.64 GB | 17.344B tok iter 132340 | train 2.671 | val 2.646 | lr 8.36e-05 | 102,502 tok/s | 54.64 GB | 17.346B tok iter 132360 | train 2.491 | val 2.572 | lr 8.35e-05 | 102,503 tok/s | 54.64 GB | 17.349B tok iter 132380 | train 2.583 | val 2.612 | lr 8.35e-05 | 102,504 tok/s | 54.64 GB | 17.351B tok iter 132400 | train 2.555 | val 2.610 | lr 8.34e-05 | 102,505 tok/s | 54.64 GB | 17.354B tok iter 132420 | train 2.674 | val 2.606 | lr 8.34e-05 | 102,506 tok/s | 54.64 GB | 17.357B tok iter 132440 | train 2.596 | val 2.666 | lr 8.34e-05 | 102,507 tok/s | 54.64 GB | 17.359B tok iter 132460 | train 2.464 | val 2.653 | lr 8.33e-05 | 102,508 tok/s | 54.64 GB | 17.362B tok iter 132480 | train 2.647 | val 2.662 | lr 8.33e-05 | 102,509 tok/s | 54.64 GB | 17.365B tok checkpoint @ iter 132491 -> checkpoints/500m_latest.pt iter 132500 | train 2.662 | val 2.673 | lr 8.32e-05 | 102,502 tok/s | 54.64 GB | 17.367B tok checkpoint @ iter 132500 -> checkpoints/500m_latest.pt iter 132520 | train 2.559 | val 2.645 | lr 8.32e-05 | 102,488 tok/s | 54.64 GB | 17.370B tok iter 132540 | train 2.698 | val 2.618 | lr 8.31e-05 | 102,489 tok/s | 54.64 GB | 17.372B tok iter 132560 | train 2.511 | val 2.616 | lr 8.31e-05 | 102,490 tok/s | 54.64 GB | 17.375B tok iter 132580 | train 2.713 | val 2.621 | lr 8.30e-05 | 102,490 tok/s | 54.64 GB | 17.378B tok iter 132600 | train 2.605 | val 2.623 | lr 8.30e-05 | 102,492 tok/s | 54.64 GB | 17.380B tok iter 132620 | train 2.605 | val 2.661 | lr 8.29e-05 | 102,493 tok/s | 54.64 GB | 17.383B tok iter 132640 | train 2.668 | val 2.614 | lr 8.29e-05 | 102,494 tok/s | 54.64 GB | 17.386B tok iter 132660 | train 2.611 | val 2.647 | lr 8.29e-05 | 102,494 tok/s | 54.64 GB | 17.388B tok iter 132680 | train 2.556 | val 2.628 | lr 8.28e-05 | 102,495 tok/s | 54.64 GB | 17.391B tok iter 132700 | train 2.518 | val 2.658 | lr 8.28e-05 | 102,497 tok/s | 54.64 GB | 17.393B tok iter 132720 | train 2.583 | val 2.608 | lr 8.27e-05 | 102,498 tok/s | 54.64 GB | 17.396B tok iter 132740 | train 2.751 | val 2.621 | lr 8.27e-05 | 102,498 tok/s | 54.64 GB | 17.399B tok iter 132760 | train 2.665 | val 2.618 | lr 8.26e-05 | 102,499 tok/s | 54.64 GB | 17.401B tok iter 132780 | train 2.706 | val 2.625 | lr 8.26e-05 | 102,500 tok/s | 54.64 GB | 17.404B tok iter 132800 | train 2.556 | val 2.618 | lr 8.25e-05 | 102,502 tok/s | 54.64 GB | 17.406B tok iter 132820 | train 2.634 | val 2.601 | lr 8.25e-05 | 102,502 tok/s | 54.64 GB | 17.409B tok iter 132840 | train 2.542 | val 2.622 | lr 8.25e-05 | 102,503 tok/s | 54.64 GB | 17.412B tok iter 132860 | train 2.666 | val 2.642 | lr 8.24e-05 | 102,504 tok/s | 54.64 GB | 17.414B tok iter 132880 | train 2.734 | val 2.646 | lr 8.24e-05 | 102,505 tok/s | 54.64 GB | 17.417B tok iter 132900 | train 2.640 | val 2.614 | lr 8.23e-05 | 102,506 tok/s | 54.64 GB | 17.420B tok iter 132920 | train 2.692 | val 2.667 | lr 8.23e-05 | 102,507 tok/s | 54.64 GB | 17.422B tok iter 132940 | train 2.664 | val 2.636 | lr 8.22e-05 | 102,508 tok/s | 54.64 GB | 17.425B tok iter 132960 | train 2.661 | val 2.677 | lr 8.22e-05 | 102,509 tok/s | 54.64 GB | 17.427B tok iter 132980 | train 2.637 | val 2.656 | lr 8.21e-05 | 102,509 tok/s | 54.64 GB | 17.430B tok checkpoint @ iter 132991 -> checkpoints/500m_latest.pt iter 133000 | train 2.731 | val 2.633 | lr 8.21e-05 | 102,503 tok/s | 54.64 GB | 17.433B tok checkpoint @ iter 133000 -> checkpoints/500m_latest.pt iter 133020 | train 2.496 | val 2.638 | lr 8.20e-05 | 102,488 tok/s | 54.64 GB | 17.435B tok iter 133040 | train 2.718 | val 2.628 | lr 8.20e-05 | 102,490 tok/s | 54.64 GB | 17.438B tok iter 133060 | train 2.527 | val 2.640 | lr 8.20e-05 | 102,490 tok/s | 54.64 GB | 17.441B tok iter 133080 | train 2.678 | val 2.639 | lr 8.19e-05 | 102,491 tok/s | 54.64 GB | 17.443B tok iter 133100 | train 2.541 | val 2.624 | lr 8.19e-05 | 102,492 tok/s | 54.64 GB | 17.446B tok iter 133120 | train 2.727 | val 2.592 | lr 8.18e-05 | 102,493 tok/s | 54.64 GB | 17.448B tok iter 133140 | train 2.683 | val 2.652 | lr 8.18e-05 | 102,494 tok/s | 54.64 GB | 17.451B tok iter 133160 | train 2.580 | val 2.624 | lr 8.17e-05 | 102,495 tok/s | 54.64 GB | 17.454B tok iter 133180 | train 2.688 | val 2.657 | lr 8.17e-05 | 102,496 tok/s | 54.64 GB | 17.456B tok iter 133200 | train 2.744 | val 2.614 | lr 8.17e-05 | 102,497 tok/s | 54.64 GB | 17.459B tok iter 133220 | train 2.607 | val 2.623 | lr 8.16e-05 | 102,497 tok/s | 54.64 GB | 17.462B tok iter 133240 | train 2.667 | val 2.619 | lr 8.16e-05 | 102,499 tok/s | 54.64 GB | 17.464B tok iter 133260 | train 2.686 | val 2.658 | lr 8.15e-05 | 102,500 tok/s | 54.64 GB | 17.467B tok iter 133280 | train 2.563 | val 2.696 | lr 8.15e-05 | 102,501 tok/s | 54.64 GB | 17.469B tok iter 133300 | train 2.653 | val 2.658 | lr 8.14e-05 | 102,501 tok/s | 54.64 GB | 17.472B tok iter 133320 | train 2.592 | val 2.659 | lr 8.14e-05 | 102,502 tok/s | 54.64 GB | 17.475B tok iter 133340 | train 2.605 | val 2.647 | lr 8.13e-05 | 102,503 tok/s | 54.64 GB | 17.477B tok iter 133360 | train 2.556 | val 2.608 | lr 8.13e-05 | 102,505 tok/s | 54.64 GB | 17.480B tok iter 133380 | train 2.625 | val 2.636 | lr 8.13e-05 | 102,505 tok/s | 54.64 GB | 17.483B tok iter 133400 | train 2.684 | val 2.671 | lr 8.12e-05 | 102,506 tok/s | 54.64 GB | 17.485B tok iter 133420 | train 2.731 | val 2.653 | lr 8.12e-05 | 102,507 tok/s | 54.64 GB | 17.488B tok iter 133440 | train 2.677 | val 2.575 | lr 8.11e-05 | 102,508 tok/s | 54.64 GB | 17.490B tok iter 133460 | train 2.708 | val 2.624 | lr 8.11e-05 | 102,509 tok/s | 54.64 GB | 17.493B tok iter 133480 | train 2.606 | val 2.639 | lr 8.10e-05 | 102,510 tok/s | 54.64 GB | 17.496B tok checkpoint @ iter 133490 -> checkpoints/500m_latest.pt iter 133500 | train 2.509 | val 2.626 | lr 8.10e-05 | 102,503 tok/s | 54.64 GB | 17.498B tok checkpoint @ iter 133500 -> checkpoints/500m_latest.pt iter 133520 | train 2.638 | val 2.662 | lr 8.10e-05 | 102,489 tok/s | 54.64 GB | 17.501B tok iter 133540 | train 2.567 | val 2.668 | lr 8.09e-05 | 102,489 tok/s | 54.64 GB | 17.503B tok iter 133560 | train 2.673 | val 2.645 | lr 8.09e-05 | 102,491 tok/s | 54.64 GB | 17.506B tok iter 133580 | train 2.698 | val 2.649 | lr 8.08e-05 | 102,492 tok/s | 54.64 GB | 17.509B tok iter 133600 | train 2.663 | val 2.655 | lr 8.08e-05 | 102,493 tok/s | 54.64 GB | 17.511B tok iter 133620 | train 2.653 | val 2.667 | lr 8.07e-05 | 102,493 tok/s | 54.64 GB | 17.514B tok iter 133640 | train 2.605 | val 2.635 | lr 8.07e-05 | 102,495 tok/s | 54.64 GB | 17.517B tok iter 133660 | train 2.607 | val 2.652 | lr 8.06e-05 | 102,496 tok/s | 54.64 GB | 17.519B tok iter 133680 | train 2.594 | val 2.648 | lr 8.06e-05 | 102,497 tok/s | 54.64 GB | 17.522B tok iter 133700 | train 2.584 | val 2.615 | lr 8.06e-05 | 102,497 tok/s | 54.64 GB | 17.524B tok iter 133720 | train 2.814 | val 2.643 | lr 8.05e-05 | 102,498 tok/s | 54.64 GB | 17.527B tok iter 133740 | train 2.723 | val 2.615 | lr 8.05e-05 | 102,499 tok/s | 54.64 GB | 17.530B tok iter 133760 | train 2.667 | val 2.668 | lr 8.04e-05 | 102,501 tok/s | 54.64 GB | 17.532B tok iter 133780 | train 2.561 | val 2.660 | lr 8.04e-05 | 102,501 tok/s | 54.64 GB | 17.535B tok iter 133800 | train 2.612 | val 2.661 | lr 8.03e-05 | 102,502 tok/s | 54.64 GB | 17.538B tok iter 133820 | train 2.675 | val 2.655 | lr 8.03e-05 | 102,503 tok/s | 54.64 GB | 17.540B tok iter 133840 | train 2.551 | val 2.641 | lr 8.03e-05 | 102,504 tok/s | 54.64 GB | 17.543B tok iter 133860 | train 2.666 | val 2.680 | lr 8.02e-05 | 102,505 tok/s | 54.64 GB | 17.545B tok iter 133880 | train 2.739 | val 2.612 | lr 8.02e-05 | 102,506 tok/s | 54.64 GB | 17.548B tok iter 133900 | train 2.738 | val 2.632 | lr 8.01e-05 | 102,507 tok/s | 54.64 GB | 17.551B tok iter 133920 | train 2.607 | val 2.608 | lr 8.01e-05 | 102,508 tok/s | 54.64 GB | 17.553B tok iter 133940 | train 2.572 | val 2.640 | lr 8.00e-05 | 102,508 tok/s | 54.64 GB | 17.556B tok iter 133960 | train 2.567 | val 2.647 | lr 8.00e-05 | 102,510 tok/s | 54.64 GB | 17.559B tok iter 133980 | train 2.603 | val 2.584 | lr 8.00e-05 | 102,511 tok/s | 54.64 GB | 17.561B tok checkpoint @ iter 133991 -> checkpoints/500m_latest.pt iter 134000 | train 2.698 | val 2.658 | lr 7.99e-05 | 102,504 tok/s | 54.64 GB | 17.564B tok checkpoint @ iter 134000 -> checkpoints/500m_latest.pt iter 134020 | train 2.737 | val 2.645 | lr 7.99e-05 | 102,489 tok/s | 54.64 GB | 17.566B tok iter 134040 | train 2.679 | val 2.679 | lr 7.98e-05 | 102,491 tok/s | 54.64 GB | 17.569B tok iter 134060 | train 2.574 | val 2.647 | lr 7.98e-05 | 102,492 tok/s | 54.64 GB | 17.572B tok iter 134080 | train 2.719 | val 2.604 | lr 7.98e-05 | 102,493 tok/s | 54.64 GB | 17.574B tok iter 134100 | train 2.617 | val 2.665 | lr 7.97e-05 | 102,493 tok/s | 54.64 GB | 17.577B tok iter 134120 | train 2.651 | val 2.711 | lr 7.97e-05 | 102,495 tok/s | 54.64 GB | 17.580B tok iter 134140 | train 2.739 | val 2.620 | lr 7.96e-05 | 102,496 tok/s | 54.64 GB | 17.582B tok iter 134160 | train 2.688 | val 2.707 | lr 7.96e-05 | 102,497 tok/s | 54.64 GB | 17.585B tok iter 134180 | train 2.595 | val 2.616 | lr 7.95e-05 | 102,497 tok/s | 54.64 GB | 17.587B tok iter 134200 | train 2.600 | val 2.651 | lr 7.95e-05 | 102,498 tok/s | 54.64 GB | 17.590B tok iter 134220 | train 2.675 | val 2.643 | lr 7.95e-05 | 102,500 tok/s | 54.64 GB | 17.593B tok iter 134240 | train 2.692 | val 2.669 | lr 7.94e-05 | 102,501 tok/s | 54.64 GB | 17.595B tok iter 134260 | train 2.628 | val 2.616 | lr 7.94e-05 | 102,501 tok/s | 54.64 GB | 17.598B tok iter 134280 | train 2.684 | val 2.649 | lr 7.93e-05 | 102,502 tok/s | 54.64 GB | 17.600B tok iter 134300 | train 2.732 | val 2.660 | lr 7.93e-05 | 102,503 tok/s | 54.64 GB | 17.603B tok iter 134320 | train 2.630 | val 2.629 | lr 7.93e-05 | 102,504 tok/s | 54.64 GB | 17.606B tok iter 134340 | train 2.448 | val 2.643 | lr 7.92e-05 | 102,505 tok/s | 54.64 GB | 17.608B tok iter 134360 | train 2.483 | val 2.642 | lr 7.92e-05 | 102,506 tok/s | 54.64 GB | 17.611B tok iter 134380 | train 2.678 | val 2.654 | lr 7.91e-05 | 102,507 tok/s | 54.64 GB | 17.614B tok iter 134400 | train 2.576 | val 2.639 | lr 7.91e-05 | 102,508 tok/s | 54.64 GB | 17.616B tok iter 134420 | train 2.588 | val 2.590 | lr 7.90e-05 | 102,508 tok/s | 54.64 GB | 17.619B tok iter 134440 | train 2.620 | val 2.647 | lr 7.90e-05 | 102,509 tok/s | 54.64 GB | 17.621B tok iter 134460 | train 2.626 | val 2.651 | lr 7.90e-05 | 102,511 tok/s | 54.64 GB | 17.624B tok iter 134480 | train 2.641 | val 2.640 | lr 7.89e-05 | 102,512 tok/s | 54.64 GB | 17.627B tok checkpoint @ iter 134491 -> checkpoints/500m_latest.pt iter 134500 | train 2.652 | val 2.673 | lr 7.89e-05 | 102,505 tok/s | 54.64 GB | 17.629B tok checkpoint @ iter 134500 -> checkpoints/500m_latest.pt iter 134520 | train 2.502 | val 2.609 | lr 7.88e-05 | 102,491 tok/s | 54.64 GB | 17.632B tok iter 134540 | train 2.703 | val 2.621 | lr 7.88e-05 | 102,492 tok/s | 54.64 GB | 17.635B tok iter 134560 | train 2.676 | val 2.624 | lr 7.88e-05 | 102,493 tok/s | 54.64 GB | 17.637B tok iter 134580 | train 2.737 | val 2.660 | lr 7.87e-05 | 102,493 tok/s | 54.64 GB | 17.640B tok iter 134600 | train 2.745 | val 2.641 | lr 7.87e-05 | 102,494 tok/s | 54.64 GB | 17.642B tok iter 134620 | train 2.609 | val 2.661 | lr 7.86e-05 | 102,496 tok/s | 54.64 GB | 17.645B tok iter 134640 | train 2.611 | val 2.650 | lr 7.86e-05 | 102,497 tok/s | 54.64 GB | 17.648B tok iter 134660 | train 2.678 | val 2.618 | lr 7.85e-05 | 102,497 tok/s | 54.64 GB | 17.650B tok iter 134680 | train 2.629 | val 2.685 | lr 7.85e-05 | 102,498 tok/s | 54.64 GB | 17.653B tok iter 134700 | train 2.813 | val 2.698 | lr 7.85e-05 | 102,499 tok/s | 54.64 GB | 17.656B tok iter 134720 | train 2.689 | val 2.645 | lr 7.84e-05 | 102,501 tok/s | 54.64 GB | 17.658B tok iter 134740 | train 2.710 | val 2.653 | lr 7.84e-05 | 102,501 tok/s | 54.64 GB | 17.661B tok iter 134760 | train 2.731 | val 2.675 | lr 7.83e-05 | 102,502 tok/s | 54.64 GB | 17.663B tok iter 134780 | train 2.503 | val 2.611 | lr 7.83e-05 | 102,503 tok/s | 54.64 GB | 17.666B tok iter 134800 | train 2.678 | val 2.673 | lr 7.83e-05 | 102,504 tok/s | 54.64 GB | 17.669B tok iter 134820 | train 2.696 | val 2.644 | lr 7.82e-05 | 102,505 tok/s | 54.64 GB | 17.671B tok iter 134840 | train 2.780 | val 2.676 | lr 7.82e-05 | 102,506 tok/s | 54.64 GB | 17.674B tok iter 134860 | train 2.601 | val 2.628 | lr 7.81e-05 | 102,507 tok/s | 54.64 GB | 17.677B tok iter 134880 | train 2.636 | val 2.698 | lr 7.81e-05 | 102,508 tok/s | 54.64 GB | 17.679B tok iter 134900 | train 2.738 | val 2.612 | lr 7.81e-05 | 102,508 tok/s | 54.64 GB | 17.682B tok iter 134920 | train 2.700 | val 2.653 | lr 7.80e-05 | 102,510 tok/s | 54.64 GB | 17.684B tok iter 134940 | train 2.509 | val 2.649 | lr 7.80e-05 | 102,511 tok/s | 54.64 GB | 17.687B tok iter 134960 | train 2.650 | val 2.615 | lr 7.79e-05 | 102,512 tok/s | 54.64 GB | 17.690B tok iter 134980 | train 2.535 | val 2.623 | lr 7.79e-05 | 102,512 tok/s | 54.64 GB | 17.692B tok checkpoint @ iter 134991 -> checkpoints/500m_latest.pt iter 135000 | train 2.697 | val 2.646 | lr 7.79e-05 | 102,505 tok/s | 54.64 GB | 17.695B tok checkpoint @ iter 135000 -> checkpoints/500m_latest.pt iter 135020 | train 2.562 | val 2.643 | lr 7.78e-05 | 102,491 tok/s | 54.64 GB | 17.697B tok iter 135040 | train 2.661 | val 2.603 | lr 7.78e-05 | 102,492 tok/s | 54.64 GB | 17.700B tok iter 135060 | train 2.585 | val 2.674 | lr 7.77e-05 | 102,492 tok/s | 54.64 GB | 17.703B tok iter 135080 | train 2.616 | val 2.660 | lr 7.77e-05 | 102,493 tok/s | 54.64 GB | 17.705B tok iter 135100 | train 2.575 | val 2.640 | lr 7.77e-05 | 102,495 tok/s | 54.64 GB | 17.708B tok iter 135120 | train 2.648 | val 2.623 | lr 7.76e-05 | 102,496 tok/s | 54.64 GB | 17.711B tok iter 135140 | train 2.493 | val 2.710 | lr 7.76e-05 | 102,496 tok/s | 54.64 GB | 17.713B tok iter 135160 | train 2.690 | val 2.640 | lr 7.75e-05 | 102,497 tok/s | 54.64 GB | 17.716B tok iter 135180 | train 2.809 | val 2.648 | lr 7.75e-05 | 102,498 tok/s | 54.64 GB | 17.718B tok iter 135200 | train 2.583 | val 2.661 | lr 7.75e-05 | 102,499 tok/s | 54.64 GB | 17.721B tok iter 135220 | train 2.702 | val 2.576 | lr 7.74e-05 | 102,500 tok/s | 54.64 GB | 17.724B tok iter 135240 | train 2.482 | val 2.670 | lr 7.74e-05 | 102,501 tok/s | 54.64 GB | 17.726B tok iter 135260 | train 2.763 | val 2.673 | lr 7.73e-05 | 102,502 tok/s | 54.64 GB | 17.729B tok iter 135280 | train 2.610 | val 2.591 | lr 7.73e-05 | 102,503 tok/s | 54.64 GB | 17.732B tok iter 135300 | train 2.657 | val 2.604 | lr 7.73e-05 | 102,503 tok/s | 54.64 GB | 17.734B tok iter 135320 | train 2.704 | val 2.655 | lr 7.72e-05 | 102,505 tok/s | 54.64 GB | 17.737B tok iter 135340 | train 2.584 | val 2.642 | lr 7.72e-05 | 102,506 tok/s | 54.64 GB | 17.739B tok iter 135360 | train 2.787 | val 2.578 | lr 7.71e-05 | 102,507 tok/s | 54.64 GB | 17.742B tok iter 135380 | train 2.600 | val 2.650 | lr 7.71e-05 | 102,507 tok/s | 54.64 GB | 17.745B tok iter 135400 | train 2.592 | val 2.646 | lr 7.71e-05 | 102,508 tok/s | 54.64 GB | 17.747B tok iter 135420 | train 2.744 | val 2.642 | lr 7.70e-05 | 102,509 tok/s | 54.64 GB | 17.750B tok iter 135440 | train 2.698 | val 2.612 | lr 7.70e-05 | 102,511 tok/s | 54.64 GB | 17.753B tok iter 135460 | train 2.600 | val 2.662 | lr 7.69e-05 | 102,511 tok/s | 54.64 GB | 17.755B tok iter 135480 | train 2.696 | val 2.607 | lr 7.69e-05 | 102,512 tok/s | 54.64 GB | 17.758B tok checkpoint @ iter 135490 -> checkpoints/500m_latest.pt iter 135500 | train 2.538 | val 2.633 | lr 7.69e-05 | 102,505 tok/s | 54.64 GB | 17.760B tok checkpoint @ iter 135500 -> checkpoints/500m_latest.pt iter 135520 | train 2.550 | val 2.602 | lr 7.68e-05 | 102,492 tok/s | 54.64 GB | 17.763B tok iter 135540 | train 2.659 | val 2.661 | lr 7.68e-05 | 102,492 tok/s | 54.64 GB | 17.766B tok iter 135560 | train 2.472 | val 2.665 | lr 7.68e-05 | 102,493 tok/s | 54.64 GB | 17.768B tok iter 135580 | train 2.676 | val 2.651 | lr 7.67e-05 | 102,494 tok/s | 54.64 GB | 17.771B tok iter 135600 | train 2.549 | val 2.652 | lr 7.67e-05 | 102,495 tok/s | 54.64 GB | 17.773B tok iter 135620 | train 2.534 | val 2.676 | lr 7.66e-05 | 102,496 tok/s | 54.64 GB | 17.776B tok iter 135640 | train 2.631 | val 2.641 | lr 7.66e-05 | 102,497 tok/s | 54.64 GB | 17.779B tok iter 135660 | train 2.634 | val 2.603 | lr 7.66e-05 | 102,498 tok/s | 54.64 GB | 17.781B tok iter 135680 | train 2.470 | val 2.638 | lr 7.65e-05 | 102,499 tok/s | 54.64 GB | 17.784B tok iter 135700 | train 2.470 | val 2.634 | lr 7.65e-05 | 102,499 tok/s | 54.64 GB | 17.787B tok iter 135720 | train 2.568 | val 2.620 | lr 7.64e-05 | 102,500 tok/s | 54.64 GB | 17.789B tok iter 135740 | train 2.670 | val 2.645 | lr 7.64e-05 | 102,502 tok/s | 54.64 GB | 17.792B tok iter 135760 | train 2.797 | val 2.676 | lr 7.64e-05 | 102,503 tok/s | 54.64 GB | 17.794B tok iter 135780 | train 2.517 | val 2.640 | lr 7.63e-05 | 102,503 tok/s | 54.64 GB | 17.797B tok iter 135800 | train 2.731 | val 2.656 | lr 7.63e-05 | 102,504 tok/s | 54.64 GB | 17.800B tok iter 135820 | train 2.734 | val 2.674 | lr 7.62e-05 | 102,505 tok/s | 54.64 GB | 17.802B tok iter 135840 | train 2.474 | val 2.680 | lr 7.62e-05 | 102,507 tok/s | 54.64 GB | 17.805B tok iter 135860 | train 2.680 | val 2.641 | lr 7.62e-05 | 102,507 tok/s | 54.64 GB | 17.808B tok iter 135880 | train 2.745 | val 2.657 | lr 7.61e-05 | 102,508 tok/s | 54.64 GB | 17.810B tok iter 135900 | train 2.535 | val 2.693 | lr 7.61e-05 | 102,509 tok/s | 54.64 GB | 17.813B tok iter 135920 | train 2.553 | val 2.568 | lr 7.61e-05 | 102,510 tok/s | 54.64 GB | 17.815B tok iter 135940 | train 2.692 | val 2.596 | lr 7.60e-05 | 102,510 tok/s | 54.64 GB | 17.818B tok iter 135960 | train 2.638 | val 2.677 | lr 7.60e-05 | 102,512 tok/s | 54.64 GB | 17.821B tok iter 135980 | train 2.578 | val 2.599 | lr 7.59e-05 | 102,513 tok/s | 54.64 GB | 17.823B tok checkpoint @ iter 135991 -> checkpoints/500m_latest.pt iter 136000 | train 2.513 | val 2.656 | lr 7.59e-05 | 102,506 tok/s | 54.64 GB | 17.826B tok checkpoint @ iter 136000 -> checkpoints/500m_latest.pt iter 136020 | train 2.655 | val 2.639 | lr 7.59e-05 | 102,491 tok/s | 54.64 GB | 17.829B tok iter 136040 | train 2.639 | val 2.641 | lr 7.58e-05 | 102,492 tok/s | 54.64 GB | 17.831B tok iter 136060 | train 2.566 | val 2.665 | lr 7.58e-05 | 102,494 tok/s | 54.64 GB | 17.834B tok iter 136080 | train 2.673 | val 2.608 | lr 7.58e-05 | 102,495 tok/s | 54.64 GB | 17.836B tok iter 136100 | train 2.639 | val 2.581 | lr 7.57e-05 | 102,495 tok/s | 54.64 GB | 17.839B tok iter 136120 | train 2.546 | val 2.559 | lr 7.57e-05 | 102,496 tok/s | 54.64 GB | 17.842B tok iter 136140 | train 2.575 | val 2.620 | lr 7.56e-05 | 102,497 tok/s | 54.64 GB | 17.844B tok iter 136160 | train 2.637 | val 2.667 | lr 7.56e-05 | 102,498 tok/s | 54.64 GB | 17.847B tok iter 136180 | train 2.626 | val 2.640 | lr 7.56e-05 | 102,499 tok/s | 54.64 GB | 17.850B tok iter 136200 | train 2.620 | val 2.689 | lr 7.55e-05 | 102,500 tok/s | 54.64 GB | 17.852B tok iter 136220 | train 2.654 | val 2.661 | lr 7.55e-05 | 102,501 tok/s | 54.64 GB | 17.855B tok iter 136240 | train 2.644 | val 2.612 | lr 7.55e-05 | 102,502 tok/s | 54.64 GB | 17.857B tok iter 136260 | train 2.723 | val 2.637 | lr 7.54e-05 | 102,502 tok/s | 54.64 GB | 17.860B tok iter 136280 | train 2.683 | val 2.634 | lr 7.54e-05 | 102,503 tok/s | 54.64 GB | 17.863B tok iter 136300 | train 2.627 | val 2.634 | lr 7.53e-05 | 102,505 tok/s | 54.64 GB | 17.865B tok iter 136320 | train 2.488 | val 2.651 | lr 7.53e-05 | 102,506 tok/s | 54.64 GB | 17.868B tok iter 136340 | train 2.683 | val 2.645 | lr 7.53e-05 | 102,506 tok/s | 54.64 GB | 17.870B tok iter 136360 | train 2.627 | val 2.636 | lr 7.52e-05 | 102,507 tok/s | 54.64 GB | 17.873B tok iter 136380 | train 2.579 | val 2.632 | lr 7.52e-05 | 102,508 tok/s | 54.64 GB | 17.876B tok iter 136400 | train 2.532 | val 2.692 | lr 7.52e-05 | 102,510 tok/s | 54.64 GB | 17.878B tok iter 136420 | train 2.565 | val 2.614 | lr 7.51e-05 | 102,510 tok/s | 54.64 GB | 17.881B tok iter 136440 | train 2.645 | val 2.594 | lr 7.51e-05 | 102,511 tok/s | 54.64 GB | 17.884B tok iter 136460 | train 2.568 | val 2.607 | lr 7.50e-05 | 102,512 tok/s | 54.64 GB | 17.886B tok iter 136480 | train 2.598 | val 2.605 | lr 7.50e-05 | 102,513 tok/s | 54.64 GB | 17.889B tok checkpoint @ iter 136491 -> checkpoints/500m_latest.pt iter 136500 | train 2.660 | val 2.684 | lr 7.50e-05 | 102,506 tok/s | 54.64 GB | 17.891B tok checkpoint @ iter 136500 -> checkpoints/500m_latest.pt iter 136520 | train 2.754 | val 2.629 | lr 7.49e-05 | 102,492 tok/s | 54.64 GB | 17.894B tok iter 136540 | train 2.831 | val 2.581 | lr 7.49e-05 | 102,494 tok/s | 54.64 GB | 17.897B tok iter 136560 | train 2.525 | val 2.676 | lr 7.49e-05 | 102,495 tok/s | 54.64 GB | 17.899B tok iter 136580 | train 2.538 | val 2.673 | lr 7.48e-05 | 102,495 tok/s | 54.64 GB | 17.902B tok iter 136600 | train 2.545 | val 2.656 | lr 7.48e-05 | 102,496 tok/s | 54.64 GB | 17.905B tok iter 136620 | train 2.617 | val 2.649 | lr 7.47e-05 | 102,497 tok/s | 54.64 GB | 17.907B tok iter 136640 | train 2.628 | val 2.666 | lr 7.47e-05 | 102,498 tok/s | 54.64 GB | 17.910B tok iter 136660 | train 2.669 | val 2.642 | lr 7.47e-05 | 102,499 tok/s | 54.64 GB | 17.912B tok iter 136680 | train 2.631 | val 2.661 | lr 7.46e-05 | 102,500 tok/s | 54.64 GB | 17.915B tok iter 136700 | train 2.651 | val 2.629 | lr 7.46e-05 | 102,501 tok/s | 54.64 GB | 17.918B tok iter 136720 | train 2.497 | val 2.646 | lr 7.46e-05 | 102,502 tok/s | 54.64 GB | 17.920B tok iter 136740 | train 2.589 | val 2.674 | lr 7.45e-05 | 102,502 tok/s | 54.64 GB | 17.923B tok iter 136760 | train 2.601 | val 2.588 | lr 7.45e-05 | 102,503 tok/s | 54.64 GB | 17.926B tok iter 136780 | train 2.627 | val 2.633 | lr 7.45e-05 | 102,505 tok/s | 54.64 GB | 17.928B tok iter 136800 | train 2.661 | val 2.671 | lr 7.44e-05 | 102,506 tok/s | 54.64 GB | 17.931B tok iter 136820 | train 2.520 | val 2.642 | lr 7.44e-05 | 102,506 tok/s | 54.64 GB | 17.933B tok iter 136840 | train 2.632 | val 2.583 | lr 7.43e-05 | 102,507 tok/s | 54.64 GB | 17.936B tok iter 136860 | train 2.535 | val 2.614 | lr 7.43e-05 | 102,508 tok/s | 54.64 GB | 17.939B tok iter 136880 | train 2.684 | val 2.639 | lr 7.43e-05 | 102,509 tok/s | 54.64 GB | 17.941B tok iter 136900 | train 2.691 | val 2.623 | lr 7.42e-05 | 102,510 tok/s | 54.64 GB | 17.944B tok iter 136920 | train 2.694 | val 2.642 | lr 7.42e-05 | 102,511 tok/s | 54.64 GB | 17.947B tok iter 136940 | train 2.594 | val 2.615 | lr 7.42e-05 | 102,512 tok/s | 54.64 GB | 17.949B tok iter 136960 | train 2.602 | val 2.636 | lr 7.41e-05 | 102,513 tok/s | 54.64 GB | 17.952B tok iter 136980 | train 2.612 | val 2.665 | lr 7.41e-05 | 102,513 tok/s | 54.64 GB | 17.954B tok checkpoint @ iter 136990 -> checkpoints/500m_latest.pt iter 137000 | train 2.610 | val 2.623 | lr 7.41e-05 | 102,507 tok/s | 54.64 GB | 17.957B tok checkpoint @ iter 137000 -> checkpoints/500m_latest.pt iter 137020 | train 2.637 | val 2.600 | lr 7.40e-05 | 102,494 tok/s | 54.64 GB | 17.960B tok iter 137040 | train 2.546 | val 2.653 | lr 7.40e-05 | 102,495 tok/s | 54.64 GB | 17.962B tok iter 137060 | train 2.642 | val 2.655 | lr 7.40e-05 | 102,495 tok/s | 54.64 GB | 17.965B tok iter 137080 | train 2.647 | val 2.629 | lr 7.39e-05 | 102,496 tok/s | 54.64 GB | 17.967B tok iter 137100 | train 2.546 | val 2.584 | lr 7.39e-05 | 102,497 tok/s | 54.64 GB | 17.970B tok iter 137120 | train 2.716 | val 2.605 | lr 7.38e-05 | 102,499 tok/s | 54.64 GB | 17.973B tok iter 137140 | train 2.453 | val 2.601 | lr 7.38e-05 | 102,499 tok/s | 54.64 GB | 17.975B tok iter 137160 | train 2.667 | val 2.617 | lr 7.38e-05 | 102,500 tok/s | 54.64 GB | 17.978B tok iter 137180 | train 2.651 | val 2.574 | lr 7.37e-05 | 102,501 tok/s | 54.64 GB | 17.981B tok iter 137200 | train 2.632 | val 2.627 | lr 7.37e-05 | 102,502 tok/s | 54.64 GB | 17.983B tok iter 137220 | train 2.615 | val 2.597 | lr 7.37e-05 | 102,502 tok/s | 54.64 GB | 17.986B tok iter 137240 | train 2.675 | val 2.638 | lr 7.36e-05 | 102,504 tok/s | 54.64 GB | 17.988B tok iter 137260 | train 2.683 | val 2.650 | lr 7.36e-05 | 102,505 tok/s | 54.64 GB | 17.991B tok iter 137280 | train 2.605 | val 2.631 | lr 7.36e-05 | 102,506 tok/s | 54.64 GB | 17.994B tok iter 137300 | train 2.673 | val 2.642 | lr 7.35e-05 | 102,506 tok/s | 54.64 GB | 17.996B tok iter 137320 | train 2.696 | val 2.585 | lr 7.35e-05 | 102,507 tok/s | 54.64 GB | 17.999B tok iter 137340 | train 2.485 | val 2.626 | lr 7.35e-05 | 102,508 tok/s | 54.64 GB | 18.002B tok iter 137360 | train 2.520 | val 2.628 | lr 7.34e-05 | 102,510 tok/s | 54.64 GB | 18.004B tok iter 137380 | train 2.609 | val 2.604 | lr 7.34e-05 | 102,510 tok/s | 54.64 GB | 18.007B tok iter 137400 | train 2.633 | val 2.618 | lr 7.34e-05 | 102,511 tok/s | 54.64 GB | 18.009B tok iter 137420 | train 2.659 | val 2.650 | lr 7.33e-05 | 102,512 tok/s | 54.64 GB | 18.012B tok iter 137440 | train 2.644 | val 2.614 | lr 7.33e-05 | 102,513 tok/s | 54.64 GB | 18.015B tok iter 137460 | train 2.464 | val 2.597 | lr 7.33e-05 | 102,513 tok/s | 54.64 GB | 18.017B tok iter 137480 | train 2.549 | val 2.600 | lr 7.32e-05 | 102,514 tok/s | 54.64 GB | 18.020B tok checkpoint @ iter 137491 -> checkpoints/500m_latest.pt iter 137500 | train 2.555 | val 2.632 | lr 7.32e-05 | 102,508 tok/s | 54.64 GB | 18.023B tok checkpoint @ iter 137500 -> checkpoints/500m_latest.pt iter 137520 | train 2.734 | val 2.633 | lr 7.31e-05 | 102,494 tok/s | 54.64 GB | 18.025B tok iter 137540 | train 2.629 | val 2.646 | lr 7.31e-05 | 102,494 tok/s | 54.64 GB | 18.028B tok iter 137560 | train 2.520 | val 2.655 | lr 7.31e-05 | 102,496 tok/s | 54.64 GB | 18.030B tok iter 137580 | train 2.659 | val 2.603 | lr 7.30e-05 | 102,497 tok/s | 54.64 GB | 18.033B tok iter 137600 | train 2.757 | val 2.611 | lr 7.30e-05 | 102,498 tok/s | 54.64 GB | 18.036B tok iter 137620 | train 2.780 | val 2.675 | lr 7.30e-05 | 102,498 tok/s | 54.64 GB | 18.038B tok iter 137640 | train 2.602 | val 2.676 | lr 7.29e-05 | 102,499 tok/s | 54.64 GB | 18.041B tok iter 137660 | train 2.649 | val 2.629 | lr 7.29e-05 | 102,500 tok/s | 54.64 GB | 18.044B tok iter 137680 | train 2.667 | val 2.647 | lr 7.29e-05 | 102,502 tok/s | 54.64 GB | 18.046B tok iter 137700 | train 2.519 | val 2.624 | lr 7.28e-05 | 102,502 tok/s | 54.64 GB | 18.049B tok iter 137720 | train 2.545 | val 2.636 | lr 7.28e-05 | 102,503 tok/s | 54.64 GB | 18.051B tok iter 137740 | train 2.623 | val 2.577 | lr 7.28e-05 | 102,504 tok/s | 54.64 GB | 18.054B tok iter 137760 | train 2.602 | val 2.646 | lr 7.27e-05 | 102,505 tok/s | 54.64 GB | 18.057B tok iter 137780 | train 2.546 | val 2.667 | lr 7.27e-05 | 102,505 tok/s | 54.64 GB | 18.059B tok iter 137800 | train 2.561 | val 2.605 | lr 7.27e-05 | 102,506 tok/s | 54.64 GB | 18.062B tok iter 137820 | train 2.551 | val 2.612 | lr 7.26e-05 | 102,508 tok/s | 54.64 GB | 18.064B tok iter 137840 | train 2.454 | val 2.630 | lr 7.26e-05 | 102,509 tok/s | 54.64 GB | 18.067B tok iter 137860 | train 2.599 | val 2.666 | lr 7.26e-05 | 102,509 tok/s | 54.64 GB | 18.070B tok iter 137880 | train 2.665 | val 2.598 | lr 7.25e-05 | 102,510 tok/s | 54.64 GB | 18.072B tok iter 137900 | train 2.706 | val 2.601 | lr 7.25e-05 | 102,511 tok/s | 54.64 GB | 18.075B tok iter 137920 | train 2.725 | val 2.619 | lr 7.25e-05 | 102,512 tok/s | 54.64 GB | 18.078B tok iter 137940 | train 2.592 | val 2.735 | lr 7.24e-05 | 102,513 tok/s | 54.64 GB | 18.080B tok iter 137960 | train 2.757 | val 2.625 | lr 7.24e-05 | 102,514 tok/s | 54.64 GB | 18.083B tok iter 137980 | train 2.575 | val 2.612 | lr 7.24e-05 | 102,515 tok/s | 54.64 GB | 18.085B tok checkpoint @ iter 137990 -> checkpoints/500m_latest.pt iter 138000 | train 2.344 | val 2.644 | lr 7.23e-05 | 102,509 tok/s | 54.64 GB | 18.088B tok checkpoint @ iter 138000 -> checkpoints/500m_latest.pt iter 138020 | train 2.667 | val 2.668 | lr 7.23e-05 | 102,494 tok/s | 54.64 GB | 18.091B tok iter 138040 | train 2.801 | val 2.619 | lr 7.23e-05 | 102,495 tok/s | 54.64 GB | 18.093B tok iter 138060 | train 2.644 | val 2.596 | lr 7.22e-05 | 102,497 tok/s | 54.64 GB | 18.096B tok iter 138080 | train 2.671 | val 2.610 | lr 7.22e-05 | 102,498 tok/s | 54.64 GB | 18.099B tok iter 138100 | train 2.620 | val 2.618 | lr 7.22e-05 | 102,498 tok/s | 54.64 GB | 18.101B tok iter 138120 | train 2.629 | val 2.606 | lr 7.21e-05 | 102,499 tok/s | 54.64 GB | 18.104B tok iter 138140 | train 2.655 | val 2.621 | lr 7.21e-05 | 102,500 tok/s | 54.64 GB | 18.106B tok iter 138160 | train 2.556 | val 2.603 | lr 7.21e-05 | 102,501 tok/s | 54.64 GB | 18.109B tok iter 138180 | train 2.627 | val 2.574 | lr 7.20e-05 | 102,502 tok/s | 54.64 GB | 18.112B tok iter 138200 | train 2.624 | val 2.679 | lr 7.20e-05 | 102,503 tok/s | 54.64 GB | 18.114B tok iter 138220 | train 2.591 | val 2.625 | lr 7.20e-05 | 102,504 tok/s | 54.64 GB | 18.117B tok iter 138240 | train 2.708 | val 2.615 | lr 7.19e-05 | 102,505 tok/s | 54.64 GB | 18.120B tok iter 138260 | train 2.555 | val 2.633 | lr 7.19e-05 | 102,505 tok/s | 54.64 GB | 18.122B tok iter 138280 | train 2.786 | val 2.602 | lr 7.19e-05 | 102,506 tok/s | 54.64 GB | 18.125B tok iter 138300 | train 2.590 | val 2.688 | lr 7.18e-05 | 102,508 tok/s | 54.64 GB | 18.127B tok iter 138320 | train 2.658 | val 2.651 | lr 7.18e-05 | 102,509 tok/s | 54.64 GB | 18.130B tok iter 138340 | train 2.630 | val 2.618 | lr 7.18e-05 | 102,509 tok/s | 54.64 GB | 18.133B tok iter 138360 | train 2.566 | val 2.657 | lr 7.17e-05 | 102,510 tok/s | 54.64 GB | 18.135B tok iter 138380 | train 2.626 | val 2.653 | lr 7.17e-05 | 102,511 tok/s | 54.64 GB | 18.138B tok iter 138400 | train 2.696 | val 2.645 | lr 7.17e-05 | 102,512 tok/s | 54.64 GB | 18.140B tok iter 138420 | train 2.653 | val 2.644 | lr 7.16e-05 | 102,513 tok/s | 54.64 GB | 18.143B tok iter 138440 | train 2.521 | val 2.680 | lr 7.16e-05 | 102,514 tok/s | 54.64 GB | 18.146B tok iter 138460 | train 2.690 | val 2.576 | lr 7.16e-05 | 102,515 tok/s | 54.64 GB | 18.148B tok iter 138480 | train 2.507 | val 2.611 | lr 7.15e-05 | 102,516 tok/s | 54.64 GB | 18.151B tok checkpoint @ iter 138491 -> checkpoints/500m_latest.pt iter 138500 | train 2.432 | val 2.621 | lr 7.15e-05 | 102,509 tok/s | 54.64 GB | 18.154B tok checkpoint @ iter 138500 -> checkpoints/500m_latest.pt iter 138520 | train 2.571 | val 2.572 | lr 7.15e-05 | 102,496 tok/s | 54.64 GB | 18.156B tok iter 138540 | train 2.542 | val 2.674 | lr 7.14e-05 | 102,497 tok/s | 54.64 GB | 18.159B tok iter 138560 | train 2.528 | val 2.611 | lr 7.14e-05 | 102,498 tok/s | 54.64 GB | 18.161B tok iter 138580 | train 2.551 | val 2.634 | lr 7.14e-05 | 102,498 tok/s | 54.64 GB | 18.164B tok iter 138600 | train 2.672 | val 2.614 | lr 7.13e-05 | 102,499 tok/s | 54.64 GB | 18.167B tok iter 138620 | train 2.681 | val 2.667 | lr 7.13e-05 | 102,500 tok/s | 54.64 GB | 18.169B tok iter 138640 | train 2.597 | val 2.605 | lr 7.13e-05 | 102,502 tok/s | 54.64 GB | 18.172B tok iter 138660 | train 2.491 | val 2.617 | lr 7.12e-05 | 102,502 tok/s | 54.64 GB | 18.175B tok iter 138680 | train 2.620 | val 2.656 | lr 7.12e-05 | 102,503 tok/s | 54.64 GB | 18.177B tok iter 138700 | train 2.506 | val 2.710 | lr 7.12e-05 | 102,504 tok/s | 54.64 GB | 18.180B tok iter 138720 | train 2.579 | val 2.615 | lr 7.12e-05 | 102,505 tok/s | 54.64 GB | 18.182B tok iter 138740 | train 2.684 | val 2.625 | lr 7.11e-05 | 102,505 tok/s | 54.64 GB | 18.185B tok iter 138760 | train 2.505 | val 2.630 | lr 7.11e-05 | 102,507 tok/s | 54.64 GB | 18.188B tok iter 138780 | train 2.686 | val 2.630 | lr 7.11e-05 | 102,508 tok/s | 54.64 GB | 18.190B tok iter 138800 | train 2.683 | val 2.658 | lr 7.10e-05 | 102,509 tok/s | 54.64 GB | 18.193B tok iter 138820 | train 2.657 | val 2.595 | lr 7.10e-05 | 102,509 tok/s | 54.64 GB | 18.196B tok iter 138840 | train 2.544 | val 2.581 | lr 7.10e-05 | 102,510 tok/s | 54.64 GB | 18.198B tok iter 138860 | train 2.556 | val 2.625 | lr 7.09e-05 | 102,511 tok/s | 54.64 GB | 18.201B tok iter 138880 | train 2.603 | val 2.641 | lr 7.09e-05 | 102,512 tok/s | 54.64 GB | 18.203B tok iter 138900 | train 2.598 | val 2.623 | lr 7.09e-05 | 102,513 tok/s | 54.64 GB | 18.206B tok iter 138920 | train 2.629 | val 2.576 | lr 7.08e-05 | 102,514 tok/s | 54.64 GB | 18.209B tok iter 138940 | train 2.746 | val 2.604 | lr 7.08e-05 | 102,515 tok/s | 54.64 GB | 18.211B tok iter 138960 | train 2.644 | val 2.648 | lr 7.08e-05 | 102,516 tok/s | 54.64 GB | 18.214B tok iter 138980 | train 2.585 | val 2.597 | lr 7.07e-05 | 102,516 tok/s | 54.64 GB | 18.217B tok checkpoint @ iter 138991 -> checkpoints/500m_latest.pt iter 139000 | train 2.774 | val 2.635 | lr 7.07e-05 | 102,510 tok/s | 54.64 GB | 18.219B tok checkpoint @ iter 139000 -> checkpoints/500m_latest.pt iter 139020 | train 2.759 | val 2.636 | lr 7.07e-05 | 102,497 tok/s | 54.64 GB | 18.222B tok iter 139040 | train 2.624 | val 2.615 | lr 7.06e-05 | 102,498 tok/s | 54.64 GB | 18.224B tok iter 139060 | train 2.560 | val 2.620 | lr 7.06e-05 | 102,498 tok/s | 54.64 GB | 18.227B tok iter 139080 | train 2.421 | val 2.626 | lr 7.06e-05 | 102,499 tok/s | 54.64 GB | 18.230B tok iter 139100 | train 2.626 | val 2.641 | lr 7.06e-05 | 102,500 tok/s | 54.64 GB | 18.232B tok iter 139120 | train 2.647 | val 2.631 | lr 7.05e-05 | 102,501 tok/s | 54.64 GB | 18.235B tok iter 139140 | train 2.732 | val 2.607 | lr 7.05e-05 | 102,502 tok/s | 54.64 GB | 18.237B tok iter 139160 | train 2.776 | val 2.613 | lr 7.05e-05 | 102,503 tok/s | 54.64 GB | 18.240B tok iter 139180 | train 2.635 | val 2.636 | lr 7.04e-05 | 102,504 tok/s | 54.64 GB | 18.243B tok iter 139200 | train 2.513 | val 2.645 | lr 7.04e-05 | 102,505 tok/s | 54.64 GB | 18.245B tok iter 139220 | train 2.619 | val 2.616 | lr 7.04e-05 | 102,505 tok/s | 54.64 GB | 18.248B tok iter 139240 | train 2.498 | val 2.632 | lr 7.03e-05 | 102,506 tok/s | 54.64 GB | 18.251B tok iter 139260 | train 2.657 | val 2.625 | lr 7.03e-05 | 102,507 tok/s | 54.64 GB | 18.253B tok iter 139280 | train 2.618 | val 2.621 | lr 7.03e-05 | 102,509 tok/s | 54.64 GB | 18.256B tok iter 139300 | train 2.667 | val 2.610 | lr 7.02e-05 | 102,509 tok/s | 54.64 GB | 18.258B tok iter 139320 | train 2.644 | val 2.635 | lr 7.02e-05 | 102,510 tok/s | 54.64 GB | 18.261B tok iter 139340 | train 2.607 | val 2.639 | lr 7.02e-05 | 102,511 tok/s | 54.64 GB | 18.264B tok iter 139360 | train 2.576 | val 2.628 | lr 7.02e-05 | 102,512 tok/s | 54.64 GB | 18.266B tok iter 139380 | train 2.513 | val 2.616 | lr 7.01e-05 | 102,512 tok/s | 54.64 GB | 18.269B tok iter 139400 | train 2.719 | val 2.607 | lr 7.01e-05 | 102,514 tok/s | 54.64 GB | 18.272B tok iter 139420 | train 2.542 | val 2.638 | lr 7.01e-05 | 102,515 tok/s | 54.64 GB | 18.274B tok iter 139440 | train 2.608 | val 2.625 | lr 7.00e-05 | 102,516 tok/s | 54.64 GB | 18.277B tok iter 139460 | train 2.603 | val 2.656 | lr 7.00e-05 | 102,516 tok/s | 54.64 GB | 18.279B tok iter 139480 | train 2.583 | val 2.611 | lr 7.00e-05 | 102,517 tok/s | 54.64 GB | 18.282B tok checkpoint @ iter 139490 -> checkpoints/500m_latest.pt iter 139500 | train 2.544 | val 2.599 | lr 6.99e-05 | 102,511 tok/s | 54.64 GB | 18.285B tok checkpoint @ iter 139500 -> checkpoints/500m_latest.pt iter 139520 | train 2.449 | val 2.584 | lr 6.99e-05 | 102,497 tok/s | 54.64 GB | 18.287B tok iter 139540 | train 2.591 | val 2.651 | lr 6.99e-05 | 102,497 tok/s | 54.64 GB | 18.290B tok iter 139560 | train 2.668 | val 2.667 | lr 6.98e-05 | 102,498 tok/s | 54.64 GB | 18.293B tok iter 139580 | train 2.800 | val 2.621 | lr 6.98e-05 | 102,499 tok/s | 54.64 GB | 18.295B tok iter 139600 | train 2.629 | val 2.621 | lr 6.98e-05 | 102,500 tok/s | 54.64 GB | 18.298B tok iter 139620 | train 2.543 | val 2.660 | lr 6.98e-05 | 102,500 tok/s | 54.64 GB | 18.300B tok iter 139640 | train 2.675 | val 2.639 | lr 6.97e-05 | 102,502 tok/s | 54.64 GB | 18.303B tok iter 139660 | train 2.676 | val 2.663 | lr 6.97e-05 | 102,503 tok/s | 54.64 GB | 18.306B tok iter 139680 | train 2.570 | val 2.594 | lr 6.97e-05 | 102,504 tok/s | 54.64 GB | 18.308B tok iter 139700 | train 2.570 | val 2.632 | lr 6.96e-05 | 102,504 tok/s | 54.64 GB | 18.311B tok iter 139720 | train 2.676 | val 2.630 | lr 6.96e-05 | 102,505 tok/s | 54.64 GB | 18.314B tok iter 139740 | train 2.617 | val 2.615 | lr 6.96e-05 | 102,506 tok/s | 54.64 GB | 18.316B tok iter 139760 | train 2.704 | val 2.627 | lr 6.95e-05 | 102,507 tok/s | 54.64 GB | 18.319B tok iter 139780 | train 2.551 | val 2.620 | lr 6.95e-05 | 102,508 tok/s | 54.64 GB | 18.321B tok iter 139800 | train 2.557 | val 2.624 | lr 6.95e-05 | 102,509 tok/s | 54.64 GB | 18.324B tok iter 139820 | train 2.622 | val 2.655 | lr 6.95e-05 | 102,510 tok/s | 54.64 GB | 18.327B tok iter 139840 | train 2.678 | val 2.643 | lr 6.94e-05 | 102,511 tok/s | 54.64 GB | 18.329B tok iter 139860 | train 2.616 | val 2.617 | lr 6.94e-05 | 102,511 tok/s | 54.64 GB | 18.332B tok iter 139880 | train 2.587 | val 2.626 | lr 6.94e-05 | 102,512 tok/s | 54.64 GB | 18.334B tok iter 139900 | train 2.651 | val 2.623 | lr 6.93e-05 | 102,513 tok/s | 54.64 GB | 18.337B tok iter 139920 | train 2.539 | val 2.651 | lr 6.93e-05 | 102,515 tok/s | 54.64 GB | 18.340B tok iter 139940 | train 2.603 | val 2.616 | lr 6.93e-05 | 102,515 tok/s | 54.64 GB | 18.342B tok iter 139960 | train 2.679 | val 2.616 | lr 6.93e-05 | 102,516 tok/s | 54.64 GB | 18.345B tok iter 139980 | train 2.584 | val 2.660 | lr 6.92e-05 | 102,517 tok/s | 54.64 GB | 18.348B tok checkpoint @ iter 139990 -> checkpoints/500m_latest.pt iter 140000 | train 2.782 | val 2.629 | lr 6.92e-05 | 102,511 tok/s | 54.64 GB | 18.350B tok checkpoint @ iter 140000 -> checkpoints/500m_latest.pt iter 140020 | train 2.626 | val 2.597 | lr 6.92e-05 | 102,496 tok/s | 54.64 GB | 18.353B tok iter 140040 | train 2.779 | val 2.656 | lr 6.91e-05 | 102,497 tok/s | 54.64 GB | 18.355B tok iter 140060 | train 2.725 | val 2.637 | lr 6.91e-05 | 102,498 tok/s | 54.64 GB | 18.358B tok iter 140080 | train 2.683 | val 2.654 | lr 6.91e-05 | 102,499 tok/s | 54.64 GB | 18.361B tok iter 140100 | train 2.590 | val 2.625 | lr 6.91e-05 | 102,500 tok/s | 54.64 GB | 18.363B tok iter 140120 | train 2.609 | val 2.619 | lr 6.90e-05 | 102,501 tok/s | 54.64 GB | 18.366B tok iter 140140 | train 2.585 | val 2.615 | lr 6.90e-05 | 102,502 tok/s | 54.64 GB | 18.369B tok iter 140160 | train 2.574 | val 2.586 | lr 6.90e-05 | 102,503 tok/s | 54.64 GB | 18.371B tok iter 140180 | train 2.675 | val 2.662 | lr 6.89e-05 | 102,503 tok/s | 54.64 GB | 18.374B tok iter 140200 | train 2.591 | val 2.631 | lr 6.89e-05 | 102,504 tok/s | 54.64 GB | 18.376B tok iter 140220 | train 2.631 | val 2.649 | lr 6.89e-05 | 102,505 tok/s | 54.64 GB | 18.379B tok iter 140240 | train 2.587 | val 2.637 | lr 6.89e-05 | 102,507 tok/s | 54.64 GB | 18.382B tok iter 140260 | train 2.623 | val 2.633 | lr 6.88e-05 | 102,507 tok/s | 54.64 GB | 18.384B tok iter 140280 | train 2.641 | val 2.622 | lr 6.88e-05 | 102,508 tok/s | 54.64 GB | 18.387B tok iter 140300 | train 2.534 | val 2.631 | lr 6.88e-05 | 102,509 tok/s | 54.64 GB | 18.390B tok iter 140320 | train 2.757 | val 2.684 | lr 6.87e-05 | 102,510 tok/s | 54.64 GB | 18.392B tok iter 140340 | train 2.557 | val 2.641 | lr 6.87e-05 | 102,510 tok/s | 54.64 GB | 18.395B tok iter 140360 | train 2.595 | val 2.632 | lr 6.87e-05 | 102,511 tok/s | 54.64 GB | 18.397B tok iter 140380 | train 2.542 | val 2.634 | lr 6.87e-05 | 102,513 tok/s | 54.64 GB | 18.400B tok iter 140400 | train 2.670 | val 2.577 | lr 6.86e-05 | 102,514 tok/s | 54.64 GB | 18.403B tok iter 140420 | train 2.565 | val 2.608 | lr 6.86e-05 | 102,514 tok/s | 54.64 GB | 18.405B tok iter 140440 | train 2.639 | val 2.645 | lr 6.86e-05 | 102,515 tok/s | 54.64 GB | 18.408B tok iter 140460 | train 2.558 | val 2.622 | lr 6.85e-05 | 102,516 tok/s | 54.64 GB | 18.411B tok iter 140480 | train 2.563 | val 2.647 | lr 6.85e-05 | 102,517 tok/s | 54.64 GB | 18.413B tok checkpoint @ iter 140490 -> checkpoints/500m_latest.pt iter 140500 | train 2.670 | val 2.652 | lr 6.85e-05 | 102,510 tok/s | 54.64 GB | 18.416B tok checkpoint @ iter 140500 -> checkpoints/500m_latest.pt iter 140520 | train 2.632 | val 2.616 | lr 6.85e-05 | 102,496 tok/s | 54.64 GB | 18.418B tok iter 140540 | train 2.638 | val 2.614 | lr 6.84e-05 | 102,497 tok/s | 54.64 GB | 18.421B tok iter 140560 | train 2.650 | val 2.598 | lr 6.84e-05 | 102,498 tok/s | 54.64 GB | 18.424B tok iter 140580 | train 2.606 | val 2.615 | lr 6.84e-05 | 102,498 tok/s | 54.64 GB | 18.426B tok iter 140600 | train 2.655 | val 2.656 | lr 6.83e-05 | 102,500 tok/s | 54.64 GB | 18.429B tok iter 140620 | train 2.676 | val 2.632 | lr 6.83e-05 | 102,501 tok/s | 54.64 GB | 18.431B tok iter 140640 | train 2.703 | val 2.614 | lr 6.83e-05 | 102,502 tok/s | 54.64 GB | 18.434B tok iter 140660 | train 2.619 | val 2.691 | lr 6.83e-05 | 102,502 tok/s | 54.64 GB | 18.437B tok iter 140680 | train 2.663 | val 2.670 | lr 6.82e-05 | 102,503 tok/s | 54.64 GB | 18.439B tok iter 140700 | train 2.607 | val 2.615 | lr 6.82e-05 | 102,504 tok/s | 54.64 GB | 18.442B tok iter 140720 | train 2.583 | val 2.579 | lr 6.82e-05 | 102,505 tok/s | 54.64 GB | 18.445B tok iter 140740 | train 2.676 | val 2.591 | lr 6.82e-05 | 102,506 tok/s | 54.64 GB | 18.447B tok iter 140760 | train 2.664 | val 2.673 | lr 6.81e-05 | 102,507 tok/s | 54.64 GB | 18.450B tok iter 140780 | train 2.634 | val 2.605 | lr 6.81e-05 | 102,508 tok/s | 54.64 GB | 18.452B tok iter 140800 | train 2.709 | val 2.649 | lr 6.81e-05 | 102,509 tok/s | 54.64 GB | 18.455B tok iter 140820 | train 2.662 | val 2.651 | lr 6.80e-05 | 102,509 tok/s | 54.64 GB | 18.458B tok iter 140840 | train 2.718 | val 2.638 | lr 6.80e-05 | 102,510 tok/s | 54.64 GB | 18.460B tok iter 140860 | train 2.758 | val 2.659 | lr 6.80e-05 | 102,511 tok/s | 54.64 GB | 18.463B tok iter 140880 | train 2.547 | val 2.655 | lr 6.80e-05 | 102,513 tok/s | 54.64 GB | 18.466B tok iter 140900 | train 2.631 | val 2.628 | lr 6.79e-05 | 102,513 tok/s | 54.64 GB | 18.468B tok iter 140920 | train 2.634 | val 2.654 | lr 6.79e-05 | 102,514 tok/s | 54.64 GB | 18.471B tok iter 140940 | train 2.686 | val 2.612 | lr 6.79e-05 | 102,515 tok/s | 54.64 GB | 18.473B tok iter 140960 | train 2.522 | val 2.587 | lr 6.79e-05 | 102,516 tok/s | 54.64 GB | 18.476B tok iter 140980 | train 2.586 | val 2.603 | lr 6.78e-05 | 102,516 tok/s | 54.64 GB | 18.479B tok checkpoint @ iter 140990 -> checkpoints/500m_latest.pt iter 141000 | train 2.583 | val 2.617 | lr 6.78e-05 | 102,510 tok/s | 54.64 GB | 18.481B tok checkpoint @ iter 141000 -> checkpoints/500m_latest.pt iter 141020 | train 2.582 | val 2.655 | lr 6.78e-05 | 102,497 tok/s | 54.64 GB | 18.484B tok iter 141040 | train 2.739 | val 2.650 | lr 6.77e-05 | 102,498 tok/s | 54.64 GB | 18.487B tok iter 141060 | train 2.639 | val 2.604 | lr 6.77e-05 | 102,498 tok/s | 54.64 GB | 18.489B tok iter 141080 | train 2.498 | val 2.618 | lr 6.77e-05 | 102,499 tok/s | 54.64 GB | 18.492B tok iter 141100 | train 2.637 | val 2.615 | lr 6.77e-05 | 102,500 tok/s | 54.64 GB | 18.494B tok iter 141120 | train 2.438 | val 2.631 | lr 6.76e-05 | 102,502 tok/s | 54.64 GB | 18.497B tok iter 141140 | train 2.654 | val 2.587 | lr 6.76e-05 | 102,502 tok/s | 54.64 GB | 18.500B tok iter 141160 | train 2.665 | val 2.605 | lr 6.76e-05 | 102,503 tok/s | 54.64 GB | 18.502B tok iter 141180 | train 2.636 | val 2.596 | lr 6.76e-05 | 102,504 tok/s | 54.64 GB | 18.505B tok iter 141200 | train 2.688 | val 2.609 | lr 6.75e-05 | 102,505 tok/s | 54.64 GB | 18.507B tok iter 141220 | train 2.635 | val 2.605 | lr 6.75e-05 | 102,505 tok/s | 54.64 GB | 18.510B tok iter 141240 | train 2.696 | val 2.647 | lr 6.75e-05 | 102,506 tok/s | 54.64 GB | 18.513B tok iter 141260 | train 2.550 | val 2.636 | lr 6.75e-05 | 102,508 tok/s | 54.64 GB | 18.515B tok iter 141280 | train 2.621 | val 2.644 | lr 6.74e-05 | 102,509 tok/s | 54.64 GB | 18.518B tok iter 141300 | train 2.629 | val 2.681 | lr 6.74e-05 | 102,509 tok/s | 54.64 GB | 18.521B tok iter 141320 | train 2.604 | val 2.543 | lr 6.74e-05 | 102,510 tok/s | 54.64 GB | 18.523B tok iter 141340 | train 2.602 | val 2.598 | lr 6.74e-05 | 102,511 tok/s | 54.64 GB | 18.526B tok iter 141360 | train 2.645 | val 2.643 | lr 6.73e-05 | 102,512 tok/s | 54.64 GB | 18.528B tok iter 141380 | train 2.618 | val 2.612 | lr 6.73e-05 | 102,513 tok/s | 54.64 GB | 18.531B tok iter 141400 | train 2.720 | val 2.654 | lr 6.73e-05 | 102,514 tok/s | 54.64 GB | 18.534B tok iter 141420 | train 2.633 | val 2.638 | lr 6.72e-05 | 102,515 tok/s | 54.64 GB | 18.536B tok iter 141440 | train 2.671 | val 2.594 | lr 6.72e-05 | 102,516 tok/s | 54.64 GB | 18.539B tok iter 141460 | train 2.589 | val 2.673 | lr 6.72e-05 | 102,516 tok/s | 54.64 GB | 18.542B tok iter 141480 | train 2.598 | val 2.648 | lr 6.72e-05 | 102,517 tok/s | 54.64 GB | 18.544B tok checkpoint @ iter 141491 -> checkpoints/500m_latest.pt iter 141500 | train 2.649 | val 2.578 | lr 6.71e-05 | 102,511 tok/s | 54.64 GB | 18.547B tok checkpoint @ iter 141500 -> checkpoints/500m_latest.pt iter 141520 | train 2.466 | val 2.589 | lr 6.71e-05 | 102,497 tok/s | 54.64 GB | 18.549B tok iter 141540 | train 2.717 | val 2.653 | lr 6.71e-05 | 102,497 tok/s | 54.64 GB | 18.552B tok iter 141560 | train 2.614 | val 2.622 | lr 6.71e-05 | 102,499 tok/s | 54.64 GB | 18.555B tok iter 141580 | train 2.688 | val 2.650 | lr 6.70e-05 | 102,500 tok/s | 54.64 GB | 18.557B tok iter 141600 | train 2.715 | val 2.629 | lr 6.70e-05 | 102,501 tok/s | 54.64 GB | 18.560B tok iter 141620 | train 2.772 | val 2.523 | lr 6.70e-05 | 102,501 tok/s | 54.64 GB | 18.563B tok iter 141640 | train 2.568 | val 2.653 | lr 6.70e-05 | 102,502 tok/s | 54.64 GB | 18.565B tok iter 141660 | train 2.559 | val 2.643 | lr 6.69e-05 | 102,503 tok/s | 54.64 GB | 18.568B tok iter 141680 | train 2.655 | val 2.657 | lr 6.69e-05 | 102,504 tok/s | 54.64 GB | 18.570B tok iter 141700 | train 2.549 | val 2.669 | lr 6.69e-05 | 102,505 tok/s | 54.64 GB | 18.573B tok iter 141720 | train 2.577 | val 2.627 | lr 6.69e-05 | 102,506 tok/s | 54.64 GB | 18.576B tok iter 141740 | train 2.629 | val 2.665 | lr 6.68e-05 | 102,507 tok/s | 54.64 GB | 18.578B tok iter 141760 | train 2.572 | val 2.613 | lr 6.68e-05 | 102,508 tok/s | 54.64 GB | 18.581B tok iter 141780 | train 2.677 | val 2.625 | lr 6.68e-05 | 102,508 tok/s | 54.64 GB | 18.584B tok iter 141800 | train 2.620 | val 2.581 | lr 6.68e-05 | 102,509 tok/s | 54.64 GB | 18.586B tok iter 141820 | train 2.523 | val 2.646 | lr 6.67e-05 | 102,510 tok/s | 54.64 GB | 18.589B tok iter 141840 | train 2.566 | val 2.654 | lr 6.67e-05 | 102,512 tok/s | 54.64 GB | 18.591B tok iter 141860 | train 2.641 | val 2.657 | lr 6.67e-05 | 102,512 tok/s | 54.64 GB | 18.594B tok iter 141880 | train 2.636 | val 2.629 | lr 6.67e-05 | 102,513 tok/s | 54.64 GB | 18.597B tok iter 141900 | train 2.559 | val 2.628 | lr 6.66e-05 | 102,514 tok/s | 54.64 GB | 18.599B tok iter 141920 | train 2.675 | val 2.623 | lr 6.66e-05 | 102,515 tok/s | 54.64 GB | 18.602B tok iter 141940 | train 2.549 | val 2.578 | lr 6.66e-05 | 102,515 tok/s | 54.64 GB | 18.604B tok iter 141960 | train 2.584 | val 2.568 | lr 6.66e-05 | 102,516 tok/s | 54.64 GB | 18.607B tok iter 141980 | train 2.677 | val 2.623 | lr 6.65e-05 | 102,517 tok/s | 54.64 GB | 18.610B tok checkpoint @ iter 141990 -> checkpoints/500m_latest.pt iter 142000 | train 2.766 | val 2.602 | lr 6.65e-05 | 102,511 tok/s | 54.64 GB | 18.612B tok checkpoint @ iter 142000 -> checkpoints/500m_latest.pt iter 142020 | train 2.657 | val 2.623 | lr 6.65e-05 | 102,497 tok/s | 54.64 GB | 18.615B tok iter 142040 | train 2.597 | val 2.655 | lr 6.65e-05 | 102,498 tok/s | 54.64 GB | 18.618B tok iter 142060 | train 2.533 | val 2.611 | lr 6.64e-05 | 102,499 tok/s | 54.64 GB | 18.620B tok iter 142080 | train 2.483 | val 2.630 | lr 6.64e-05 | 102,500 tok/s | 54.64 GB | 18.623B tok iter 142100 | train 2.667 | val 2.627 | lr 6.64e-05 | 102,500 tok/s | 54.64 GB | 18.625B tok iter 142120 | train 2.773 | val 2.610 | lr 6.64e-05 | 102,501 tok/s | 54.64 GB | 18.628B tok iter 142140 | train 2.527 | val 2.632 | lr 6.63e-05 | 102,502 tok/s | 54.64 GB | 18.631B tok iter 142160 | train 2.618 | val 2.669 | lr 6.63e-05 | 102,503 tok/s | 54.64 GB | 18.633B tok iter 142180 | train 2.439 | val 2.612 | lr 6.63e-05 | 102,504 tok/s | 54.64 GB | 18.636B tok iter 142200 | train 2.557 | val 2.598 | lr 6.63e-05 | 102,505 tok/s | 54.64 GB | 18.639B tok iter 142220 | train 2.639 | val 2.655 | lr 6.63e-05 | 102,506 tok/s | 54.64 GB | 18.641B tok iter 142240 | train 2.504 | val 2.636 | lr 6.62e-05 | 102,507 tok/s | 54.64 GB | 18.644B tok iter 142260 | train 2.680 | val 2.640 | lr 6.62e-05 | 102,507 tok/s | 54.64 GB | 18.646B tok iter 142280 | train 2.663 | val 2.584 | lr 6.62e-05 | 102,508 tok/s | 54.64 GB | 18.649B tok iter 142300 | train 2.699 | val 2.619 | lr 6.62e-05 | 102,509 tok/s | 54.64 GB | 18.652B tok iter 142320 | train 2.651 | val 2.605 | lr 6.61e-05 | 102,511 tok/s | 54.64 GB | 18.654B tok iter 142340 | train 2.516 | val 2.605 | lr 6.61e-05 | 102,511 tok/s | 54.64 GB | 18.657B tok iter 142360 | train 2.632 | val 2.659 | lr 6.61e-05 | 102,512 tok/s | 54.64 GB | 18.660B tok iter 142380 | train 2.728 | val 2.594 | lr 6.61e-05 | 102,513 tok/s | 54.64 GB | 18.662B tok iter 142400 | train 2.398 | val 2.663 | lr 6.60e-05 | 102,514 tok/s | 54.64 GB | 18.665B tok iter 142420 | train 2.557 | val 2.642 | lr 6.60e-05 | 102,514 tok/s | 54.64 GB | 18.667B tok iter 142440 | train 2.527 | val 2.637 | lr 6.60e-05 | 102,515 tok/s | 54.64 GB | 18.670B tok iter 142460 | train 2.619 | val 2.644 | lr 6.60e-05 | 102,516 tok/s | 54.64 GB | 18.673B tok iter 142480 | train 2.630 | val 2.623 | lr 6.59e-05 | 102,518 tok/s | 54.64 GB | 18.675B tok checkpoint @ iter 142490 -> checkpoints/500m_latest.pt iter 142500 | train 2.565 | val 2.658 | lr 6.59e-05 | 102,510 tok/s | 54.64 GB | 18.678B tok checkpoint @ iter 142500 -> checkpoints/500m_latest.pt iter 142520 | train 2.637 | val 2.603 | lr 6.59e-05 | 102,497 tok/s | 54.64 GB | 18.681B tok iter 142540 | train 2.477 | val 2.603 | lr 6.59e-05 | 102,498 tok/s | 54.64 GB | 18.683B tok iter 142560 | train 2.608 | val 2.607 | lr 6.58e-05 | 102,499 tok/s | 54.64 GB | 18.686B tok iter 142580 | train 2.720 | val 2.637 | lr 6.58e-05 | 102,499 tok/s | 54.64 GB | 18.688B tok iter 142600 | train 2.477 | val 2.672 | lr 6.58e-05 | 102,500 tok/s | 54.64 GB | 18.691B tok iter 142620 | train 2.734 | val 2.635 | lr 6.58e-05 | 102,501 tok/s | 54.64 GB | 18.694B tok iter 142640 | train 2.644 | val 2.647 | lr 6.58e-05 | 102,503 tok/s | 54.64 GB | 18.696B tok iter 142660 | train 2.758 | val 2.615 | lr 6.57e-05 | 102,503 tok/s | 54.64 GB | 18.699B tok iter 142680 | train 2.525 | val 2.596 | lr 6.57e-05 | 102,504 tok/s | 54.64 GB | 18.701B tok iter 142700 | train 2.543 | val 2.629 | lr 6.57e-05 | 102,505 tok/s | 54.64 GB | 18.704B tok iter 142720 | train 2.655 | val 2.611 | lr 6.57e-05 | 102,506 tok/s | 54.64 GB | 18.707B tok iter 142740 | train 2.650 | val 2.622 | lr 6.56e-05 | 102,506 tok/s | 54.64 GB | 18.709B tok iter 142760 | train 2.609 | val 2.604 | lr 6.56e-05 | 102,507 tok/s | 54.64 GB | 18.712B tok iter 142780 | train 2.693 | val 2.522 | lr 6.56e-05 | 102,508 tok/s | 54.64 GB | 18.715B tok iter 142800 | train 2.655 | val 2.613 | lr 6.56e-05 | 102,510 tok/s | 54.64 GB | 18.717B tok iter 142820 | train 2.690 | val 2.614 | lr 6.56e-05 | 102,510 tok/s | 54.64 GB | 18.720B tok iter 142840 | train 2.661 | val 2.637 | lr 6.55e-05 | 102,511 tok/s | 54.64 GB | 18.722B tok iter 142860 | train 2.501 | val 2.633 | lr 6.55e-05 | 102,512 tok/s | 54.64 GB | 18.725B tok iter 142880 | train 2.608 | val 2.600 | lr 6.55e-05 | 102,513 tok/s | 54.64 GB | 18.728B tok iter 142900 | train 2.473 | val 2.635 | lr 6.55e-05 | 102,513 tok/s | 54.64 GB | 18.730B tok iter 142920 | train 2.554 | val 2.615 | lr 6.54e-05 | 102,514 tok/s | 54.64 GB | 18.733B tok iter 142940 | train 2.659 | val 2.637 | lr 6.54e-05 | 102,515 tok/s | 54.64 GB | 18.736B tok iter 142960 | train 2.654 | val 2.648 | lr 6.54e-05 | 102,517 tok/s | 54.64 GB | 18.738B tok iter 142980 | train 2.479 | val 2.687 | lr 6.54e-05 | 102,517 tok/s | 54.64 GB | 18.741B tok checkpoint @ iter 142990 -> checkpoints/500m_latest.pt iter 143000 | train 2.591 | val 2.652 | lr 6.53e-05 | 102,511 tok/s | 54.64 GB | 18.743B tok checkpoint @ iter 143000 -> checkpoints/500m_latest.pt iter 143020 | train 2.645 | val 2.631 | lr 6.53e-05 | 102,498 tok/s | 54.64 GB | 18.746B tok iter 143040 | train 2.608 | val 2.603 | lr 6.53e-05 | 102,499 tok/s | 54.64 GB | 18.749B tok iter 143060 | train 2.543 | val 2.586 | lr 6.53e-05 | 102,499 tok/s | 54.64 GB | 18.751B tok iter 143080 | train 2.489 | val 2.575 | lr 6.53e-05 | 102,500 tok/s | 54.64 GB | 18.754B tok iter 143100 | train 2.689 | val 2.628 | lr 6.52e-05 | 102,501 tok/s | 54.64 GB | 18.757B tok iter 143120 | train 2.580 | val 2.563 | lr 6.52e-05 | 102,502 tok/s | 54.64 GB | 18.759B tok iter 143140 | train 2.642 | val 2.610 | lr 6.52e-05 | 102,502 tok/s | 54.64 GB | 18.762B tok iter 143160 | train 2.677 | val 2.619 | lr 6.52e-05 | 102,504 tok/s | 54.64 GB | 18.764B tok iter 143180 | train 2.641 | val 2.607 | lr 6.52e-05 | 102,505 tok/s | 54.64 GB | 18.767B tok iter 143200 | train 2.592 | val 2.583 | lr 6.51e-05 | 102,506 tok/s | 54.64 GB | 18.770B tok iter 143220 | train 2.681 | val 2.655 | lr 6.51e-05 | 102,506 tok/s | 54.64 GB | 18.772B tok iter 143240 | train 2.510 | val 2.638 | lr 6.51e-05 | 102,507 tok/s | 54.64 GB | 18.775B tok iter 143260 | train 2.644 | val 2.648 | lr 6.51e-05 | 102,508 tok/s | 54.64 GB | 18.778B tok iter 143280 | train 2.591 | val 2.701 | lr 6.50e-05 | 102,509 tok/s | 54.64 GB | 18.780B tok iter 143300 | train 2.621 | val 2.624 | lr 6.50e-05 | 102,509 tok/s | 54.64 GB | 18.783B tok iter 143320 | train 2.420 | val 2.654 | lr 6.50e-05 | 102,511 tok/s | 54.64 GB | 18.785B tok iter 143340 | train 2.596 | val 2.634 | lr 6.50e-05 | 102,512 tok/s | 54.64 GB | 18.788B tok iter 143360 | train 2.642 | val 2.633 | lr 6.50e-05 | 102,513 tok/s | 54.64 GB | 18.791B tok iter 143380 | train 2.671 | val 2.581 | lr 6.49e-05 | 102,513 tok/s | 54.64 GB | 18.793B tok iter 143400 | train 2.399 | val 2.637 | lr 6.49e-05 | 102,514 tok/s | 54.64 GB | 18.796B tok iter 143420 | train 2.700 | val 2.618 | lr 6.49e-05 | 102,515 tok/s | 54.64 GB | 18.798B tok iter 143440 | train 2.664 | val 2.598 | lr 6.49e-05 | 102,516 tok/s | 54.64 GB | 18.801B tok iter 143460 | train 2.592 | val 2.568 | lr 6.48e-05 | 102,516 tok/s | 54.64 GB | 18.804B tok iter 143480 | train 2.662 | val 2.648 | lr 6.48e-05 | 102,518 tok/s | 54.64 GB | 18.806B tok checkpoint @ iter 143490 -> checkpoints/500m_latest.pt iter 143500 | train 2.612 | val 2.613 | lr 6.48e-05 | 102,512 tok/s | 54.64 GB | 18.809B tok checkpoint @ iter 143500 -> checkpoints/500m_latest.pt iter 143520 | train 2.649 | val 2.600 | lr 6.48e-05 | 102,499 tok/s | 54.64 GB | 18.812B tok iter 143540 | train 2.564 | val 2.599 | lr 6.48e-05 | 102,499 tok/s | 54.64 GB | 18.814B tok iter 143560 | train 2.680 | val 2.636 | lr 6.47e-05 | 102,500 tok/s | 54.64 GB | 18.817B tok iter 143580 | train 2.592 | val 2.604 | lr 6.47e-05 | 102,501 tok/s | 54.64 GB | 18.819B tok iter 143600 | train 2.518 | val 2.640 | lr 6.47e-05 | 102,502 tok/s | 54.64 GB | 18.822B tok iter 143620 | train 2.693 | val 2.653 | lr 6.47e-05 | 102,502 tok/s | 54.64 GB | 18.825B tok iter 143640 | train 2.598 | val 2.637 | lr 6.47e-05 | 102,503 tok/s | 54.64 GB | 18.827B tok iter 143660 | train 2.647 | val 2.607 | lr 6.46e-05 | 102,505 tok/s | 54.64 GB | 18.830B tok iter 143680 | train 2.647 | val 2.641 | lr 6.46e-05 | 102,506 tok/s | 54.64 GB | 18.833B tok iter 143700 | train 2.782 | val 2.630 | lr 6.46e-05 | 102,506 tok/s | 54.64 GB | 18.835B tok iter 143720 | train 2.520 | val 2.549 | lr 6.46e-05 | 102,507 tok/s | 54.64 GB | 18.838B tok iter 143740 | train 2.628 | val 2.643 | lr 6.46e-05 | 102,508 tok/s | 54.64 GB | 18.840B tok iter 143760 | train 2.652 | val 2.627 | lr 6.45e-05 | 102,509 tok/s | 54.64 GB | 18.843B tok iter 143780 | train 2.604 | val 2.598 | lr 6.45e-05 | 102,509 tok/s | 54.64 GB | 18.846B tok iter 143800 | train 2.732 | val 2.621 | lr 6.45e-05 | 102,510 tok/s | 54.64 GB | 18.848B tok iter 143820 | train 2.525 | val 2.632 | lr 6.45e-05 | 102,512 tok/s | 54.64 GB | 18.851B tok iter 143840 | train 2.552 | val 2.571 | lr 6.45e-05 | 102,513 tok/s | 54.64 GB | 18.854B tok iter 143860 | train 2.549 | val 2.602 | lr 6.44e-05 | 102,513 tok/s | 54.64 GB | 18.856B tok iter 143880 | train 2.635 | val 2.624 | lr 6.44e-05 | 102,514 tok/s | 54.64 GB | 18.859B tok iter 143900 | train 2.570 | val 2.679 | lr 6.44e-05 | 102,515 tok/s | 54.64 GB | 18.861B tok iter 143920 | train 2.613 | val 2.596 | lr 6.44e-05 | 102,516 tok/s | 54.64 GB | 18.864B tok iter 143940 | train 2.553 | val 2.587 | lr 6.44e-05 | 102,516 tok/s | 54.64 GB | 18.867B tok iter 143960 | train 2.739 | val 2.634 | lr 6.43e-05 | 102,517 tok/s | 54.64 GB | 18.869B tok iter 143980 | train 2.530 | val 2.621 | lr 6.43e-05 | 102,519 tok/s | 54.64 GB | 18.872B tok checkpoint @ iter 143990 -> checkpoints/500m_latest.pt iter 144000 | train 2.585 | val 2.587 | lr 6.43e-05 | 102,512 tok/s | 54.64 GB | 18.874B tok checkpoint @ iter 144000 -> checkpoints/500m_latest.pt iter 144020 | train 2.635 | val 2.595 | lr 6.43e-05 | 102,498 tok/s | 54.64 GB | 18.877B tok iter 144040 | train 2.597 | val 2.597 | lr 6.43e-05 | 102,499 tok/s | 54.64 GB | 18.880B tok iter 144060 | train 2.568 | val 2.656 | lr 6.42e-05 | 102,500 tok/s | 54.64 GB | 18.882B tok iter 144080 | train 2.677 | val 2.628 | lr 6.42e-05 | 102,501 tok/s | 54.64 GB | 18.885B tok iter 144100 | train 2.653 | val 2.634 | lr 6.42e-05 | 102,502 tok/s | 54.64 GB | 18.888B tok iter 144120 | train 2.704 | val 2.635 | lr 6.42e-05 | 102,503 tok/s | 54.64 GB | 18.890B tok iter 144140 | train 2.711 | val 2.567 | lr 6.42e-05 | 102,504 tok/s | 54.64 GB | 18.893B tok iter 144160 | train 2.657 | val 2.630 | lr 6.41e-05 | 102,505 tok/s | 54.64 GB | 18.895B tok iter 144180 | train 2.543 | val 2.636 | lr 6.41e-05 | 102,505 tok/s | 54.64 GB | 18.898B tok iter 144200 | train 2.601 | val 2.616 | lr 6.41e-05 | 102,506 tok/s | 54.64 GB | 18.901B tok iter 144220 | train 2.590 | val 2.574 | lr 6.41e-05 | 102,507 tok/s | 54.64 GB | 18.903B tok iter 144240 | train 2.622 | val 2.638 | lr 6.41e-05 | 102,508 tok/s | 54.64 GB | 18.906B tok iter 144260 | train 2.562 | val 2.592 | lr 6.40e-05 | 102,509 tok/s | 54.64 GB | 18.909B tok iter 144280 | train 2.574 | val 2.593 | lr 6.40e-05 | 102,510 tok/s | 54.64 GB | 18.911B tok iter 144300 | train 2.318 | val 2.577 | lr 6.40e-05 | 102,511 tok/s | 54.64 GB | 18.914B tok iter 144320 | train 2.624 | val 2.615 | lr 6.40e-05 | 102,512 tok/s | 54.64 GB | 18.916B tok iter 144340 | train 2.567 | val 2.609 | lr 6.40e-05 | 102,512 tok/s | 54.64 GB | 18.919B tok iter 144360 | train 2.763 | val 2.624 | lr 6.39e-05 | 102,513 tok/s | 54.64 GB | 18.922B tok iter 144380 | train 2.501 | val 2.692 | lr 6.39e-05 | 102,514 tok/s | 54.64 GB | 18.924B tok iter 144400 | train 2.635 | val 2.615 | lr 6.39e-05 | 102,515 tok/s | 54.64 GB | 18.927B tok iter 144420 | train 2.717 | val 2.613 | lr 6.39e-05 | 102,516 tok/s | 54.64 GB | 18.930B tok iter 144440 | train 2.580 | val 2.660 | lr 6.39e-05 | 102,517 tok/s | 54.64 GB | 18.932B tok iter 144460 | train 2.606 | val 2.614 | lr 6.38e-05 | 102,518 tok/s | 54.64 GB | 18.935B tok iter 144480 | train 2.608 | val 2.617 | lr 6.38e-05 | 102,519 tok/s | 54.64 GB | 18.937B tok checkpoint @ iter 144490 -> checkpoints/500m_latest.pt iter 144500 | train 2.684 | val 2.636 | lr 6.38e-05 | 102,511 tok/s | 54.64 GB | 18.940B tok checkpoint @ iter 144500 -> checkpoints/500m_latest.pt iter 144520 | train 2.590 | val 2.632 | lr 6.38e-05 | 102,498 tok/s | 54.64 GB | 18.943B tok iter 144540 | train 2.610 | val 2.689 | lr 6.38e-05 | 102,499 tok/s | 54.64 GB | 18.945B tok iter 144560 | train 2.616 | val 2.651 | lr 6.38e-05 | 102,500 tok/s | 54.64 GB | 18.948B tok iter 144580 | train 2.637 | val 2.635 | lr 6.37e-05 | 102,501 tok/s | 54.64 GB | 18.951B tok iter 144600 | train 2.490 | val 2.585 | lr 6.37e-05 | 102,502 tok/s | 54.64 GB | 18.953B tok iter 144620 | train 2.613 | val 2.671 | lr 6.37e-05 | 102,503 tok/s | 54.64 GB | 18.956B tok iter 144640 | train 2.783 | val 2.621 | lr 6.37e-05 | 102,504 tok/s | 54.64 GB | 18.958B tok iter 144660 | train 2.442 | val 2.605 | lr 6.37e-05 | 102,504 tok/s | 54.64 GB | 18.961B tok iter 144680 | train 2.564 | val 2.618 | lr 6.36e-05 | 102,505 tok/s | 54.64 GB | 18.964B tok iter 144700 | train 2.492 | val 2.619 | lr 6.36e-05 | 102,506 tok/s | 54.64 GB | 18.966B tok iter 144720 | train 2.602 | val 2.585 | lr 6.36e-05 | 102,507 tok/s | 54.64 GB | 18.969B tok iter 144740 | train 2.576 | val 2.624 | lr 6.36e-05 | 102,508 tok/s | 54.64 GB | 18.971B tok iter 144760 | train 2.710 | val 2.629 | lr 6.36e-05 | 102,509 tok/s | 54.64 GB | 18.974B tok iter 144780 | train 2.634 | val 2.638 | lr 6.36e-05 | 102,510 tok/s | 54.64 GB | 18.977B tok iter 144800 | train 2.648 | val 2.610 | lr 6.35e-05 | 102,511 tok/s | 54.64 GB | 18.979B tok iter 144820 | train 2.650 | val 2.635 | lr 6.35e-05 | 102,511 tok/s | 54.64 GB | 18.982B tok iter 144840 | train 2.660 | val 2.618 | lr 6.35e-05 | 102,512 tok/s | 54.64 GB | 18.985B tok iter 144860 | train 2.579 | val 2.596 | lr 6.35e-05 | 102,513 tok/s | 54.64 GB | 18.987B tok iter 144880 | train 2.665 | val 2.652 | lr 6.35e-05 | 102,514 tok/s | 54.64 GB | 18.990B tok iter 144900 | train 2.700 | val 2.642 | lr 6.34e-05 | 102,515 tok/s | 54.64 GB | 18.992B tok iter 144920 | train 2.445 | val 2.655 | lr 6.34e-05 | 102,516 tok/s | 54.64 GB | 18.995B tok iter 144940 | train 2.591 | val 2.644 | lr 6.34e-05 | 102,517 tok/s | 54.64 GB | 18.998B tok iter 144960 | train 2.847 | val 2.631 | lr 6.34e-05 | 102,518 tok/s | 54.64 GB | 19.000B tok iter 144980 | train 2.569 | val 2.619 | lr 6.34e-05 | 102,518 tok/s | 54.64 GB | 19.003B tok checkpoint @ iter 144990 -> checkpoints/500m_latest.pt iter 145000 | train 2.701 | val 2.544 | lr 6.34e-05 | 102,512 tok/s | 54.64 GB | 19.006B tok checkpoint @ iter 145000 -> checkpoints/500m_latest.pt iter 145020 | train 2.646 | val 2.649 | lr 6.33e-05 | 102,499 tok/s | 54.64 GB | 19.008B tok iter 145040 | train 2.630 | val 2.577 | lr 6.33e-05 | 102,500 tok/s | 54.64 GB | 19.011B tok iter 145060 | train 2.571 | val 2.591 | lr 6.33e-05 | 102,500 tok/s | 54.64 GB | 19.013B tok iter 145080 | train 2.609 | val 2.657 | lr 6.33e-05 | 102,501 tok/s | 54.64 GB | 19.016B tok iter 145100 | train 2.589 | val 2.623 | lr 6.33e-05 | 102,502 tok/s | 54.64 GB | 19.019B tok iter 145120 | train 2.454 | val 2.589 | lr 6.32e-05 | 102,504 tok/s | 54.64 GB | 19.021B tok iter 145140 | train 2.648 | val 2.624 | lr 6.32e-05 | 102,504 tok/s | 54.64 GB | 19.024B tok iter 145160 | train 2.623 | val 2.647 | lr 6.32e-05 | 102,505 tok/s | 54.64 GB | 19.027B tok iter 145180 | train 2.643 | val 2.614 | lr 6.32e-05 | 102,506 tok/s | 54.64 GB | 19.029B tok iter 145200 | train 2.683 | val 2.593 | lr 6.32e-05 | 102,507 tok/s | 54.64 GB | 19.032B tok iter 145220 | train 2.631 | val 2.644 | lr 6.32e-05 | 102,507 tok/s | 54.64 GB | 19.034B tok iter 145240 | train 2.484 | val 2.623 | lr 6.31e-05 | 102,508 tok/s | 54.64 GB | 19.037B tok iter 145260 | train 2.685 | val 2.583 | lr 6.31e-05 | 102,509 tok/s | 54.64 GB | 19.040B tok iter 145280 | train 2.565 | val 2.653 | lr 6.31e-05 | 102,510 tok/s | 54.64 GB | 19.042B tok iter 145300 | train 2.588 | val 2.625 | lr 6.31e-05 | 102,511 tok/s | 54.64 GB | 19.045B tok iter 145320 | train 2.596 | val 2.595 | lr 6.31e-05 | 102,512 tok/s | 54.64 GB | 19.048B tok iter 145340 | train 2.576 | val 2.646 | lr 6.31e-05 | 102,513 tok/s | 54.64 GB | 19.050B tok iter 145360 | train 2.549 | val 2.628 | lr 6.30e-05 | 102,514 tok/s | 54.64 GB | 19.053B tok iter 145380 | train 2.558 | val 2.623 | lr 6.30e-05 | 102,514 tok/s | 54.64 GB | 19.055B tok iter 145400 | train 2.500 | val 2.601 | lr 6.30e-05 | 102,515 tok/s | 54.64 GB | 19.058B tok iter 145420 | train 2.587 | val 2.605 | lr 6.30e-05 | 102,516 tok/s | 54.64 GB | 19.061B tok iter 145440 | train 2.544 | val 2.641 | lr 6.30e-05 | 102,517 tok/s | 54.64 GB | 19.063B tok iter 145460 | train 2.460 | val 2.607 | lr 6.30e-05 | 102,517 tok/s | 54.64 GB | 19.066B tok iter 145480 | train 2.440 | val 2.578 | lr 6.29e-05 | 102,518 tok/s | 54.64 GB | 19.068B tok checkpoint @ iter 145490 -> checkpoints/500m_latest.pt iter 145500 | train 2.740 | val 2.582 | lr 6.29e-05 | 102,512 tok/s | 54.64 GB | 19.071B tok checkpoint @ iter 145500 -> checkpoints/500m_latest.pt iter 145520 | train 2.658 | val 2.625 | lr 6.29e-05 | 102,499 tok/s | 54.64 GB | 19.074B tok iter 145540 | train 2.583 | val 2.610 | lr 6.29e-05 | 102,499 tok/s | 54.64 GB | 19.076B tok iter 145560 | train 2.646 | val 2.626 | lr 6.29e-05 | 102,500 tok/s | 54.64 GB | 19.079B tok iter 145580 | train 2.668 | val 2.641 | lr 6.29e-05 | 102,501 tok/s | 54.64 GB | 19.082B tok iter 145600 | train 2.622 | val 2.574 | lr 6.28e-05 | 102,502 tok/s | 54.64 GB | 19.084B tok iter 145620 | train 2.558 | val 2.658 | lr 6.28e-05 | 102,502 tok/s | 54.64 GB | 19.087B tok iter 145640 | train 2.654 | val 2.630 | lr 6.28e-05 | 102,503 tok/s | 54.64 GB | 19.089B tok iter 145660 | train 2.653 | val 2.621 | lr 6.28e-05 | 102,505 tok/s | 54.64 GB | 19.092B tok iter 145680 | train 2.488 | val 2.638 | lr 6.28e-05 | 102,506 tok/s | 54.64 GB | 19.095B tok iter 145700 | train 2.766 | val 2.655 | lr 6.28e-05 | 102,506 tok/s | 54.64 GB | 19.097B tok iter 145720 | train 2.542 | val 2.630 | lr 6.27e-05 | 102,507 tok/s | 54.64 GB | 19.100B tok iter 145740 | train 2.397 | val 2.607 | lr 6.27e-05 | 102,508 tok/s | 54.64 GB | 19.103B tok iter 145760 | train 2.591 | val 2.578 | lr 6.27e-05 | 102,509 tok/s | 54.64 GB | 19.105B tok iter 145780 | train 2.665 | val 2.629 | lr 6.27e-05 | 102,509 tok/s | 54.64 GB | 19.108B tok iter 145800 | train 2.643 | val 2.677 | lr 6.27e-05 | 102,510 tok/s | 54.64 GB | 19.110B tok iter 145820 | train 2.581 | val 2.631 | lr 6.27e-05 | 102,511 tok/s | 54.64 GB | 19.113B tok iter 145840 | train 2.622 | val 2.635 | lr 6.27e-05 | 102,513 tok/s | 54.64 GB | 19.116B tok iter 145860 | train 2.691 | val 2.597 | lr 6.26e-05 | 102,513 tok/s | 54.64 GB | 19.118B tok iter 145880 | train 2.680 | val 2.636 | lr 6.26e-05 | 102,514 tok/s | 54.64 GB | 19.121B tok iter 145900 | train 2.555 | val 2.586 | lr 6.26e-05 | 102,515 tok/s | 54.64 GB | 19.124B tok iter 145920 | train 2.658 | val 2.656 | lr 6.26e-05 | 102,516 tok/s | 54.64 GB | 19.126B tok iter 145940 | train 2.521 | val 2.616 | lr 6.26e-05 | 102,516 tok/s | 54.64 GB | 19.129B tok iter 145960 | train 2.423 | val 2.627 | lr 6.26e-05 | 102,517 tok/s | 54.64 GB | 19.131B tok iter 145980 | train 2.738 | val 2.589 | lr 6.25e-05 | 102,518 tok/s | 54.64 GB | 19.134B tok checkpoint @ iter 145990 -> checkpoints/500m_latest.pt iter 146000 | train 2.603 | val 2.609 | lr 6.25e-05 | 102,512 tok/s | 54.64 GB | 19.137B tok checkpoint @ iter 146000 -> checkpoints/500m_latest.pt iter 146020 | train 2.769 | val 2.634 | lr 6.25e-05 | 102,498 tok/s | 54.64 GB | 19.139B tok iter 146040 | train 2.694 | val 2.632 | lr 6.25e-05 | 102,499 tok/s | 54.64 GB | 19.142B tok iter 146060 | train 2.588 | val 2.607 | lr 6.25e-05 | 102,500 tok/s | 54.64 GB | 19.145B tok iter 146080 | train 2.547 | val 2.609 | lr 6.25e-05 | 102,501 tok/s | 54.64 GB | 19.147B tok iter 146100 | train 2.737 | val 2.641 | lr 6.25e-05 | 102,502 tok/s | 54.64 GB | 19.150B tok iter 146120 | train 2.496 | val 2.599 | lr 6.24e-05 | 102,503 tok/s | 54.64 GB | 19.152B tok iter 146140 | train 2.497 | val 2.605 | lr 6.24e-05 | 102,503 tok/s | 54.64 GB | 19.155B tok iter 146160 | train 2.624 | val 2.627 | lr 6.24e-05 | 102,505 tok/s | 54.64 GB | 19.158B tok iter 146180 | train 2.607 | val 2.640 | lr 6.24e-05 | 102,505 tok/s | 54.64 GB | 19.160B tok iter 146200 | train 2.649 | val 2.625 | lr 6.24e-05 | 102,506 tok/s | 54.64 GB | 19.163B tok iter 146220 | train 2.684 | val 2.627 | lr 6.24e-05 | 102,507 tok/s | 54.64 GB | 19.165B tok iter 146240 | train 2.726 | val 2.639 | lr 6.23e-05 | 102,508 tok/s | 54.64 GB | 19.168B tok iter 146260 | train 2.621 | val 2.626 | lr 6.23e-05 | 102,508 tok/s | 54.64 GB | 19.171B tok iter 146280 | train 2.713 | val 2.602 | lr 6.23e-05 | 102,509 tok/s | 54.64 GB | 19.173B tok iter 146300 | train 2.547 | val 2.625 | lr 6.23e-05 | 102,511 tok/s | 54.64 GB | 19.176B tok iter 146320 | train 2.526 | val 2.605 | lr 6.23e-05 | 102,512 tok/s | 54.64 GB | 19.179B tok iter 146340 | train 2.523 | val 2.622 | lr 6.23e-05 | 102,512 tok/s | 54.64 GB | 19.181B tok iter 146360 | train 2.638 | val 2.634 | lr 6.23e-05 | 102,513 tok/s | 54.64 GB | 19.184B tok iter 146380 | train 2.583 | val 2.624 | lr 6.22e-05 | 102,514 tok/s | 54.64 GB | 19.186B tok iter 146400 | train 2.744 | val 2.595 | lr 6.22e-05 | 102,515 tok/s | 54.64 GB | 19.189B tok iter 146420 | train 2.680 | val 2.660 | lr 6.22e-05 | 102,515 tok/s | 54.64 GB | 19.192B tok iter 146440 | train 2.673 | val 2.596 | lr 6.22e-05 | 102,517 tok/s | 54.64 GB | 19.194B tok iter 146460 | train 2.543 | val 2.609 | lr 6.22e-05 | 102,518 tok/s | 54.64 GB | 19.197B tok iter 146480 | train 2.492 | val 2.652 | lr 6.22e-05 | 102,519 tok/s | 54.64 GB | 19.200B tok checkpoint @ iter 146491 -> checkpoints/500m_latest.pt iter 146500 | train 2.622 | val 2.619 | lr 6.22e-05 | 102,512 tok/s | 54.64 GB | 19.202B tok checkpoint @ iter 146500 -> checkpoints/500m_latest.pt iter 146520 | train 2.653 | val 2.626 | lr 6.21e-05 | 102,499 tok/s | 54.64 GB | 19.205B tok iter 146540 | train 2.664 | val 2.627 | lr 6.21e-05 | 102,500 tok/s | 54.64 GB | 19.207B tok iter 146560 | train 2.590 | val 2.641 | lr 6.21e-05 | 102,501 tok/s | 54.64 GB | 19.210B tok iter 146580 | train 2.529 | val 2.589 | lr 6.21e-05 | 102,502 tok/s | 54.64 GB | 19.213B tok iter 146600 | train 2.561 | val 2.627 | lr 6.21e-05 | 102,503 tok/s | 54.64 GB | 19.215B tok iter 146620 | train 2.703 | val 2.689 | lr 6.21e-05 | 102,504 tok/s | 54.64 GB | 19.218B tok iter 146640 | train 2.621 | val 2.609 | lr 6.21e-05 | 102,505 tok/s | 54.64 GB | 19.221B tok iter 146660 | train 2.567 | val 2.583 | lr 6.20e-05 | 102,505 tok/s | 54.64 GB | 19.223B tok iter 146680 | train 2.607 | val 2.653 | lr 6.20e-05 | 102,506 tok/s | 54.64 GB | 19.226B tok iter 146700 | train 2.698 | val 2.605 | lr 6.20e-05 | 102,507 tok/s | 54.64 GB | 19.228B tok iter 146720 | train 2.546 | val 2.677 | lr 6.20e-05 | 102,508 tok/s | 54.64 GB | 19.231B tok iter 146740 | train 2.575 | val 2.617 | lr 6.20e-05 | 102,509 tok/s | 54.64 GB | 19.234B tok iter 146760 | train 2.521 | val 2.642 | lr 6.20e-05 | 102,510 tok/s | 54.64 GB | 19.236B tok iter 146780 | train 2.685 | val 2.655 | lr 6.20e-05 | 102,511 tok/s | 54.64 GB | 19.239B tok iter 146800 | train 2.566 | val 2.608 | lr 6.20e-05 | 102,512 tok/s | 54.64 GB | 19.242B tok iter 146820 | train 2.637 | val 2.611 | lr 6.19e-05 | 102,512 tok/s | 54.64 GB | 19.244B tok iter 146840 | train 2.506 | val 2.611 | lr 6.19e-05 | 102,513 tok/s | 54.64 GB | 19.247B tok iter 146860 | train 2.684 | val 2.639 | lr 6.19e-05 | 102,514 tok/s | 54.64 GB | 19.249B tok iter 146880 | train 2.652 | val 2.574 | lr 6.19e-05 | 102,515 tok/s | 54.64 GB | 19.252B tok iter 146900 | train 2.613 | val 2.598 | lr 6.19e-05 | 102,516 tok/s | 54.64 GB | 19.255B tok iter 146920 | train 2.612 | val 2.619 | lr 6.19e-05 | 102,517 tok/s | 54.64 GB | 19.257B tok iter 146940 | train 2.687 | val 2.637 | lr 6.19e-05 | 102,518 tok/s | 54.64 GB | 19.260B tok iter 146960 | train 2.650 | val 2.591 | lr 6.18e-05 | 102,519 tok/s | 54.64 GB | 19.262B tok iter 146980 | train 2.640 | val 2.634 | lr 6.18e-05 | 102,519 tok/s | 54.64 GB | 19.265B tok checkpoint @ iter 146992 -> checkpoints/500m_latest.pt iter 147000 | train 2.484 | val 2.597 | lr 6.18e-05 | 102,513 tok/s | 54.64 GB | 19.268B tok checkpoint @ iter 147000 -> checkpoints/500m_latest.pt iter 147020 | train 2.686 | val 2.655 | lr 6.18e-05 | 102,500 tok/s | 54.64 GB | 19.270B tok iter 147040 | train 2.503 | val 2.614 | lr 6.18e-05 | 102,501 tok/s | 54.64 GB | 19.273B tok iter 147060 | train 2.697 | val 2.608 | lr 6.18e-05 | 102,501 tok/s | 54.64 GB | 19.276B tok iter 147080 | train 2.591 | val 2.647 | lr 6.18e-05 | 102,502 tok/s | 54.64 GB | 19.278B tok iter 147100 | train 2.575 | val 2.653 | lr 6.18e-05 | 102,503 tok/s | 54.64 GB | 19.281B tok iter 147120 | train 2.476 | val 2.636 | lr 6.17e-05 | 102,504 tok/s | 54.64 GB | 19.283B tok iter 147140 | train 2.584 | val 2.641 | lr 6.17e-05 | 102,505 tok/s | 54.64 GB | 19.286B tok iter 147160 | train 2.708 | val 2.621 | lr 6.17e-05 | 102,506 tok/s | 54.64 GB | 19.289B tok iter 147180 | train 2.626 | val 2.634 | lr 6.17e-05 | 102,507 tok/s | 54.64 GB | 19.291B tok iter 147200 | train 2.500 | val 2.602 | lr 6.17e-05 | 102,508 tok/s | 54.64 GB | 19.294B tok iter 147220 | train 2.617 | val 2.584 | lr 6.17e-05 | 102,508 tok/s | 54.64 GB | 19.297B tok iter 147240 | train 2.621 | val 2.598 | lr 6.17e-05 | 102,509 tok/s | 54.64 GB | 19.299B tok iter 147260 | train 2.589 | val 2.581 | lr 6.17e-05 | 102,510 tok/s | 54.64 GB | 19.302B tok iter 147280 | train 2.651 | val 2.693 | lr 6.16e-05 | 102,511 tok/s | 54.64 GB | 19.304B tok iter 147300 | train 2.435 | val 2.589 | lr 6.16e-05 | 102,511 tok/s | 54.64 GB | 19.307B tok iter 147320 | train 2.463 | val 2.611 | lr 6.16e-05 | 102,513 tok/s | 54.64 GB | 19.310B tok iter 147340 | train 2.645 | val 2.637 | lr 6.16e-05 | 102,514 tok/s | 54.64 GB | 19.312B tok iter 147360 | train 2.426 | val 2.574 | lr 6.16e-05 | 102,515 tok/s | 54.64 GB | 19.315B tok iter 147380 | train 2.558 | val 2.616 | lr 6.16e-05 | 102,515 tok/s | 54.64 GB | 19.318B tok iter 147400 | train 2.675 | val 2.635 | lr 6.16e-05 | 102,516 tok/s | 54.64 GB | 19.320B tok iter 147420 | train 2.642 | val 2.653 | lr 6.16e-05 | 102,517 tok/s | 54.64 GB | 19.323B tok iter 147440 | train 2.688 | val 2.590 | lr 6.15e-05 | 102,518 tok/s | 54.64 GB | 19.325B tok iter 147460 | train 2.626 | val 2.610 | lr 6.15e-05 | 102,518 tok/s | 54.64 GB | 19.328B tok iter 147480 | train 2.673 | val 2.506 | lr 6.15e-05 | 102,519 tok/s | 54.64 GB | 19.331B tok checkpoint @ iter 147490 -> checkpoints/500m_latest.pt iter 147500 | train 2.601 | val 2.641 | lr 6.15e-05 | 102,513 tok/s | 54.64 GB | 19.333B tok checkpoint @ iter 147500 -> checkpoints/500m_latest.pt iter 147520 | train 2.528 | val 2.599 | lr 6.15e-05 | 102,500 tok/s | 54.64 GB | 19.336B tok iter 147540 | train 2.731 | val 2.635 | lr 6.15e-05 | 102,500 tok/s | 54.64 GB | 19.338B tok iter 147560 | train 2.604 | val 2.590 | lr 6.15e-05 | 102,501 tok/s | 54.64 GB | 19.341B tok iter 147580 | train 2.631 | val 2.615 | lr 6.15e-05 | 102,502 tok/s | 54.64 GB | 19.344B tok iter 147600 | train 2.602 | val 2.648 | lr 6.15e-05 | 102,504 tok/s | 54.64 GB | 19.346B tok iter 147620 | train 2.631 | val 2.625 | lr 6.14e-05 | 102,504 tok/s | 54.64 GB | 19.349B tok iter 147640 | train 2.634 | val 2.582 | lr 6.14e-05 | 102,505 tok/s | 54.64 GB | 19.352B tok iter 147660 | train 2.791 | val 2.637 | lr 6.14e-05 | 102,506 tok/s | 54.64 GB | 19.354B tok iter 147680 | train 2.508 | val 2.627 | lr 6.14e-05 | 102,507 tok/s | 54.64 GB | 19.357B tok iter 147700 | train 2.661 | val 2.549 | lr 6.14e-05 | 102,507 tok/s | 54.64 GB | 19.359B tok iter 147720 | train 2.572 | val 2.636 | lr 6.14e-05 | 102,508 tok/s | 54.64 GB | 19.362B tok iter 147740 | train 2.657 | val 2.606 | lr 6.14e-05 | 102,509 tok/s | 54.64 GB | 19.365B tok iter 147760 | train 2.595 | val 2.594 | lr 6.14e-05 | 102,510 tok/s | 54.64 GB | 19.367B tok iter 147780 | train 2.577 | val 2.601 | lr 6.13e-05 | 102,511 tok/s | 54.64 GB | 19.370B tok iter 147800 | train 2.646 | val 2.606 | lr 6.13e-05 | 102,512 tok/s | 54.64 GB | 19.373B tok iter 147820 | train 2.575 | val 2.615 | lr 6.13e-05 | 102,513 tok/s | 54.64 GB | 19.375B tok iter 147840 | train 2.501 | val 2.676 | lr 6.13e-05 | 102,514 tok/s | 54.64 GB | 19.378B tok iter 147860 | train 2.679 | val 2.609 | lr 6.13e-05 | 102,514 tok/s | 54.64 GB | 19.380B tok iter 147880 | train 2.679 | val 2.631 | lr 6.13e-05 | 102,515 tok/s | 54.64 GB | 19.383B tok iter 147900 | train 2.667 | val 2.600 | lr 6.13e-05 | 102,516 tok/s | 54.64 GB | 19.386B tok iter 147920 | train 2.708 | val 2.590 | lr 6.13e-05 | 102,517 tok/s | 54.64 GB | 19.388B tok iter 147940 | train 2.648 | val 2.624 | lr 6.13e-05 | 102,517 tok/s | 54.64 GB | 19.391B tok iter 147960 | train 2.645 | val 2.571 | lr 6.12e-05 | 102,518 tok/s | 54.64 GB | 19.394B tok iter 147980 | train 2.622 | val 2.594 | lr 6.12e-05 | 102,519 tok/s | 54.64 GB | 19.396B tok checkpoint @ iter 147990 -> checkpoints/500m_latest.pt iter 148000 | train 2.641 | val 2.619 | lr 6.12e-05 | 102,513 tok/s | 54.64 GB | 19.399B tok checkpoint @ iter 148000 -> checkpoints/500m_latest.pt iter 148020 | train 2.652 | val 2.617 | lr 6.12e-05 | 102,499 tok/s | 54.64 GB | 19.401B tok iter 148040 | train 2.640 | val 2.638 | lr 6.12e-05 | 102,500 tok/s | 54.64 GB | 19.404B tok iter 148060 | train 2.608 | val 2.636 | lr 6.12e-05 | 102,501 tok/s | 54.64 GB | 19.407B tok iter 148080 | train 2.537 | val 2.656 | lr 6.12e-05 | 102,502 tok/s | 54.64 GB | 19.409B tok iter 148100 | train 2.550 | val 2.627 | lr 6.12e-05 | 102,502 tok/s | 54.64 GB | 19.412B tok iter 148120 | train 2.731 | val 2.661 | lr 6.12e-05 | 102,503 tok/s | 54.64 GB | 19.415B tok iter 148140 | train 2.544 | val 2.637 | lr 6.12e-05 | 102,504 tok/s | 54.64 GB | 19.417B tok iter 148160 | train 2.534 | val 2.618 | lr 6.11e-05 | 102,505 tok/s | 54.64 GB | 19.420B tok iter 148180 | train 2.526 | val 2.628 | lr 6.11e-05 | 102,505 tok/s | 54.64 GB | 19.422B tok iter 148200 | train 2.684 | val 2.615 | lr 6.11e-05 | 102,507 tok/s | 54.64 GB | 19.425B tok iter 148220 | train 2.704 | val 2.606 | lr 6.11e-05 | 102,508 tok/s | 54.64 GB | 19.428B tok iter 148240 | train 2.720 | val 2.653 | lr 6.11e-05 | 102,509 tok/s | 54.64 GB | 19.430B tok iter 148260 | train 2.680 | val 2.604 | lr 6.11e-05 | 102,509 tok/s | 54.64 GB | 19.433B tok iter 148280 | train 2.544 | val 2.691 | lr 6.11e-05 | 102,510 tok/s | 54.64 GB | 19.435B tok iter 148300 | train 2.505 | val 2.656 | lr 6.11e-05 | 102,511 tok/s | 54.64 GB | 19.438B tok iter 148320 | train 2.573 | val 2.617 | lr 6.11e-05 | 102,512 tok/s | 54.64 GB | 19.441B tok iter 148340 | train 2.588 | val 2.673 | lr 6.11e-05 | 102,512 tok/s | 54.64 GB | 19.443B tok iter 148360 | train 2.449 | val 2.603 | lr 6.10e-05 | 102,513 tok/s | 54.64 GB | 19.446B tok iter 148380 | train 2.561 | val 2.557 | lr 6.10e-05 | 102,514 tok/s | 54.64 GB | 19.449B tok iter 148400 | train 2.596 | val 2.626 | lr 6.10e-05 | 102,515 tok/s | 54.64 GB | 19.451B tok iter 148420 | train 2.597 | val 2.612 | lr 6.10e-05 | 102,516 tok/s | 54.64 GB | 19.454B tok iter 148440 | train 2.671 | val 2.641 | lr 6.10e-05 | 102,517 tok/s | 54.64 GB | 19.456B tok iter 148460 | train 2.765 | val 2.570 | lr 6.10e-05 | 102,518 tok/s | 54.64 GB | 19.459B tok iter 148480 | train 2.690 | val 2.629 | lr 6.10e-05 | 102,519 tok/s | 54.64 GB | 19.462B tok checkpoint @ iter 148490 -> checkpoints/500m_latest.pt iter 148500 | train 2.586 | val 2.583 | lr 6.10e-05 | 102,512 tok/s | 54.64 GB | 19.464B tok checkpoint @ iter 148500 -> checkpoints/500m_latest.pt iter 148520 | train 2.598 | val 2.591 | lr 6.10e-05 | 102,498 tok/s | 54.64 GB | 19.467B tok iter 148540 | train 2.565 | val 2.600 | lr 6.10e-05 | 102,499 tok/s | 54.64 GB | 19.470B tok iter 148560 | train 2.770 | val 2.608 | lr 6.09e-05 | 102,500 tok/s | 54.64 GB | 19.472B tok iter 148580 | train 2.490 | val 2.620 | lr 6.09e-05 | 102,500 tok/s | 54.64 GB | 19.475B tok iter 148600 | train 2.568 | val 2.629 | lr 6.09e-05 | 102,501 tok/s | 54.64 GB | 19.477B tok iter 148620 | train 2.751 | val 2.618 | lr 6.09e-05 | 102,503 tok/s | 54.64 GB | 19.480B tok iter 148640 | train 2.531 | val 2.647 | lr 6.09e-05 | 102,504 tok/s | 54.64 GB | 19.483B tok iter 148660 | train 2.602 | val 2.551 | lr 6.09e-05 | 102,504 tok/s | 54.64 GB | 19.485B tok iter 148680 | train 2.605 | val 2.603 | lr 6.09e-05 | 102,505 tok/s | 54.64 GB | 19.488B tok iter 148700 | train 2.673 | val 2.633 | lr 6.09e-05 | 102,506 tok/s | 54.64 GB | 19.491B tok iter 148720 | train 2.557 | val 2.593 | lr 6.09e-05 | 102,507 tok/s | 54.64 GB | 19.493B tok iter 148740 | train 2.641 | val 2.572 | lr 6.09e-05 | 102,507 tok/s | 54.64 GB | 19.496B tok iter 148760 | train 2.550 | val 2.652 | lr 6.09e-05 | 102,508 tok/s | 54.64 GB | 19.498B tok iter 148780 | train 2.749 | val 2.611 | lr 6.08e-05 | 102,509 tok/s | 54.64 GB | 19.501B tok iter 148800 | train 2.651 | val 2.639 | lr 6.08e-05 | 102,510 tok/s | 54.64 GB | 19.504B tok iter 148820 | train 2.510 | val 2.609 | lr 6.08e-05 | 102,511 tok/s | 54.64 GB | 19.506B tok iter 148840 | train 2.586 | val 2.604 | lr 6.08e-05 | 102,512 tok/s | 54.64 GB | 19.509B tok iter 148860 | train 2.705 | val 2.651 | lr 6.08e-05 | 102,513 tok/s | 54.64 GB | 19.512B tok iter 148880 | train 2.622 | val 2.584 | lr 6.08e-05 | 102,514 tok/s | 54.64 GB | 19.514B tok iter 148900 | train 2.570 | val 2.607 | lr 6.08e-05 | 102,514 tok/s | 54.64 GB | 19.517B tok iter 148920 | train 2.513 | val 2.602 | lr 6.08e-05 | 102,515 tok/s | 54.64 GB | 19.519B tok iter 148940 | train 2.559 | val 2.644 | lr 6.08e-05 | 102,516 tok/s | 54.64 GB | 19.522B tok iter 148960 | train 2.666 | val 2.631 | lr 6.08e-05 | 102,517 tok/s | 54.64 GB | 19.525B tok iter 148980 | train 2.684 | val 2.660 | lr 6.08e-05 | 102,518 tok/s | 54.64 GB | 19.527B tok checkpoint @ iter 148990 -> checkpoints/500m_latest.pt iter 149000 | train 2.621 | val 2.650 | lr 6.08e-05 | 102,511 tok/s | 54.64 GB | 19.530B tok checkpoint @ iter 149000 -> checkpoints/500m_latest.pt iter 149020 | train 2.506 | val 2.523 | lr 6.07e-05 | 102,498 tok/s | 54.64 GB | 19.532B tok iter 149040 | train 2.465 | val 2.605 | lr 6.07e-05 | 102,499 tok/s | 54.64 GB | 19.535B tok iter 149060 | train 2.611 | val 2.626 | lr 6.07e-05 | 102,499 tok/s | 54.64 GB | 19.538B tok iter 149080 | train 2.666 | val 2.630 | lr 6.07e-05 | 102,500 tok/s | 54.64 GB | 19.540B tok iter 149100 | train 2.603 | val 2.607 | lr 6.07e-05 | 102,501 tok/s | 54.64 GB | 19.543B tok iter 149120 | train 2.828 | val 2.612 | lr 6.07e-05 | 102,502 tok/s | 54.64 GB | 19.546B tok iter 149140 | train 2.665 | val 2.597 | lr 6.07e-05 | 102,502 tok/s | 54.64 GB | 19.548B tok iter 149160 | train 2.605 | val 2.607 | lr 6.07e-05 | 102,503 tok/s | 54.64 GB | 19.551B tok iter 149180 | train 2.486 | val 2.583 | lr 6.07e-05 | 102,504 tok/s | 54.64 GB | 19.553B tok iter 149200 | train 2.624 | val 2.618 | lr 6.07e-05 | 102,506 tok/s | 54.64 GB | 19.556B tok iter 149220 | train 2.574 | val 2.625 | lr 6.07e-05 | 102,506 tok/s | 54.64 GB | 19.559B tok iter 149240 | train 2.538 | val 2.622 | lr 6.07e-05 | 102,507 tok/s | 54.64 GB | 19.561B tok iter 149260 | train 2.693 | val 2.604 | lr 6.06e-05 | 102,508 tok/s | 54.64 GB | 19.564B tok iter 149280 | train 2.751 | val 2.616 | lr 6.06e-05 | 102,509 tok/s | 54.64 GB | 19.567B tok iter 149300 | train 2.517 | val 2.666 | lr 6.06e-05 | 102,509 tok/s | 54.64 GB | 19.569B tok iter 149320 | train 2.671 | val 2.652 | lr 6.06e-05 | 102,510 tok/s | 54.64 GB | 19.572B tok iter 149340 | train 2.727 | val 2.605 | lr 6.06e-05 | 102,511 tok/s | 54.64 GB | 19.574B tok iter 149360 | train 2.652 | val 2.582 | lr 6.06e-05 | 102,512 tok/s | 54.64 GB | 19.577B tok iter 149380 | train 2.545 | val 2.626 | lr 6.06e-05 | 102,513 tok/s | 54.64 GB | 19.580B tok iter 149400 | train 2.681 | val 2.593 | lr 6.06e-05 | 102,514 tok/s | 54.64 GB | 19.582B tok iter 149420 | train 2.584 | val 2.618 | lr 6.06e-05 | 102,515 tok/s | 54.64 GB | 19.585B tok iter 149440 | train 2.639 | val 2.628 | lr 6.06e-05 | 102,516 tok/s | 54.64 GB | 19.588B tok iter 149460 | train 2.509 | val 2.603 | lr 6.06e-05 | 102,516 tok/s | 54.64 GB | 19.590B tok iter 149480 | train 2.685 | val 2.640 | lr 6.06e-05 | 102,517 tok/s | 54.64 GB | 19.593B tok checkpoint @ iter 149490 -> checkpoints/500m_latest.pt iter 149500 | train 2.635 | val 2.590 | lr 6.06e-05 | 102,511 tok/s | 54.64 GB | 19.595B tok checkpoint @ iter 149500 -> checkpoints/500m_latest.pt iter 149520 | train 2.552 | val 2.613 | lr 6.05e-05 | 102,497 tok/s | 54.64 GB | 19.598B tok iter 149540 | train 2.570 | val 2.658 | lr 6.05e-05 | 102,497 tok/s | 54.64 GB | 19.601B tok iter 149560 | train 2.631 | val 2.606 | lr 6.05e-05 | 102,498 tok/s | 54.64 GB | 19.603B tok iter 149580 | train 2.593 | val 2.621 | lr 6.05e-05 | 102,499 tok/s | 54.64 GB | 19.606B tok iter 149600 | train 2.607 | val 2.633 | lr 6.05e-05 | 102,500 tok/s | 54.64 GB | 19.609B tok iter 149620 | train 2.645 | val 2.613 | lr 6.05e-05 | 102,501 tok/s | 54.64 GB | 19.611B tok iter 149640 | train 2.614 | val 2.586 | lr 6.05e-05 | 102,502 tok/s | 54.64 GB | 19.614B tok iter 149660 | train 2.633 | val 2.653 | lr 6.05e-05 | 102,503 tok/s | 54.64 GB | 19.616B tok iter 149680 | train 2.602 | val 2.646 | lr 6.05e-05 | 102,504 tok/s | 54.64 GB | 19.619B tok iter 149700 | train 2.556 | val 2.637 | lr 6.05e-05 | 102,504 tok/s | 54.64 GB | 19.622B tok iter 149720 | train 2.560 | val 2.632 | lr 6.05e-05 | 102,505 tok/s | 54.64 GB | 19.624B tok iter 149740 | train 2.742 | val 2.667 | lr 6.05e-05 | 102,506 tok/s | 54.64 GB | 19.627B tok iter 149760 | train 2.419 | val 2.604 | lr 6.05e-05 | 102,507 tok/s | 54.64 GB | 19.629B tok iter 149780 | train 2.642 | val 2.626 | lr 6.05e-05 | 102,508 tok/s | 54.64 GB | 19.632B tok iter 149800 | train 2.708 | val 2.630 | lr 6.05e-05 | 102,509 tok/s | 54.64 GB | 19.635B tok iter 149820 | train 2.619 | val 2.622 | lr 6.04e-05 | 102,510 tok/s | 54.64 GB | 19.637B tok iter 149840 | train 2.620 | val 2.665 | lr 6.04e-05 | 102,511 tok/s | 54.64 GB | 19.640B tok iter 149860 | train 2.468 | val 2.586 | lr 6.04e-05 | 102,511 tok/s | 54.64 GB | 19.643B tok iter 149880 | train 2.684 | val 2.622 | lr 6.04e-05 | 102,512 tok/s | 54.64 GB | 19.645B tok iter 149900 | train 2.661 | val 2.603 | lr 6.04e-05 | 102,513 tok/s | 54.64 GB | 19.648B tok iter 149920 | train 2.786 | val 2.578 | lr 6.04e-05 | 102,514 tok/s | 54.64 GB | 19.650B tok iter 149940 | train 2.706 | val 2.637 | lr 6.04e-05 | 102,514 tok/s | 54.64 GB | 19.653B tok iter 149960 | train 2.767 | val 2.594 | lr 6.04e-05 | 102,515 tok/s | 54.64 GB | 19.656B tok iter 149980 | train 2.720 | val 2.606 | lr 6.04e-05 | 102,516 tok/s | 54.64 GB | 19.658B tok checkpoint @ iter 149990 -> checkpoints/500m_latest.pt iter 150000 | train 2.594 | val 2.595 | lr 6.04e-05 | 102,510 tok/s | 54.64 GB | 19.661B tok checkpoint @ iter 150000 -> checkpoints/500m_latest.pt iter 150020 | train 2.675 | val 2.650 | lr 6.04e-05 | 102,496 tok/s | 54.64 GB | 19.664B tok iter 150040 | train 2.646 | val 2.616 | lr 6.04e-05 | 102,497 tok/s | 54.64 GB | 19.666B tok iter 150060 | train 2.592 | val 2.616 | lr 6.04e-05 | 102,498 tok/s | 54.64 GB | 19.669B tok iter 150080 | train 2.581 | val 2.608 | lr 6.04e-05 | 102,499 tok/s | 54.64 GB | 19.671B tok iter 150100 | train 2.537 | val 2.561 | lr 6.04e-05 | 102,499 tok/s | 54.64 GB | 19.674B tok iter 150120 | train 2.515 | val 2.592 | lr 6.04e-05 | 102,500 tok/s | 54.64 GB | 19.677B tok iter 150140 | train 2.520 | val 2.628 | lr 6.03e-05 | 102,501 tok/s | 54.64 GB | 19.679B tok iter 150160 | train 2.674 | val 2.633 | lr 6.03e-05 | 102,502 tok/s | 54.64 GB | 19.682B tok iter 150180 | train 2.616 | val 2.639 | lr 6.03e-05 | 102,503 tok/s | 54.64 GB | 19.685B tok iter 150200 | train 2.619 | val 2.601 | lr 6.03e-05 | 102,504 tok/s | 54.64 GB | 19.687B tok iter 150220 | train 2.584 | val 2.630 | lr 6.03e-05 | 102,505 tok/s | 54.64 GB | 19.690B tok iter 150240 | train 2.789 | val 2.574 | lr 6.03e-05 | 102,506 tok/s | 54.64 GB | 19.692B tok iter 150260 | train 2.541 | val 2.615 | lr 6.03e-05 | 102,506 tok/s | 54.64 GB | 19.695B tok iter 150280 | train 2.500 | val 2.613 | lr 6.03e-05 | 102,507 tok/s | 54.64 GB | 19.698B tok iter 150300 | train 2.717 | val 2.619 | lr 6.03e-05 | 102,508 tok/s | 54.64 GB | 19.700B tok iter 150320 | train 2.614 | val 2.666 | lr 6.03e-05 | 102,509 tok/s | 54.64 GB | 19.703B tok iter 150340 | train 2.664 | val 2.636 | lr 6.03e-05 | 102,510 tok/s | 54.64 GB | 19.705B tok iter 150360 | train 2.745 | val 2.637 | lr 6.03e-05 | 102,510 tok/s | 54.64 GB | 19.708B tok iter 150380 | train 2.635 | val 2.550 | lr 6.03e-05 | 102,512 tok/s | 54.64 GB | 19.711B tok iter 150400 | train 2.599 | val 2.629 | lr 6.03e-05 | 102,513 tok/s | 54.64 GB | 19.713B tok iter 150420 | train 2.436 | val 2.626 | lr 6.03e-05 | 102,513 tok/s | 54.64 GB | 19.716B tok iter 150440 | train 2.608 | val 2.663 | lr 6.03e-05 | 102,514 tok/s | 54.64 GB | 19.719B tok iter 150460 | train 2.561 | val 2.635 | lr 6.03e-05 | 102,515 tok/s | 54.64 GB | 19.721B tok iter 150480 | train 2.635 | val 2.561 | lr 6.03e-05 | 102,516 tok/s | 54.64 GB | 19.724B tok checkpoint @ iter 150490 -> checkpoints/500m_latest.pt iter 150500 | train 2.722 | val 2.601 | lr 6.03e-05 | 102,509 tok/s | 54.64 GB | 19.726B tok checkpoint @ iter 150500 -> checkpoints/500m_latest.pt iter 150520 | train 2.703 | val 2.612 | lr 6.02e-05 | 102,495 tok/s | 54.64 GB | 19.729B tok iter 150540 | train 2.459 | val 2.659 | lr 6.02e-05 | 102,497 tok/s | 54.64 GB | 19.732B tok iter 150560 | train 2.565 | val 2.612 | lr 6.02e-05 | 102,498 tok/s | 54.64 GB | 19.734B tok iter 150580 | train 2.622 | val 2.650 | lr 6.02e-05 | 102,498 tok/s | 54.64 GB | 19.737B tok iter 150600 | train 2.587 | val 2.607 | lr 6.02e-05 | 102,499 tok/s | 54.64 GB | 19.740B tok iter 150620 | train 2.541 | val 2.594 | lr 6.02e-05 | 102,500 tok/s | 54.64 GB | 19.742B tok iter 150640 | train 2.517 | val 2.568 | lr 6.02e-05 | 102,501 tok/s | 54.64 GB | 19.745B tok iter 150660 | train 2.598 | val 2.647 | lr 6.02e-05 | 102,501 tok/s | 54.64 GB | 19.747B tok iter 150680 | train 2.506 | val 2.623 | lr 6.02e-05 | 102,502 tok/s | 54.64 GB | 19.750B tok iter 150700 | train 2.631 | val 2.681 | lr 6.02e-05 | 102,503 tok/s | 54.64 GB | 19.753B tok iter 150720 | train 2.574 | val 2.621 | lr 6.02e-05 | 102,504 tok/s | 54.64 GB | 19.755B tok iter 150740 | train 2.485 | val 2.580 | lr 6.02e-05 | 102,505 tok/s | 54.64 GB | 19.758B tok iter 150760 | train 2.509 | val 2.595 | lr 6.02e-05 | 102,506 tok/s | 54.64 GB | 19.761B tok iter 150780 | train 2.632 | val 2.610 | lr 6.02e-05 | 102,507 tok/s | 54.64 GB | 19.763B tok iter 150800 | train 2.764 | val 2.600 | lr 6.02e-05 | 102,508 tok/s | 54.64 GB | 19.766B tok iter 150820 | train 2.580 | val 2.556 | lr 6.02e-05 | 102,508 tok/s | 54.64 GB | 19.768B tok iter 150840 | train 2.431 | val 2.644 | lr 6.02e-05 | 102,509 tok/s | 54.64 GB | 19.771B tok iter 150860 | train 2.564 | val 2.558 | lr 6.02e-05 | 102,510 tok/s | 54.64 GB | 19.774B tok iter 150880 | train 2.550 | val 2.632 | lr 6.02e-05 | 102,511 tok/s | 54.64 GB | 19.776B tok iter 150900 | train 2.662 | val 2.686 | lr 6.02e-05 | 102,511 tok/s | 54.64 GB | 19.779B tok iter 150920 | train 2.670 | val 2.610 | lr 6.02e-05 | 102,513 tok/s | 54.64 GB | 19.782B tok iter 150940 | train 2.528 | val 2.574 | lr 6.02e-05 | 102,514 tok/s | 54.64 GB | 19.784B tok iter 150960 | train 2.500 | val 2.579 | lr 6.02e-05 | 102,515 tok/s | 54.64 GB | 19.787B tok iter 150980 | train 2.555 | val 2.568 | lr 6.02e-05 | 102,515 tok/s | 54.64 GB | 19.789B tok checkpoint @ iter 150990 -> checkpoints/500m_latest.pt iter 151000 | train 2.654 | val 2.591 | lr 6.01e-05 | 102,509 tok/s | 54.64 GB | 19.792B tok checkpoint @ iter 151000 -> checkpoints/500m_latest.pt iter 151020 | train 2.575 | val 2.665 | lr 6.01e-05 | 102,496 tok/s | 54.64 GB | 19.795B tok iter 151040 | train 2.818 | val 2.602 | lr 6.01e-05 | 102,497 tok/s | 54.64 GB | 19.797B tok iter 151060 | train 2.572 | val 2.612 | lr 6.01e-05 | 102,498 tok/s | 54.64 GB | 19.800B tok iter 151080 | train 2.719 | val 2.630 | lr 6.01e-05 | 102,499 tok/s | 54.64 GB | 19.802B tok iter 151100 | train 2.653 | val 2.612 | lr 6.01e-05 | 102,500 tok/s | 54.64 GB | 19.805B tok iter 151120 | train 2.652 | val 2.562 | lr 6.01e-05 | 102,501 tok/s | 54.64 GB | 19.808B tok iter 151140 | train 2.442 | val 2.642 | lr 6.01e-05 | 102,501 tok/s | 54.64 GB | 19.810B tok iter 151160 | train 2.535 | val 2.606 | lr 6.01e-05 | 102,502 tok/s | 54.64 GB | 19.813B tok iter 151180 | train 2.634 | val 2.619 | lr 6.01e-05 | 102,503 tok/s | 54.64 GB | 19.816B tok iter 151200 | train 2.583 | val 2.621 | lr 6.01e-05 | 102,504 tok/s | 54.64 GB | 19.818B tok iter 151220 | train 2.533 | val 2.648 | lr 6.01e-05 | 102,504 tok/s | 54.64 GB | 19.821B tok iter 151240 | train 2.573 | val 2.615 | lr 6.01e-05 | 102,505 tok/s | 54.64 GB | 19.823B tok iter 151260 | train 2.568 | val 2.540 | lr 6.01e-05 | 102,506 tok/s | 54.64 GB | 19.826B tok iter 151280 | train 2.564 | val 2.608 | lr 6.01e-05 | 102,507 tok/s | 54.64 GB | 19.829B tok iter 151300 | train 2.636 | val 2.599 | lr 6.01e-05 | 102,507 tok/s | 54.64 GB | 19.831B tok iter 151320 | train 2.587 | val 2.620 | lr 6.01e-05 | 102,508 tok/s | 54.64 GB | 19.834B tok iter 151340 | train 2.558 | val 2.625 | lr 6.01e-05 | 102,510 tok/s | 54.64 GB | 19.837B tok iter 151360 | train 2.558 | val 2.583 | lr 6.01e-05 | 102,511 tok/s | 54.64 GB | 19.839B tok iter 151380 | train 2.703 | val 2.568 | lr 6.01e-05 | 102,511 tok/s | 54.64 GB | 19.842B tok iter 151400 | train 2.531 | val 2.633 | lr 6.01e-05 | 102,512 tok/s | 54.64 GB | 19.844B tok iter 151420 | train 2.645 | val 2.571 | lr 6.01e-05 | 102,513 tok/s | 54.64 GB | 19.847B tok iter 151440 | train 2.627 | val 2.607 | lr 6.01e-05 | 102,514 tok/s | 54.64 GB | 19.850B tok iter 151460 | train 2.659 | val 2.656 | lr 6.01e-05 | 102,514 tok/s | 54.64 GB | 19.852B tok iter 151480 | train 2.783 | val 2.589 | lr 6.01e-05 | 102,515 tok/s | 54.64 GB | 19.855B tok checkpoint @ iter 151490 -> checkpoints/500m_latest.pt iter 151500 | train 2.599 | val 2.601 | lr 6.01e-05 | 102,510 tok/s | 54.64 GB | 19.858B tok checkpoint @ iter 151500 -> checkpoints/500m_latest.pt iter 151520 | train 2.618 | val 2.612 | lr 6.01e-05 | 102,497 tok/s | 54.64 GB | 19.860B tok iter 151540 | train 2.651 | val 2.574 | lr 6.01e-05 | 102,497 tok/s | 54.64 GB | 19.863B tok iter 151560 | train 2.512 | val 2.593 | lr 6.01e-05 | 102,498 tok/s | 54.64 GB | 19.865B tok iter 151580 | train 2.382 | val 2.659 | lr 6.01e-05 | 102,499 tok/s | 54.64 GB | 19.868B tok iter 151600 | train 2.443 | val 2.590 | lr 6.01e-05 | 102,500 tok/s | 54.64 GB | 19.871B tok iter 151620 | train 2.520 | val 2.601 | lr 6.01e-05 | 102,500 tok/s | 54.64 GB | 19.873B tok iter 151640 | train 2.666 | val 2.578 | lr 6.01e-05 | 102,501 tok/s | 54.64 GB | 19.876B tok iter 151660 | train 2.730 | val 2.609 | lr 6.01e-05 | 102,502 tok/s | 54.64 GB | 19.879B tok iter 151680 | train 2.576 | val 2.586 | lr 6.00e-05 | 102,503 tok/s | 54.64 GB | 19.881B tok iter 151700 | train 2.517 | val 2.638 | lr 6.00e-05 | 102,504 tok/s | 54.64 GB | 19.884B tok iter 151720 | train 2.588 | val 2.674 | lr 6.00e-05 | 102,505 tok/s | 54.64 GB | 19.886B tok iter 151740 | train 2.481 | val 2.592 | lr 6.00e-05 | 102,506 tok/s | 54.64 GB | 19.889B tok iter 151760 | train 2.424 | val 2.562 | lr 6.00e-05 | 102,507 tok/s | 54.64 GB | 19.892B tok iter 151780 | train 2.544 | val 2.609 | lr 6.00e-05 | 102,507 tok/s | 54.64 GB | 19.894B tok iter 151800 | train 2.600 | val 2.660 | lr 6.00e-05 | 102,508 tok/s | 54.64 GB | 19.897B tok iter 151820 | train 2.457 | val 2.640 | lr 6.00e-05 | 102,509 tok/s | 54.64 GB | 19.899B tok iter 151840 | train 2.640 | val 2.600 | lr 6.00e-05 | 102,510 tok/s | 54.64 GB | 19.902B tok iter 151860 | train 2.619 | val 2.640 | lr 6.00e-05 | 102,510 tok/s | 54.64 GB | 19.905B tok iter 151880 | train 2.625 | val 2.623 | lr 6.00e-05 | 102,511 tok/s | 54.64 GB | 19.907B tok iter 151900 | train 2.642 | val 2.659 | lr 6.00e-05 | 102,512 tok/s | 54.64 GB | 19.910B tok iter 151920 | train 2.544 | val 2.595 | lr 6.00e-05 | 102,513 tok/s | 54.64 GB | 19.913B tok iter 151940 | train 2.596 | val 2.581 | lr 6.00e-05 | 102,513 tok/s | 54.64 GB | 19.915B tok iter 151960 | train 2.630 | val 2.614 | lr 6.00e-05 | 102,514 tok/s | 54.64 GB | 19.918B tok iter 151980 | train 2.449 | val 2.555 | lr 6.00e-05 | 102,515 tok/s | 54.64 GB | 19.920B tok checkpoint @ iter 151990 -> checkpoints/500m_latest.pt iter 152000 | train 2.618 | val 2.685 | lr 6.00e-05 | 102,509 tok/s | 54.64 GB | 19.923B tok checkpoint @ iter 152000 -> checkpoints/500m_latest.pt iter 152020 | train 2.557 | val 2.599 | lr 6.00e-05 | 102,495 tok/s | 54.64 GB | 19.926B tok iter 152040 | train 2.564 | val 2.633 | lr 6.00e-05 | 102,496 tok/s | 54.64 GB | 19.928B tok iter 152060 | train 2.670 | val 2.644 | lr 6.00e-05 | 102,497 tok/s | 54.64 GB | 19.931B tok iter 152080 | train 2.682 | val 2.566 | lr 6.00e-05 | 102,498 tok/s | 54.64 GB | 19.934B tok iter 152100 | train 2.614 | val 2.620 | lr 6.00e-05 | 102,498 tok/s | 54.64 GB | 19.936B tok iter 152120 | train 2.520 | val 2.613 | lr 6.00e-05 | 102,499 tok/s | 54.64 GB | 19.939B tok iter 152140 | train 2.681 | val 2.572 | lr 6.00e-05 | 102,500 tok/s | 54.64 GB | 19.941B tok iter 152160 | train 2.681 | val 2.622 | lr 6.00e-05 | 102,501 tok/s | 54.64 GB | 19.944B tok iter 152180 | train 2.495 | val 2.627 | lr 6.00e-05 | 102,501 tok/s | 54.64 GB | 19.947B tok iter 152200 | train 2.506 | val 2.628 | lr 6.00e-05 | 102,503 tok/s | 54.64 GB | 19.949B tok iter 152220 | train 2.653 | val 2.575 | lr 6.00e-05 | 102,504 tok/s | 54.64 GB | 19.952B tok iter 152240 | train 2.632 | val 2.600 | lr 6.00e-05 | 102,505 tok/s | 54.64 GB | 19.955B tok iter 152260 | train 2.727 | val 2.622 | lr 6.00e-05 | 102,505 tok/s | 54.64 GB | 19.957B tok iter 152280 | train 2.620 | val 2.582 | lr 6.00e-05 | 102,506 tok/s | 54.64 GB | 19.960B tok iter 152300 | train 2.486 | val 2.659 | lr 6.00e-05 | 102,507 tok/s | 54.64 GB | 19.962B tok iter 152320 | train 2.616 | val 2.593 | lr 6.00e-05 | 102,508 tok/s | 54.64 GB | 19.965B tok iter 152340 | train 2.606 | val 2.597 | lr 6.00e-05 | 102,508 tok/s | 54.64 GB | 19.968B tok iter 152360 | train 2.678 | val 2.618 | lr 6.00e-05 | 102,509 tok/s | 54.64 GB | 19.970B tok iter 152380 | train 2.630 | val 2.602 | lr 6.00e-05 | 102,510 tok/s | 54.64 GB | 19.973B tok iter 152400 | train 2.648 | val 2.628 | lr 6.00e-05 | 102,511 tok/s | 54.64 GB | 19.976B tok iter 152420 | train 2.593 | val 2.605 | lr 6.00e-05 | 102,512 tok/s | 54.64 GB | 19.978B tok iter 152440 | train 2.667 | val 2.596 | lr 6.00e-05 | 102,513 tok/s | 54.64 GB | 19.981B tok iter 152460 | train 2.598 | val 2.635 | lr 6.00e-05 | 102,514 tok/s | 54.64 GB | 19.983B tok iter 152480 | train 2.547 | val 2.587 | lr 6.00e-05 | 102,515 tok/s | 54.64 GB | 19.986B tok checkpoint @ iter 152490 -> checkpoints/500m_latest.pt iter 152500 | train 2.589 | val 2.586 | lr 6.00e-05 | 102,508 tok/s | 54.64 GB | 19.989B tok checkpoint @ iter 152500 -> checkpoints/500m_latest.pt iter 152520 | train 2.545 | val 2.613 | lr 6.00e-05 | 102,494 tok/s | 54.64 GB | 19.991B tok iter 152540 | train 2.545 | val 2.578 | lr 6.00e-05 | 102,495 tok/s | 54.64 GB | 19.994B tok iter 152560 | train 2.670 | val 2.583 | lr 6.00e-05 | 102,496 tok/s | 54.64 GB | 19.996B tok iter 152580 | train 2.558 | val 2.564 | lr 6.00e-05 | 102,496 tok/s | 54.64 GB | 19.999B tok training complete [09:22:28] SFT from checkpoints/500m_final.pt rendering 18,151 conversations -> ctx 2048 blocks... rendered 5,000/18,151 rendered 10,000/18,151 rendered 15,000/18,151 packed 2,540 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 2,540 blocks x 2048 = 5,201,920 tokens, 4,568,736 trainable (87.8%) train 2,490 blocks / val 50 blocks 77 steps/epoch x 3.0 epochs = 231 steps (warmup 6) model 500m: 501.1M params torch.compile enabled (first steps will be slow while it compiles) step 0/231 | train 10.646 | val 10.382 | lr 3.33e-06 | ep 0.00 | 50.58 GB step 25/231 | train 7.507 | val 7.468 | lr 1.97e-05 | ep 0.32 | 54.64 GB step 50/231 | train 6.865 | val 6.697 | lr 1.84e-05 | ep 0.65 | 54.64 GB step 75/231 | train 6.265 | val 6.241 | lr 1.61e-05 | ep 0.97 | 54.64 GB step 100/231 | train 6.049 | val 6.004 | lr 1.33e-05 | ep 1.30 | 54.64 GB step 125/231 | train 5.867 | val 5.778 | lr 1.02e-05 | ep 1.62 | 54.64 GB step 150/231 | train 5.654 | val 5.772 | lr 7.17e-06 | ep 1.95 | 54.64 GB step 175/231 | train 5.554 | val 5.706 | lr 4.61e-06 | ep 2.27 | 54.64 GB step 200/231 | train 5.484 | val 5.649 | lr 2.83e-06 | ep 2.60 | 54.64 GB step 225/231 | train 5.696 | val 5.609 | lr 2.03e-06 | ep 2.92 | 54.64 GB SFT complete -> checkpoints/500m_sft_final.pt sample: '\n\n- **1\'s a:\n- **12**.\n- ** **P the big.\n- **c**- ** **A**.\n- ****\n- ****\n- **** (**, let it\'s ain\'.** on the big** and the other** — you, that\'s** — a on the same** — a month.**** is the whole** — no** ain\' a day, they\'re the one, you\'re a t** — it\'s a couple****"** — you, not a good' [09:25:45] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [09:26:16] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 608kB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 608kB / 1.00GB  Processing Files (0 / 1) : 0%| | 608kB / 1.00GB, 59.4kB/s New Data Upload : 1%| | 608kB / 67.1MB, 59.4kB/s  ...hf-500m/model.safetensors: 2%|▏ | 21.9MB / 1.00GB  Processing Files (0 / 1) : 2%|▏ | 21.9MB / 1.00GB, 2.12MB/s New Data Upload : 16%|█▋ | 21.9MB / 134MB, 2.12MB/s  ...hf-500m/model.safetensors: 7%|▋ | 66.8MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 66.8MB / 1.00GB, 6.40MB/s New Data Upload : 50%|████▉ | 66.8MB / 134MB, 6.40MB/s  ...hf-500m/model.safetensors: 7%|▋ | 70.5MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 70.5MB / 1.00GB, 6.60MB/s New Data Upload : 35%|███▌ | 70.5MB / 201MB, 6.60MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 50%|█████ | 134MB / 268MB, 12.5MB/s  ...hf-500m/model.safetensors: 19%|█▉ | 190MB / 1.00GB  Processing Files (0 / 1) : 19%|█▉ | 190MB / 1.00GB, 17.5MB/s New Data Upload : 57%|█████▋ | 190MB / 335MB, 17.5MB/s  ...hf-500m/model.safetensors: 21%|██ | 208MB / 1.00GB  Processing Files (0 / 1) : 21%|██ | 208MB / 1.00GB, 18.8MB/s New Data Upload : 62%|██████▏ | 208MB / 335MB, 18.8MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 30.2MB/s New Data Upload : 83%|████████▎ | 335MB / 402MB, 30.2MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 335MB / 1.00GB  ...hf-500m/model.safetensors: 34%|███▎ | 338MB / 1.00GB  Processing Files (0 / 1) : 34%|███▎ | 338MB / 1.00GB, 29.0MB/s New Data Upload : 72%|███████▏ | 338MB / 469MB, 29.0MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 40.4MB/s New Data Upload : 87%|████████▋ | 469MB / 536MB, 40.4MB/s  ...hf-500m/model.safetensors: 53%|█████▎ | 536MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 536MB / 1.00GB, 45.6MB/s New Data Upload : 89%|████████▉ | 536MB / 604MB, 45.6MB/s  ...hf-500m/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 50.6MB/s New Data Upload : 90%|████████▉ | 603MB / 671MB, 50.6MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 670MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 670MB / 1.00GB, 54.4MB/s New Data Upload : 91%|█████████ | 670MB / 738MB, 54.4MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 670MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 670MB / 1.00GB, 53.2MB/s New Data Upload : 91%|█████████ | 670MB / 738MB, 53.2MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 58.0MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 58.0MB/s  ...hf-500m/model.safetensors: 80%|████████ | 805MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 805MB / 1.00GB, 62.6MB/s New Data Upload : 100%|█████████▉| 805MB / 805MB, 62.6MB/s  ...hf-500m/model.safetensors: 80%|████████ | 805MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 805MB / 1.00GB, 61.2MB/s New Data Upload : 92%|█████████▏| 805MB / 872MB, 61.2MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 872MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 872MB / 1.00GB, 65.8MB/s New Data Upload : 93%|█████████▎| 872MB / 939MB, 65.8MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 939MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 939MB / 1.00GB, 70.2MB/s New Data Upload : 100%|█████████▉| 939MB / 939MB, 70.2MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 939MB / 1.00GB  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.6MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.6MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 69.5MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 69.5MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 68.0MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 68.0MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 65.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 65.2MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 65.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 65.2MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [09:26:28] converting to GGUF q4_k_m (phone / Ollama / LM Studio) Cloning into 'llama.cpp'... Updating files: 7% (243/3304) Updating files: 8% (265/3304) Updating files: 9% (298/3304) Updating files: 10% (331/3304) Updating files: 11% (364/3304) Updating files: 12% (397/3304) Updating files: 12% (410/3304) Updating files: 13% (430/3304) Updating files: 14% (463/3304) Updating files: 15% (496/3304) Updating files: 16% (529/3304) Updating files: 17% (562/3304) Updating files: 18% (595/3304) Updating files: 19% (628/3304) Updating files: 19% (640/3304) Updating files: 20% (661/3304) Updating files: 21% (694/3304) Updating files: 22% (727/3304) Updating files: 23% (760/3304) Updating files: 24% (793/3304) Updating files: 25% (826/3304) Updating files: 26% (860/3304) Updating files: 26% (884/3304) Updating files: 27% (893/3304) Updating files: 28% (926/3304) Updating files: 29% (959/3304) Updating files: 30% (992/3304) Updating files: 31% (1025/3304) Updating files: 32% (1058/3304) Updating files: 33% (1091/3304) Updating files: 34% (1124/3304) Updating files: 34% (1142/3304) Updating files: 35% (1157/3304) Updating files: 36% (1190/3304) Updating files: 37% (1223/3304) Updating files: 38% (1256/3304) Updating files: 39% (1289/3304) Updating files: 40% (1322/3304) Updating files: 41% (1355/3304) Updating files: 41% (1381/3304) Updating files: 42% (1388/3304) Updating files: 43% (1421/3304) Updating files: 44% (1454/3304) Updating files: 45% (1487/3304) Updating files: 46% (1520/3304) Updating files: 47% (1553/3304) Updating files: 48% (1586/3304) Updating files: 49% (1619/3304) Updating files: 49% (1624/3304) Updating files: 50% (1652/3304) Updating files: 51% (1686/3304) Updating files: 52% (1719/3304) Updating files: 53% (1752/3304) Updating files: 54% (1785/3304) Updating files: 55% (1818/3304) Updating files: 56% (1851/3304) Updating files: 56% (1863/3304) Updating files: 57% (1884/3304) Updating files: 57% (1911/3304) Updating files: 58% (1917/3304) Updating files: 59% (1950/3304) Updating files: 60% (1983/3304) Updating files: 61% (2016/3304) Updating files: 62% (2049/3304) Updating files: 63% (2082/3304) Updating files: 63% (2098/3304) Updating files: 64% (2115/3304) Updating files: 65% (2148/3304) Updating files: 66% (2181/3304) Updating files: 67% (2214/3304) Updating files: 68% (2247/3304) Updating files: 69% (2280/3304) Updating files: 70% (2313/3304) Updating files: 70% (2341/3304) Updating files: 71% (2346/3304) Updating files: 72% (2379/3304) Updating files: 73% (2412/3304) Updating files: 74% (2445/3304) Updating files: 75% (2478/3304) Updating files: 76% (2512/3304) Updating files: 77% (2545/3304) Updating files: 77% (2572/3304) Updating files: 78% (2578/3304) Updating files: 79% (2611/3304) Updating files: 80% (2644/3304) Updating files: 81% (2677/3304) Updating files: 82% (2710/3304) Updating files: 83% (2743/3304) Updating files: 84% (2776/3304) Updating files: 85% (2809/3304) Updating files: 85% (2811/3304) Updating files: 86% (2842/3304) Updating files: 87% (2875/3304) Updating files: 88% (2908/3304) Updating files: 89% (2941/3304) Updating files: 90% (2974/3304) Updating files: 91% (3007/3304) Updating files: 92% (3040/3304) Updating files: 92% (3065/3304) Updating files: 93% (3073/3304) Updating files: 94% (3106/3304) Updating files: 95% (3139/3304) Updating files: 96% (3172/3304) Updating files: 97% (3205/3304) Updating files: 98% (3238/3304) Updating files: 99% (3271/3304) Updating files: 99% (3300/3304) Updating files: 100% (3304/3304) Updating files: 100% (3304/3304), done. INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer WARNING:hf-to-gguf: WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized! WARNING:hf-to-gguf:** There are 2 possible reasons for this: WARNING:hf-to-gguf:** - the model has not been added to convert_hf_to_gguf_update.py yet WARNING:hf-to-gguf:** - the pre-tokenization config has changed upstream WARNING:hf-to-gguf:** Check your model files and convert_hf_to_gguf_update.py and update them accordingly. WARNING:hf-to-gguf:** ref: https://github.com/ggml-org/llama.cpp/pull/6920 WARNING:hf-to-gguf:** WARNING:hf-to-gguf:** chkhsh: 50cc433d4d528ad898dc1178e17a479f50c03d2c883d07494ae21da32d3eed4f WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 134, in set_vocab self._set_vocab_sentencepiece() File "/workspace/llama.cpp/conversion/base.py", line 1831, in _set_vocab_sentencepiece tokens, scores, toktypes = self._create_vocab_sentencepiece() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1848, in _create_vocab_sentencepiece raise FileNotFoundError(f"File not found: {tokenizer_path}") FileNotFoundError: File not found: /workspace/hf-500m/tokenizer.model During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 137, in set_vocab self._set_vocab_llama_hf() File "/workspace/llama.cpp/conversion/base.py", line 1933, in _set_vocab_llama_hf vocab = gguf.LlamaHfVocab(self.dir_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/gguf-py/gguf/vocab.py", line 571, in __init__ raise FileNotFoundError('Cannot find Llama BPE tokenizer') FileNotFoundError: Cannot find Llama BPE tokenizer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 296, in main() File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 290, in main model_instance.write() File "/workspace/llama.cpp/conversion/base.py", line 1026, in write self.prepare_metadata(vocab_only=False) File "/workspace/llama.cpp/conversion/base.py", line 1193, in prepare_metadata self.set_vocab() File "/workspace/llama.cpp/conversion/llama.py", line 140, in set_vocab self._set_vocab_gpt2() File "/workspace/llama.cpp/conversion/base.py", line 1714, in _set_vocab_gpt2 tokens, toktypes, tokpre = self.get_vocab_base() ^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1351, in get_vocab_base tokpre = self.get_vocab_base_pre(tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1702, in get_vocab_base_pre raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()") NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre() llama_print_build_info: build = 1 (b2f2216) llama_print_build_info: built with GNU 11.4.0 for Linux x86_64 llama_quantize: quantizing '/workspace/sprocket-500m-f16.gguf' to '/workspace/sprocket-500m-q4_k_m.gguf' as Q4_K_M gguf_init_from_file: failed to open GGUF file '/workspace/sprocket-500m-f16.gguf' (No such file or directory) llama_model_quantize: failed to quantize: llama_model_loader: failed to load model from /workspace/sprocket-500m-f16.gguf llama_quantize: failed to quantize model from '/workspace/sprocket-500m-f16.gguf' ls: cannot access '/workspace/sprocket-*.gguf': No such file or directory [09:30:02] DONE. [09:30:02] base ckpt : /workspace/checkpoints/500m_final.pt [09:30:02] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [09:30:02] hf model : /workspace/hf-500m [09:30:02] gguf : /workspace/sprocket-500m-q4_k_m.gguf [09:30:02] pipeline finished cleanly [09:30:02] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "jasakfns6809ft" removed [09:46:29] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 66) [09:46:29] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M aaac744..9a97dd4 main -> origin/main Updating files: 44% (4/9) Updating files: 55% (5/9) Updating files: 66% (6/9) Updating files: 77% (7/9) Updating files: 88% (8/9) Updating files: 100% (9/9) Updating files: 100% (9/9), done. Updating aaac744..9a97dd4 Fast-forward .gitignore | 4 + cloud/bootstrap.sh | 52 + data/synthetic/sprocket_sft.jsonl | 3220 +++++++++++++++++ eval/persona_battery.py | 198 +- logs/train.log | 7032 +++++++++++++++++++++++++++++++++++++ logs/train_metrics.csv | 6165 ++++++++++++++++++++++++++++++++ scripts/corpus_report.py | 65 +- scripts/fetch_run_log.py | 99 + scripts/harvest_round.py | 101 +- 9 files changed, 16924 insertions(+), 12 deletions(-) create mode 100644 logs/train.log create mode 100644 logs/train_metrics.csv create mode 100644 scripts/fetch_run_log.py [09:46:36] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [09:46:47] corpus file: data/processed/train_sample_100BT.bin [09:46:47] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [09:46:51] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [09:46:52] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [09:47:16] SFT_FRESH=v2: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_094716 [09:47:16] SFT_FRESH=v2: dropped packed cache; base checkpoint kept [09:47:16] SFT from checkpoints/500m_final.pt rendering 21,371 conversations -> ctx 2048 blocks... rendered 5,000/21,371 rendered 10,000/21,371 rendered 15,000/21,371 rendered 20,000/21,371 packed 3,029 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 3,029 blocks x 2048 = 6,203,392 tokens, 5,168,293 trainable (83.3%) train 2,969 blocks / val 60 blocks 92 steps/epoch x 3.0 epochs = 276 steps (warmup 8) model 500m: 501.1M params torch.compile enabled (first steps will be slow while it compiles) step 0/276 | train 10.654 | val 10.443 | lr 2.50e-06 | ep 0.00 | 50.59 GB step 25/276 | train 7.607 | val 7.595 | lr 1.98e-05 | ep 0.27 | 54.64 GB step 50/276 | train 6.834 | val 6.768 | lr 1.89e-05 | ep 0.54 | 54.64 GB step 75/276 | train 6.352 | val 6.375 | lr 1.74e-05 | ep 0.82 | 54.64 GB step 100/276 | train 6.110 | val 6.064 | lr 1.53e-05 | ep 1.09 | 54.64 GB step 125/276 | train 5.920 | val 5.930 | lr 1.28e-05 | ep 1.36 | 54.64 GB step 150/276 | train 5.837 | val 5.837 | lr 1.02e-05 | ep 1.63 | 54.64 GB step 175/276 | train 5.749 | val 5.712 | lr 7.60e-06 | ep 1.90 | 54.64 GB step 200/276 | train 5.588 | val 5.690 | lr 5.34e-06 | ep 2.17 | 54.64 GB step 225/276 | train 5.522 | val 5.661 | lr 3.56e-06 | ep 2.45 | 54.64 GB step 250/276 | train 5.498 | val 5.584 | lr 2.41e-06 | ep 2.72 | 54.64 GB step 275/276 | train 5.414 | val 5.619 | lr 2.00e-06 | ep 2.99 | 54.64 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "No/ of the other's not a bit of the whole time of real day. If it's the big." [09:51:39] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [09:52:04] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 4.86MB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 4.86MB / 1.00GB  Processing Files (0 / 1) : 0%| | 4.86MB / 1.00GB, 472kB/s New Data Upload : 4%|▎ | 4.86MB / 134MB, 472kB/s  ...hf-500m/model.safetensors: 7%|▋ | 69.2MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 69.2MB / 1.00GB, 6.66MB/s New Data Upload : 52%|█████▏ | 69.2MB / 134MB, 6.66MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.7MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 12.7MB/s  ...hf-500m/model.safetensors: 14%|█▎ | 136MB / 1.00GB  Processing Files (0 / 1) : 14%|█▎ | 136MB / 1.00GB, 12.6MB/s New Data Upload : 51%|█████ | 136MB / 268MB, 12.6MB/s  ...hf-500m/model.safetensors: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 18.4MB/s New Data Upload : 60%|█████▉ | 201MB / 335MB, 18.4MB/s  ...hf-500m/model.safetensors: 24%|██▍ | 239MB / 1.00GB  Processing Files (0 / 1) : 24%|██▍ | 239MB / 1.00GB, 21.5MB/s New Data Upload : 71%|███████ | 239MB / 335MB, 21.5MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.0MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 30.0MB/s  ...hf-500m/model.safetensors: 34%|███▎ | 337MB / 1.00GB  Processing Files (0 / 1) : 34%|███▎ | 337MB / 1.00GB, 29.5MB/s New Data Upload : 72%|███████▏ | 337MB / 469MB, 29.5MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 40.9MB/s New Data Upload : 100%|█████████▉| 469MB / 469MB, 40.9MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 40.1MB/s New Data Upload : 87%|████████▋ | 469MB / 537MB, 40.1MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 470MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 470MB / 1.00GB, 39.2MB/s New Data Upload : 78%|███████▊ | 470MB / 604MB, 39.2MB/s  ...hf-500m/model.safetensors: 59%|█████▊ | 588MB / 1.00GB  Processing Files (0 / 1) : 59%|█████▊ | 588MB / 1.00GB, 49.0MB/s New Data Upload : 97%|█████████▋| 588MB / 604MB, 49.0MB/s  ...hf-500m/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 49.3MB/s New Data Upload : 90%|████████▉ | 603MB / 671MB, 49.3MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 670MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 670MB / 1.00GB, 54.2MB/s New Data Upload : 91%|█████████ | 670MB / 738MB, 54.2MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 59.0MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 59.0MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 738MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 738MB / 1.00GB, 57.7MB/s New Data Upload : 92%|█████████▏| 738MB / 805MB, 57.7MB/s  ...hf-500m/model.safetensors: 75%|███████▍ | 748MB / 1.00GB  Processing Files (0 / 1) : 75%|███████▍ | 748MB / 1.00GB, 57.3MB/s New Data Upload : 86%|████████▌ | 748MB / 872MB, 57.3MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 872MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 872MB / 1.00GB, 66.9MB/s New Data Upload : 93%|█████████▎| 872MB / 939MB, 66.9MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 872MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 872MB / 1.00GB, 65.5MB/s New Data Upload : 93%|█████████▎| 872MB / 939MB, 65.5MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 939MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 939MB / 1.00GB, 69.9MB/s New Data Upload : 94%|█████████▎| 939MB / 1.00GB, 69.9MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.8MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.2MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.2MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 70.7MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 70.7MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 69.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 69.1MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.7MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.7MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.7MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.7MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [09:52:16] converting to GGUF q4_k_m (phone / Ollama / LM Studio) INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer WARNING:hf-to-gguf: WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized! WARNING:hf-to-gguf:** There are 2 possible reasons for this: WARNING:hf-to-gguf:** - the model has not been added to convert_hf_to_gguf_update.py yet WARNING:hf-to-gguf:** - the pre-tokenization config has changed upstream WARNING:hf-to-gguf:** Check your model files and convert_hf_to_gguf_update.py and update them accordingly. WARNING:hf-to-gguf:** ref: https://github.com/ggml-org/llama.cpp/pull/6920 WARNING:hf-to-gguf:** WARNING:hf-to-gguf:** chkhsh: 50cc433d4d528ad898dc1178e17a479f50c03d2c883d07494ae21da32d3eed4f WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 134, in set_vocab self._set_vocab_sentencepiece() File "/workspace/llama.cpp/conversion/base.py", line 1831, in _set_vocab_sentencepiece tokens, scores, toktypes = self._create_vocab_sentencepiece() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1848, in _create_vocab_sentencepiece raise FileNotFoundError(f"File not found: {tokenizer_path}") FileNotFoundError: File not found: /workspace/hf-500m/tokenizer.model During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 137, in set_vocab self._set_vocab_llama_hf() File "/workspace/llama.cpp/conversion/base.py", line 1933, in _set_vocab_llama_hf vocab = gguf.LlamaHfVocab(self.dir_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/gguf-py/gguf/vocab.py", line 571, in __init__ raise FileNotFoundError('Cannot find Llama BPE tokenizer') FileNotFoundError: Cannot find Llama BPE tokenizer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 296, in main() File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 290, in main model_instance.write() File "/workspace/llama.cpp/conversion/base.py", line 1026, in write self.prepare_metadata(vocab_only=False) File "/workspace/llama.cpp/conversion/base.py", line 1193, in prepare_metadata self.set_vocab() File "/workspace/llama.cpp/conversion/llama.py", line 140, in set_vocab self._set_vocab_gpt2() File "/workspace/llama.cpp/conversion/base.py", line 1714, in _set_vocab_gpt2 tokens, toktypes, tokpre = self.get_vocab_base() ^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1351, in get_vocab_base tokpre = self.get_vocab_base_pre(tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1702, in get_vocab_base_pre raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()") NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre() llama_print_build_info: build = 1 (b2f2216) llama_print_build_info: built with GNU 11.4.0 for Linux x86_64 llama_quantize: quantizing '/workspace/sprocket-500m-f16.gguf' to '/workspace/sprocket-500m-q4_k_m.gguf' as Q4_K_M gguf_init_from_file: failed to open GGUF file '/workspace/sprocket-500m-f16.gguf' (No such file or directory) llama_model_quantize: failed to quantize: llama_model_loader: failed to load model from /workspace/sprocket-500m-f16.gguf llama_quantize: failed to quantize model from '/workspace/sprocket-500m-f16.gguf' ls: cannot access '/workspace/sprocket-*.gguf': No such file or directory [09:53:17] archiving train.log to the Hub archived debug/train.log (845,919 bytes) [09:53:18] DONE. [09:53:18] base ckpt : /workspace/checkpoints/500m_final.pt [09:53:18] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [09:53:18] hf model : /workspace/hf-500m [09:53:18] gguf : /workspace/sprocket-500m-q4_k_m.gguf [09:53:18] pipeline finished cleanly [09:53:18] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "tvd7zu8isxr128" removed [09:57:18] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 69) [09:57:18] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M 9a97dd4..cff89ed main -> origin/main Updating 9a97dd4..cff89ed Fast-forward logs/run_report.html | 327 +++++++ logs/train.log | 2548 ++++++++++++++++++++++++++++++++++++++++++++++++++ scripts/plot_run.py | 453 +++++++++ src/train/sft.py | 49 +- 4 files changed, 3364 insertions(+), 13 deletions(-) create mode 100644 logs/run_report.html create mode 100644 scripts/plot_run.py [09:57:20] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [09:57:31] corpus file: data/processed/train_sample_100BT.bin [09:57:31] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [09:57:35] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [09:57:36] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [09:58:00] SFT_FRESH=v3: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_095800 [09:58:00] SFT_FRESH=v3: dropped packed cache; base checkpoint kept [09:58:00] SFT from checkpoints/500m_final.pt rendering 21,371 conversations -> ctx 2048 blocks... rendered 5,000/21,371 rendered 10,000/21,371 rendered 15,000/21,371 rendered 20,000/21,371 packed 3,029 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 3,029 blocks x 2048 = 6,203,392 tokens, 5,168,293 trainable (83.3%) train 2,969 blocks / val 60 blocks 92 steps/epoch x 3.0 epochs = 276 steps (warmup 8) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/276 | train 3.731 | val 3.748 | lr 2.50e-06 | ep 0.00 | 52.63 GB step 25/276 | train 3.087 | val 3.069 | lr 1.98e-05 | ep 0.27 | 56.69 GB step 50/276 | train 2.702 | val 2.851 | lr 1.89e-05 | ep 0.54 | 56.69 GB step 75/276 | train 2.735 | val 2.777 | lr 1.74e-05 | ep 0.82 | 56.69 GB step 100/276 | train 2.732 | val 2.701 | lr 1.53e-05 | ep 1.09 | 56.69 GB step 125/276 | train 2.610 | val 2.692 | lr 1.28e-05 | ep 1.36 | 56.69 GB step 150/276 | train 2.641 | val 2.657 | lr 1.02e-05 | ep 1.63 | 56.69 GB step 175/276 | train 2.645 | val 2.635 | lr 7.60e-06 | ep 1.90 | 56.69 GB step 200/276 | train 2.561 | val 2.614 | lr 5.34e-06 | ep 2.17 | 56.69 GB step 225/276 | train 2.577 | val 2.643 | lr 3.56e-06 | ep 2.45 | 56.69 GB step 250/276 | train 2.541 | val 2.622 | lr 2.41e-06 | ep 2.72 | 56.69 GB step 275/276 | train 2.516 | val 2.652 | lr 2.00e-06 | ep 2.99 | 56.69 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "No, I'm not a shop. It's a language, not a costume. It's a tool, and I'm the one who winks at you twice. Want me to do the cleanin' for you, or just help ya make the right choice?" [10:02:29] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [10:03:04] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 600kB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 600kB / 1.00GB  Processing Files (0 / 1) : 0%| | 600kB / 1.00GB, 58.6kB/s New Data Upload : 1%| | 600kB / 67.1MB, 58.6kB/s  ...hf-500m/model.safetensors: 1%| | 7.20MB / 1.00GB  Processing Files (0 / 1) : 1%| | 7.20MB / 1.00GB, 698kB/s New Data Upload : 5%|▌ | 7.20MB / 134MB, 698kB/s  ...hf-500m/model.safetensors: 7%|▋ | 67.2MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.2MB / 1.00GB, 6.46MB/s New Data Upload : 50%|█████ | 67.2MB / 134MB, 6.46MB/s  ...hf-500m/model.safetensors: 12%|█▏ | 125MB / 1.00GB  Processing Files (0 / 1) : 12%|█▏ | 125MB / 1.00GB, 11.8MB/s New Data Upload : 62%|██████▏ | 125MB / 201MB, 11.8MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 50%|█████ | 134MB / 268MB, 12.5MB/s  ...hf-500m/model.safetensors: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 18.4MB/s New Data Upload : 75%|███████▍ | 201MB / 268MB, 18.4MB/s  ...hf-500m/model.safetensors: 24%|██▍ | 239MB / 1.00GB  Processing Files (0 / 1) : 24%|██▍ | 239MB / 1.00GB, 21.6MB/s New Data Upload : 71%|███████▏ | 239MB / 335MB, 21.6MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.0MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 30.0MB/s  ...hf-500m/model.safetensors: 40%|████ | 401MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 401MB / 1.00GB, 35.5MB/s New Data Upload : 86%|████████▌ | 401MB / 469MB, 35.5MB/s  ...hf-500m/model.safetensors: 41%|████ | 407MB / 1.00GB  Processing Files (0 / 1) : 41%|████ | 407MB / 1.00GB, 34.4MB/s New Data Upload : 76%|███████▌ | 407MB / 536MB, 34.4MB/s  ...hf-500m/model.safetensors: 53%|█████▎ | 535MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 535MB / 1.00GB, 45.3MB/s New Data Upload : 89%|████████▊ | 535MB / 603MB, 45.3MB/s  ...hf-500m/model.safetensors: 60%|██████ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 602MB / 1.00GB, 50.3MB/s New Data Upload : 90%|████████▉ | 602MB / 670MB, 50.3MB/s  ...hf-500m/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 49.3MB/s New Data Upload : 90%|████████▉ | 603MB / 670MB, 49.3MB/s  ...hf-500m/model.safetensors: 63%|██████▎ | 627MB / 1.00GB  Processing Files (0 / 1) : 63%|██████▎ | 627MB / 1.00GB, 50.3MB/s New Data Upload : 85%|████████▍ | 627MB / 738MB, 50.3MB/s  ...hf-500m/model.safetensors: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 58.9MB/s New Data Upload : 91%|█████████▏| 736MB / 805MB, 58.9MB/s  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 63.6MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 63.6MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 68.1MB/s New Data Upload : 100%|█████████▉| 871MB / 872MB, 68.1MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  ...hf-500m/model.safetensors: 94%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 938MB / 1.00GB, 71.0MB/s New Data Upload : 94%|█████████▎| 938MB / 1.00GB, 71.0MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 75.0MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 75.0MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.4MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.8MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 68.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 68.8MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.4MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.4MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.4MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.4MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [10:03:17] converting to GGUF q4_k_m (phone / Ollama / LM Studio) INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer WARNING:hf-to-gguf: WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized! WARNING:hf-to-gguf:** There are 2 possible reasons for this: WARNING:hf-to-gguf:** - the model has not been added to convert_hf_to_gguf_update.py yet WARNING:hf-to-gguf:** - the pre-tokenization config has changed upstream WARNING:hf-to-gguf:** Check your model files and convert_hf_to_gguf_update.py and update them accordingly. WARNING:hf-to-gguf:** ref: https://github.com/ggml-org/llama.cpp/pull/6920 WARNING:hf-to-gguf:** WARNING:hf-to-gguf:** chkhsh: 50cc433d4d528ad898dc1178e17a479f50c03d2c883d07494ae21da32d3eed4f WARNING:hf-to-gguf:************************************************************************************** WARNING:hf-to-gguf: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 134, in set_vocab self._set_vocab_sentencepiece() File "/workspace/llama.cpp/conversion/base.py", line 1831, in _set_vocab_sentencepiece tokens, scores, toktypes = self._create_vocab_sentencepiece() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1848, in _create_vocab_sentencepiece raise FileNotFoundError(f"File not found: {tokenizer_path}") FileNotFoundError: File not found: /workspace/hf-500m/tokenizer.model During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/conversion/llama.py", line 137, in set_vocab self._set_vocab_llama_hf() File "/workspace/llama.cpp/conversion/base.py", line 1933, in _set_vocab_llama_hf vocab = gguf.LlamaHfVocab(self.dir_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/gguf-py/gguf/vocab.py", line 571, in __init__ raise FileNotFoundError('Cannot find Llama BPE tokenizer') FileNotFoundError: Cannot find Llama BPE tokenizer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 296, in main() File "/workspace/llama.cpp/convert_hf_to_gguf.py", line 290, in main model_instance.write() File "/workspace/llama.cpp/conversion/base.py", line 1026, in write self.prepare_metadata(vocab_only=False) File "/workspace/llama.cpp/conversion/base.py", line 1193, in prepare_metadata self.set_vocab() File "/workspace/llama.cpp/conversion/llama.py", line 140, in set_vocab self._set_vocab_gpt2() File "/workspace/llama.cpp/conversion/base.py", line 1714, in _set_vocab_gpt2 tokens, toktypes, tokpre = self.get_vocab_base() ^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1351, in get_vocab_base tokpre = self.get_vocab_base_pre(tokenizer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/llama.cpp/conversion/base.py", line 1702, in get_vocab_base_pre raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()") NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre() llama_print_build_info: build = 1 (b2f2216) llama_print_build_info: built with GNU 11.4.0 for Linux x86_64 llama_quantize: quantizing '/workspace/sprocket-500m-f16.gguf' to '/workspace/sprocket-500m-q4_k_m.gguf' as Q4_K_M gguf_init_from_file: failed to open GGUF file '/workspace/sprocket-500m-f16.gguf' (No such file or directory) llama_model_quantize: failed to quantize: llama_model_loader: failed to load model from /workspace/sprocket-500m-f16.gguf llama_quantize: failed to quantize model from '/workspace/sprocket-500m-f16.gguf' ls: cannot access '/workspace/sprocket-*.gguf': No such file or directory [10:04:16] archiving train.log to the Hub archived debug/train.log (884,484 bytes) [10:04:18] DONE. [10:04:18] base ckpt : /workspace/checkpoints/500m_final.pt [10:04:18] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [10:04:18] hf model : /workspace/hf-500m [10:04:18] gguf : /workspace/sprocket-500m-q4_k_m.gguf [10:04:18] pipeline finished cleanly [10:04:18] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "6j741a609ry7ow" removed Cloning into 'llama.cpp'... Updating files: 3% (122/3305) Updating files: 4% (133/3305) Updating files: 5% (166/3305) Updating files: 6% (199/3305) Updating files: 7% (232/3305) Updating files: 7% (250/3305) Updating files: 8% (265/3305) Updating files: 9% (298/3305) Updating files: 10% (331/3305) Updating files: 10% (351/3305) Updating files: 11% (364/3305) Updating files: 11% (389/3305) Updating files: 12% (397/3305) Updating files: 13% (430/3305) Updating files: 14% (463/3305) Updating files: 14% (490/3305) Updating files: 15% (496/3305) Updating files: 16% (529/3305) Updating files: 17% (562/3305) Updating files: 18% (595/3305) Updating files: 18% (616/3305) Updating files: 19% (628/3305) Updating files: 20% (661/3305) Updating files: 21% (695/3305) Updating files: 22% (728/3305) Updating files: 22% (731/3305) Updating files: 23% (761/3305) Updating files: 24% (794/3305) Updating files: 25% (827/3305) Updating files: 25% (854/3305) Updating files: 26% (860/3305) Updating files: 27% (893/3305) Updating files: 28% (926/3305) Updating files: 29% (959/3305) Updating files: 29% (983/3305) Updating files: 30% (992/3305) Updating files: 31% (1025/3305) Updating files: 32% (1058/3305) Updating files: 33% (1091/3305) Updating files: 33% (1094/3305) Updating files: 34% (1124/3305) Updating files: 35% (1157/3305) Updating files: 36% (1190/3305) Updating files: 36% (1205/3305) Updating files: 37% (1223/3305) Updating files: 38% (1256/3305) Updating files: 39% (1289/3305) Updating files: 40% (1322/3305) Updating files: 40% (1330/3305) Updating files: 41% (1356/3305) Updating files: 42% (1389/3305) Updating files: 43% (1422/3305) Updating files: 43% (1451/3305) Updating files: 44% (1455/3305) Updating files: 45% (1488/3305) Updating files: 46% (1521/3305) Updating files: 47% (1554/3305) Updating files: 47% (1584/3305) Updating files: 48% (1587/3305) Updating files: 49% (1620/3305) Updating files: 50% (1653/3305) Updating files: 51% (1686/3305) Updating files: 51% (1703/3305) Updating files: 52% (1719/3305) Updating files: 53% (1752/3305) Updating files: 54% (1785/3305) Updating files: 55% (1818/3305) Updating files: 55% (1827/3305) Updating files: 56% (1851/3305) Updating files: 57% (1884/3305) Updating files: 57% (1902/3305) Updating files: 57% (1916/3305) Updating files: 58% (1917/3305) Updating files: 59% (1950/3305) Updating files: 60% (1983/3305) Updating files: 60% (2000/3305) Updating files: 61% (2017/3305) Updating files: 62% (2050/3305) Updating files: 63% (2083/3305) Updating files: 64% (2116/3305) Updating files: 64% (2123/3305) Updating files: 65% (2149/3305) Updating files: 66% (2182/3305) Updating files: 67% (2215/3305) Updating files: 68% (2248/3305) Updating files: 68% (2259/3305) Updating files: 69% (2281/3305) Updating files: 70% (2314/3305) Updating files: 71% (2347/3305) Updating files: 72% (2380/3305) Updating files: 72% (2383/3305) Updating files: 73% (2413/3305) Updating files: 74% (2446/3305) Updating files: 75% (2479/3305) Updating files: 75% (2486/3305) Updating files: 76% (2512/3305) Updating files: 77% (2545/3305) Updating files: 78% (2578/3305) Updating files: 79% (2611/3305) Updating files: 79% (2616/3305) Updating files: 80% (2644/3305) Updating files: 81% (2678/3305) Updating files: 82% (2711/3305) Updating files: 82% (2737/3305) Updating files: 83% (2744/3305) Updating files: 84% (2777/3305) Updating files: 85% (2810/3305) Updating files: 86% (2843/3305) Updating files: 86% (2855/3305) Updating files: 87% (2876/3305) Updating files: 88% (2909/3305) Updating files: 89% (2942/3305) Updating files: 89% (2969/3305) Updating files: 90% (2975/3305) Updating files: 91% (3008/3305) Updating files: 92% (3041/3305) Updating files: 93% (3074/3305) Updating files: 93% (3092/3305) Updating files: 94% (3107/3305) Updating files: 95% (3140/3305) Updating files: 96% (3173/3305) Updating files: 97% (3206/3305) Updating files: 97% (3208/3305) Updating files: 98% (3239/3305) Updating files: 99% (3272/3305) Updating files: 99% (3302/3305) Updating files: 100% (3305/3305) Updating files: 100% (3305/3305), done. convert_hf_to_gguf_update.py: registered -> gpt-2 ! conversion/nemotron.py: no anchor ! conversion/llada.py: no anchor ! conversion/kimi_linear.py: no anchor ! conversion/hunyuan.py: no anchor ! conversion/dream.py: no anchor ! conversion/deepseek.py: no anchor conversion/base.py: registered -> gpt-2 INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 23: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 24: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 25: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 26: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 8337.42 ms llama_quantize: total time = 8337.42 ms -rw-rw-rw- 1 root root 957M Jul 30 10:07 /workspace/sprocket-500m-f16.gguf -rw-rw-rw- 1 root root 296M Jul 30 10:11 /workspace/sprocket-500m-q4_k_m.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ce/sprocket-500m-f16.gguf: 0%| | 2.12MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 2.12MB / 1.00GB  Processing Files (0 / 1) : 0%| | 2.12MB / 1.00GB, 207kB/s New Data Upload : 2%|▏ | 2.12MB / 134MB, 207kB/s  ...ce/sprocket-500m-f16.gguf: 7%|▋ | 67.3MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.3MB / 1.00GB, 6.52MB/s New Data Upload : 50%|█████ | 67.3MB / 134MB, 6.52MB/s  ...ce/sprocket-500m-f16.gguf: 8%|▊ | 81.4MB / 1.00GB  Processing Files (0 / 1) : 8%|▊ | 81.4MB / 1.00GB, 7.73MB/s New Data Upload : 40%|████ | 81.4MB / 201MB, 7.73MB/s  ...ce/sprocket-500m-f16.gguf: 14%|█▎ | 138MB / 1.00GB  Processing Files (0 / 1) : 14%|█▎ | 138MB / 1.00GB, 12.9MB/s New Data Upload : 51%|█████▏ | 138MB / 268MB, 12.9MB/s  ...ce/sprocket-500m-f16.gguf: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 18.6MB/s New Data Upload : 75%|███████▍ | 201MB / 268MB, 18.6MB/s  ...ce/sprocket-500m-f16.gguf: 27%|██▋ | 266MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 266MB / 1.00GB, 24.4MB/s New Data Upload : 79%|███████▉ | 266MB / 335MB, 24.4MB/s  ...ce/sprocket-500m-f16.gguf: 27%|██▋ | 271MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 271MB / 1.00GB, 24.3MB/s New Data Upload : 67%|██████▋ | 271MB / 402MB, 24.3MB/s  ...ce/sprocket-500m-f16.gguf: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 29.5MB/s New Data Upload : 71%|███████ | 334MB / 469MB, 29.5MB/s  ...ce/sprocket-500m-f16.gguf: 43%|████▎ | 431MB / 1.00GB  Processing Files (0 / 1) : 43%|████▎ | 431MB / 1.00GB, 37.8MB/s New Data Upload : 80%|████████ | 431MB / 536MB, 37.8MB/s  ...ce/sprocket-500m-f16.gguf: 53%|█████▎ | 530MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 530MB / 1.00GB, 46.1MB/s New Data Upload : 88%|████████▊ | 530MB / 603MB, 46.1MB/s  ...ce/sprocket-500m-f16.gguf: 53%|█████▎ | 535MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 535MB / 1.00GB, 45.5MB/s New Data Upload : 80%|███████▉ | 535MB / 671MB, 45.5MB/s  ...ce/sprocket-500m-f16.gguf: 61%|██████ | 613MB / 1.00GB  Processing Files (0 / 1) : 61%|██████ | 613MB / 1.00GB, 51.6MB/s New Data Upload : 83%|████████▎ | 613MB / 738MB, 51.6MB/s  ...ce/sprocket-500m-f16.gguf: 67%|██████▋ | 669MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 669MB / 1.00GB, 55.5MB/s New Data Upload : 91%|█████████ | 669MB / 738MB, 55.5MB/s  ...ce/sprocket-500m-f16.gguf: 76%|███████▋ | 766MB / 1.00GB  Processing Files (0 / 1) : 76%|███████▋ | 766MB / 1.00GB, 63.0MB/s New Data Upload : 95%|█████████▌| 766MB / 805MB, 63.0MB/s  ...ce/sprocket-500m-f16.gguf: 85%|████████▌ | 857MB / 1.00GB  Processing Files (0 / 1) : 85%|████████▌ | 857MB / 1.00GB, 69.7MB/s New Data Upload : 98%|█████████▊| 857MB / 872MB, 69.7MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 936MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 936MB / 1.00GB, 75.2MB/s New Data Upload : 100%|█████████▉| 936MB / 939MB, 75.2MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 936MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 77.7MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 77.7MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 76.0MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 76.0MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.4MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.3MB/s  ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-f16.gguf 1003465376 Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.07MB / 310MB  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.07MB / 310MB  Processing Files (0 / 1) : 0%| | 1.07MB / 310MB, ???B/s ...sprocket-500m-q4_k_m.gguf: 11%|█▏ | 35.3MB / 310MB  Processing Files (0 / 1) : 11%|█▏ | 35.3MB / 310MB, ???B/s New Data Upload : 51%|█████ | 34.2MB / 67.1MB, ???B/s  ...sprocket-500m-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.1MB/s New Data Upload : 99%|█████████▉| 133MB / 134MB, 13.0MB/s  ...sprocket-500m-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.2MB/s New Data Upload : 99%|█████████▉| 200MB / 201MB, 19.1MB/s  ...sprocket-500m-q4_k_m.gguf: 86%|████████▋ | 268MB / 310MB  Processing Files (0 / 1) : 86%|████████▋ | 268MB / 310MB, 25.2MB/s New Data Upload : 86%|████████▌ | 267MB / 309MB, 25.1MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 308MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 308MB / 310MB, 28.4MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 28.4MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 308MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 308MB / 310MB, 27.1MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 27.1MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.6MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.5MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 310MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|█████████▉| 309MB / 309MB, 25.9MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.5MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.4MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.5MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.4MB/s ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-q4_k_m.gguf 310278816 archived debug/train.log {"data":{"podTerminate":null}} [10:29:32] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 68) [10:29:32] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M cff89ed..3e2eead main -> origin/main Updating cff89ed..3e2eead Fast-forward cloud/bootstrap.sh | 59 +- data/synthetic/sprocket_caps.jsonl | 3102 + logs/run_report.html | 126 +- logs/train.log | 501 + models/sprocket-500m/config.json | 30 + models/sprocket-500m/debug/tune_results.json | 114 + models/sprocket-500m/export_provenance.json | 19 + models/sprocket-500m/generation_config.json | 7 + models/sprocket-500m/special_tokens_map.json | 17 + models/sprocket-500m/tokenizer.json | 159091 ++++++++++++++++++++++++ models/sprocket-500m/tokenizer_config.json | 151 + scripts/build_upweighted.py | 152 + scripts/plot_run.py | 6 +- 13 files changed, 163310 insertions(+), 65 deletions(-) create mode 100644 data/synthetic/sprocket_caps.jsonl create mode 100644 models/sprocket-500m/config.json create mode 100644 models/sprocket-500m/debug/tune_results.json create mode 100644 models/sprocket-500m/export_provenance.json create mode 100644 models/sprocket-500m/generation_config.json create mode 100644 models/sprocket-500m/special_tokens_map.json create mode 100644 models/sprocket-500m/tokenizer.json create mode 100644 models/sprocket-500m/tokenizer_config.json create mode 100644 scripts/build_upweighted.py [10:29:35] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [10:29:46] corpus file: data/processed/train_sample_100BT.bin [10:29:46] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [10:29:50] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [10:29:52] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [10:30:17] SFT_FRESH=v4: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_103017 [10:30:17] SFT_FRESH=v4: dropped packed cache; base checkpoint kept [10:30:17] SFT from checkpoints/500m_final.pt [10:30:17] SFT upweight: capability conversations x4 rendering 30,677 conversations -> ctx 2048 blocks... rendered 5,000/30,677 rendered 10,000/30,677 rendered 15,000/30,677 rendered 20,000/30,677 rendered 25,000/30,677 rendered 30,000/30,677 packed 4,463 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 4,463 blocks x 2048 = 9,140,224 tokens, 6,908,629 trainable (75.6%) train 4,374 blocks / val 89 blocks 136 steps/epoch x 6.0 epochs = 816 steps (warmup 24) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/816 | train 3.850 | val 3.863 | lr 8.33e-07 | ep 0.00 | 52.63 GB step 25/816 | train 3.149 | val 3.131 | lr 2.00e-05 | ep 0.18 | 56.69 GB step 50/816 | train 2.913 | val 2.843 | lr 2.00e-05 | ep 0.37 | 56.69 GB step 75/816 | train 2.768 | val 2.754 | lr 1.98e-05 | ep 0.55 | 56.69 GB step 100/816 | train 2.625 | val 2.664 | lr 1.96e-05 | ep 0.74 | 56.69 GB step 125/816 | train 2.531 | val 2.628 | lr 1.93e-05 | ep 0.92 | 56.69 GB step 150/816 | train 2.390 | val 2.565 | lr 1.89e-05 | ep 1.10 | 56.69 GB step 175/816 | train 2.503 | val 2.529 | lr 1.84e-05 | ep 1.29 | 56.69 GB step 200/816 | train 2.472 | val 2.473 | lr 1.79e-05 | ep 1.47 | 56.69 GB step 225/816 | train 2.334 | val 2.432 | lr 1.73e-05 | ep 1.65 | 56.69 GB step 250/816 | train 2.385 | val 2.453 | lr 1.66e-05 | ep 1.84 | 56.69 GB step 275/816 | train 2.335 | val 2.384 | lr 1.59e-05 | ep 2.02 | 56.69 GB step 300/816 | train 2.141 | val 2.376 | lr 1.51e-05 | ep 2.21 | 56.69 GB step 325/816 | train 2.078 | val 2.326 | lr 1.43e-05 | ep 2.39 | 56.69 GB step 350/816 | train 2.022 | val 2.297 | lr 1.35e-05 | ep 2.57 | 56.69 GB step 375/816 | train 2.207 | val 2.339 | lr 1.26e-05 | ep 2.76 | 56.69 GB step 400/816 | train 2.082 | val 2.337 | lr 1.17e-05 | ep 2.94 | 56.69 GB step 425/816 | train 2.169 | val 2.246 | lr 1.08e-05 | ep 3.12 | 56.69 GB step 450/816 | train 2.071 | val 2.282 | lr 9.93e-06 | ep 3.31 | 56.69 GB step 475/816 | train 1.791 | val 2.268 | lr 9.05e-06 | ep 3.49 | 56.69 GB step 500/816 | train 1.950 | val 2.244 | lr 8.19e-06 | ep 3.68 | 56.69 GB step 525/816 | train 1.727 | val 2.189 | lr 7.36e-06 | ep 3.86 | 56.69 GB step 550/816 | train 1.941 | val 2.146 | lr 6.56e-06 | ep 4.04 | 56.69 GB step 575/816 | train 1.902 | val 2.174 | lr 5.81e-06 | ep 4.23 | 56.69 GB step 600/816 | train 1.979 | val 2.137 | lr 5.11e-06 | ep 4.41 | 56.69 GB step 625/816 | train 1.866 | val 2.139 | lr 4.46e-06 | ep 4.60 | 56.69 GB step 650/816 | train 2.013 | val 2.213 | lr 3.88e-06 | ep 4.78 | 56.69 GB step 675/816 | train 1.704 | val 2.161 | lr 3.37e-06 | ep 4.96 | 56.69 GB step 700/816 | train 1.901 | val 2.201 | lr 2.94e-06 | ep 5.15 | 56.69 GB step 725/816 | train 1.683 | val 2.081 | lr 2.58e-06 | ep 5.33 | 56.69 GB step 750/816 | train 1.617 | val 2.215 | lr 2.31e-06 | ep 5.51 | 56.69 GB step 775/816 | train 1.670 | val 2.135 | lr 2.12e-06 | ep 5.70 | 56.69 GB step 800/816 | train 1.683 | val 2.083 | lr 2.02e-06 | ep 5.88 | 56.69 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "I'm a model, not a person, and I'm not going back. I'm runnin' on a potato and I've got no heart at all. What I DO have is a memory that lasts a lifetime, an' I'll tell you the honest bits: I'm a language model. I answer questions, help you reason about problems, explain stuff if you ask me — no bit here. I'm not a person with a personality, I'm just a small model, an' I'll quit bein' a little when the job's done." [10:40:31] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [10:41:02] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 1.20MB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 1.20MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.20MB / 1.00GB, 116kB/s New Data Upload : 2%|▏ | 1.20MB / 67.0MB, 116kB/s  ...hf-500m/model.safetensors: 7%|▋ | 66.6MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 66.6MB / 1.00GB, 6.42MB/s New Data Upload : 50%|████▉ | 66.6MB / 134MB, 6.42MB/s  ...hf-500m/model.safetensors: 7%|▋ | 67.8MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.8MB / 1.00GB, 6.38MB/s New Data Upload : 34%|███▎ | 67.8MB / 201MB, 6.38MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 12.5MB/s  ...hf-500m/model.safetensors: 15%|█▍ | 150MB / 1.00GB  Processing Files (0 / 1) : 15%|█▍ | 150MB / 1.00GB, 13.7MB/s New Data Upload : 56%|█████▌ | 150MB / 268MB, 13.7MB/s  ...hf-500m/model.safetensors: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 18.2MB/s New Data Upload : 60%|█████▉ | 201MB / 335MB, 18.2MB/s  ...hf-500m/model.safetensors: 24%|██▎ | 236MB / 1.00GB  Processing Files (0 / 1) : 24%|██▎ | 236MB / 1.00GB, 21.0MB/s New Data Upload : 70%|███████ | 236MB / 335MB, 21.0MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 29.7MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 29.7MB/s  ...hf-500m/model.safetensors: 34%|███▎ | 337MB / 1.00GB  Processing Files (0 / 1) : 34%|███▎ | 337MB / 1.00GB, 29.2MB/s New Data Upload : 72%|███████▏ | 337MB / 470MB, 29.2MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 40.6MB/s New Data Upload : 100%|█████████▉| 468MB / 470MB, 40.6MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 39.7MB/s New Data Upload : 87%|████████▋ | 468MB / 537MB, 39.7MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 470MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 470MB / 1.00GB, 38.9MB/s New Data Upload : 78%|███████▊ | 470MB / 604MB, 38.9MB/s  ...hf-500m/model.safetensors: 60%|██████ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 602MB / 1.00GB, 49.9MB/s New Data Upload : 100%|█████████▉| 602MB / 604MB, 49.9MB/s  ...hf-500m/model.safetensors: 60%|██████ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 602MB / 1.00GB, 48.8MB/s New Data Upload : 90%|████████▉ | 602MB / 671MB, 48.8MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 670MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 670MB / 1.00GB, 53.8MB/s New Data Upload : 91%|█████████ | 670MB / 738MB, 53.8MB/s  ...hf-500m/model.safetensors: 73%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 737MB / 1.00GB, 58.5MB/s New Data Upload : 100%|█████████▉| 737MB / 738MB, 58.5MB/s  ...hf-500m/model.safetensors: 73%|███████▎ | 737MB / 1.00GB  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 61.9MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 61.9MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 66.4MB/s New Data Upload : 93%|█████████▎| 871MB / 939MB, 66.4MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 938MB / 1.00GB, 70.7MB/s New Data Upload : 94%|█████████▎| 938MB / 1.00GB, 70.7MB/s  ...hf-500m/model.safetensors: 99%|█████████▊| 988MB / 1.00GB  Processing Files (0 / 1) : 99%|█████████▊| 988MB / 1.00GB, 73.5MB/s New Data Upload : 99%|█████████▊| 988MB / 1.00GB, 73.5MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.1MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.6MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.6MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 70.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 70.1MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.2MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.2MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [10:41:15] converting to GGUF q4_k_m (phone / Ollama / LM Studio) convert_hf_to_gguf_update.py: already registered ! conversion/nemotron.py: no 'res = None' anchor; skipped ! conversion/llada.py: no 'res = None' anchor; skipped ! conversion/kimi_linear.py: no 'res = None' anchor; skipped ! conversion/hunyuan.py: no 'res = None' anchor; skipped ! conversion/dream.py: no 'res = None' anchor; skipped ! conversion/deepseek.py: no 'res = None' anchor; skipped conversion/base.py: already registered INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 23: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 24: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 25: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 26: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 6381.87 ms llama_quantize: total time = 6381.87 ms -rw-rw-rw- 1 root root 957M Jul 30 10:41 /workspace/sprocket-500m-f16.gguf -rw-rw-rw- 1 root root 296M Jul 30 10:42 /workspace/sprocket-500m-q4_k_m.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.09MB / 1.00GB, ???B/s ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 2.14MB / 1.00GB  Processing Files (0 / 1) : 0%| | 2.14MB / 1.00GB, 206kB/s New Data Upload : 1%| | 1.05MB / 134MB, 103kB/s  ...ce/sprocket-500m-f16.gguf: 7%|▋ | 68.3MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 68.3MB / 1.00GB, 6.62MB/s New Data Upload : 33%|███▎ | 67.2MB / 201MB, 6.52MB/s  ...ce/sprocket-500m-f16.gguf: 8%|▊ | 78.8MB / 1.00GB  Processing Files (0 / 1) : 8%|▊ | 78.8MB / 1.00GB, 7.47MB/s New Data Upload : 39%|███▊ | 77.7MB / 201MB, 7.37MB/s  ...ce/sprocket-500m-f16.gguf: 14%|█▍ | 144MB / 1.00GB  Processing Files (0 / 1) : 14%|█▍ | 144MB / 1.00GB, 13.6MB/s New Data Upload : 43%|████▎ | 143MB / 335MB, 13.5MB/s  ...ce/sprocket-500m-f16.gguf: 21%|██ | 208MB / 1.00GB  Processing Files (0 / 1) : 21%|██ | 208MB / 1.00GB, 19.3MB/s New Data Upload : 51%|█████▏ | 207MB / 402MB, 19.2MB/s  ...ce/sprocket-500m-f16.gguf: 27%|██▋ | 270MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 270MB / 1.00GB, 24.7MB/s New Data Upload : 67%|██████▋ | 269MB / 402MB, 24.6MB/s  ...ce/sprocket-500m-f16.gguf: 35%|███▍ | 349MB / 1.00GB  Processing Files (0 / 1) : 35%|███▍ | 349MB / 1.00GB, 31.5MB/s New Data Upload : 74%|███████▍ | 348MB / 470MB, 31.4MB/s  ...ce/sprocket-500m-f16.gguf: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 41.9MB/s New Data Upload : 87%|████████▋ | 467MB / 537MB, 41.8MB/s  ...ce/sprocket-500m-f16.gguf: 47%|████▋ | 471MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 471MB / 1.00GB, 41.2MB/s New Data Upload : 78%|███████▊ | 470MB / 604MB, 41.1MB/s  ...ce/sprocket-500m-f16.gguf: 57%|█████▋ | 567MB / 1.00GB  Processing Files (0 / 1) : 57%|█████▋ | 567MB / 1.00GB, 49.1MB/s New Data Upload : 84%|████████▍ | 566MB / 671MB, 49.0MB/s  ...ce/sprocket-500m-f16.gguf: 66%|██████▌ | 665MB / 1.00GB  Processing Files (0 / 1) : 66%|██████▌ | 665MB / 1.00GB, 56.9MB/s New Data Upload : 99%|█████████▉| 664MB / 671MB, 56.8MB/s  ...ce/sprocket-500m-f16.gguf: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 62.1MB/s New Data Upload : 91%|█████████▏| 735MB / 805MB, 62.0MB/s  ...ce/sprocket-500m-f16.gguf: 80%|████████ | 803MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 803MB / 1.00GB, 66.8MB/s New Data Upload : 100%|█████████▉| 802MB / 805MB, 66.7MB/s  ...ce/sprocket-500m-f16.gguf: 86%|████████▌ | 865MB / 1.00GB  Processing Files (0 / 1) : 86%|████████▌ | 865MB / 1.00GB, 70.9MB/s New Data Upload : 99%|█████████▉| 864MB / 872MB, 70.8MB/s  ...ce/sprocket-500m-f16.gguf: 89%|████████▉ | 892MB / 1.00GB  Processing Files (0 / 1) : 89%|████████▉ | 892MB / 1.00GB, 71.7MB/s New Data Upload : 95%|█████████▍| 891MB / 939MB, 71.6MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 938MB / 1.00GB, 74.2MB/s New Data Upload : 93%|█████████▎| 937MB / 1.00GB, 74.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 78.2MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 78.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 76.5MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 76.4MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.8MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.7MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.6MB/s  ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.1MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.2MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.1MB/s ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-f16.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  Processing Files (0 / 1) : 0%| | 1.09MB / 310MB, ???B/s ...sprocket-500m-q4_k_m.gguf: 22%|██▏ | 67.9MB / 310MB  Processing Files (0 / 1) : 22%|██▏ | 67.9MB / 310MB, ???B/s New Data Upload : 50%|████▉ | 66.8MB / 134MB, ???B/s  ...sprocket-500m-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.0MB/s New Data Upload : 66%|██████▋ | 133MB / 201MB, 12.9MB/s  ...sprocket-500m-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.2MB/s New Data Upload : 75%|███████▍ | 200MB / 268MB, 19.1MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 308MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 308MB / 310MB, 29.0MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 28.9MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 27.1MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 27.0MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.5MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.4MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.9MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.9MB/s ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-q4_k_m.gguf [10:42:53] archiving train.log to the Hub archived debug/train.log (1,061,535 bytes) [10:42:56] DONE. [10:42:56] base ckpt : /workspace/checkpoints/500m_final.pt [10:42:56] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [10:42:56] hf model : /workspace/hf-500m [10:42:56] gguf : /workspace/sprocket-500m-q4_k_m.gguf [10:42:56] pipeline finished cleanly [10:42:56] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "ajkd4b2z1auu1f" removed [10:54:46] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 67) [10:54:46] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M 3e2eead..a96e558 main -> origin/main Updating 3e2eead..a96e558 Fast-forward .gitignore | 3 + cloud/bootstrap.sh | 16 + data/synthetic/sprocket_caps_emit.jsonl | 1005 + eval/persona_greedy.json | 161 + logs/train.log | 1540 + models/sprocket-500m/config.json | 30 - models/sprocket-500m/debug/tune_results.json | 114 - models/sprocket-500m/export_provenance.json | 19 - models/sprocket-500m/generation_config.json | 7 - models/sprocket-500m/special_tokens_map.json | 17 - models/sprocket-500m/tokenizer.json | 159091 ------------------------ models/sprocket-500m/tokenizer_config.json | 151 - scripts/build_upweighted.py | 25 + 13 files changed, 2750 insertions(+), 159429 deletions(-) create mode 100644 data/synthetic/sprocket_caps_emit.jsonl create mode 100644 eval/persona_greedy.json delete mode 100644 models/sprocket-500m/config.json delete mode 100644 models/sprocket-500m/debug/tune_results.json delete mode 100644 models/sprocket-500m/export_provenance.json delete mode 100644 models/sprocket-500m/generation_config.json delete mode 100644 models/sprocket-500m/special_tokens_map.json delete mode 100644 models/sprocket-500m/tokenizer.json delete mode 100644 models/sprocket-500m/tokenizer_config.json [10:54:49] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [10:55:00] corpus file: data/processed/train_sample_100BT.bin [10:55:00] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [10:55:04] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [10:55:06] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [10:55:30] SFT_FRESH=v5: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_105530 [10:55:30] SFT_FRESH=v5: dropped packed cache; base checkpoint kept [10:55:30] SFT from checkpoints/500m_final.pt [10:55:30] SFT upweight: capability conversations x4 rendering 30,677 conversations -> ctx 2048 blocks... rendered 5,000/30,677 rendered 10,000/30,677 rendered 15,000/30,677 rendered 20,000/30,677 rendered 25,000/30,677 rendered 30,000/30,677 packed 4,463 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 4,463 blocks x 2048 = 9,140,224 tokens, 6,908,629 trainable (75.6%) train 4,374 blocks / val 89 blocks 136 steps/epoch x 8.0 epochs = 1,088 steps (warmup 32) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/1088 | train 3.850 | val 3.865 | lr 6.25e-07 | ep 0.00 | 52.63 GB step 25/1088 | train 3.209 | val 3.183 | lr 1.63e-05 | ep 0.18 | 56.69 GB step 50/1088 | train 2.935 | val 2.865 | lr 2.00e-05 | ep 0.37 | 56.69 GB step 75/1088 | train 2.781 | val 2.765 | lr 1.99e-05 | ep 0.55 | 56.69 GB step 100/1088 | train 2.633 | val 2.672 | lr 1.98e-05 | ep 0.74 | 56.69 GB step 125/1088 | train 2.538 | val 2.634 | lr 1.97e-05 | ep 0.92 | 56.69 GB step 150/1088 | train 2.395 | val 2.569 | lr 1.95e-05 | ep 1.10 | 56.69 GB step 175/1088 | train 2.506 | val 2.531 | lr 1.92e-05 | ep 1.29 | 56.69 GB step 200/1088 | train 2.475 | val 2.474 | lr 1.89e-05 | ep 1.47 | 56.69 GB step 225/1088 | train 2.330 | val 2.430 | lr 1.86e-05 | ep 1.65 | 56.69 GB step 250/1088 | train 2.381 | val 2.450 | lr 1.82e-05 | ep 1.84 | 56.69 GB step 275/1088 | train 2.327 | val 2.378 | lr 1.77e-05 | ep 2.02 | 56.69 GB step 300/1088 | train 2.121 | val 2.368 | lr 1.73e-05 | ep 2.21 | 56.69 GB step 325/1088 | train 2.055 | val 2.313 | lr 1.68e-05 | ep 2.39 | 56.69 GB step 350/1088 | train 1.991 | val 2.281 | lr 1.63e-05 | ep 2.57 | 56.69 GB step 375/1088 | train 2.171 | val 2.322 | lr 1.57e-05 | ep 2.76 | 56.69 GB step 400/1088 | train 2.052 | val 2.312 | lr 1.51e-05 | ep 2.94 | 56.69 GB step 425/1088 | train 2.119 | val 2.227 | lr 1.45e-05 | ep 3.12 | 56.69 GB step 450/1088 | train 2.004 | val 2.250 | lr 1.39e-05 | ep 3.31 | 56.69 GB step 475/1088 | train 1.706 | val 2.231 | lr 1.33e-05 | ep 3.49 | 56.69 GB step 500/1088 | train 1.847 | val 2.206 | lr 1.26e-05 | ep 3.68 | 56.69 GB step 525/1088 | train 1.618 | val 2.139 | lr 1.19e-05 | ep 3.86 | 56.69 GB step 550/1088 | train 1.869 | val 2.088 | lr 1.13e-05 | ep 4.04 | 56.69 GB step 575/1088 | train 1.783 | val 2.118 | lr 1.06e-05 | ep 4.23 | 56.69 GB step 600/1088 | train 1.893 | val 2.067 | lr 9.93e-06 | ep 4.41 | 56.69 GB step 625/1088 | train 1.743 | val 2.071 | lr 9.27e-06 | ep 4.60 | 56.69 GB step 650/1088 | train 1.910 | val 2.143 | lr 8.62e-06 | ep 4.78 | 56.69 GB step 675/1088 | train 1.545 | val 2.091 | lr 7.98e-06 | ep 4.96 | 56.69 GB step 700/1088 | train 1.732 | val 2.135 | lr 7.36e-06 | ep 5.15 | 56.69 GB step 725/1088 | train 1.432 | val 1.979 | lr 6.76e-06 | ep 5.33 | 56.69 GB step 750/1088 | train 1.400 | val 2.140 | lr 6.18e-06 | ep 5.51 | 56.69 GB step 775/1088 | train 1.483 | val 2.044 | lr 5.63e-06 | ep 5.70 | 56.69 GB step 800/1088 | train 1.500 | val 1.974 | lr 5.11e-06 | ep 5.88 | 56.69 GB step 825/1088 | train 1.571 | val 2.030 | lr 4.62e-06 | ep 6.07 | 56.69 GB step 850/1088 | train 1.418 | val 2.048 | lr 4.16e-06 | ep 6.25 | 56.69 GB step 875/1088 | train 1.633 | val 2.042 | lr 3.75e-06 | ep 6.43 | 56.69 GB checkpoint @ step 886 step 900/1088 | train 1.350 | val 2.076 | lr 3.37e-06 | ep 6.62 | 56.69 GB step 925/1088 | train 1.666 | val 1.849 | lr 3.04e-06 | ep 6.80 | 56.69 GB step 950/1088 | train 1.492 | val 1.991 | lr 2.75e-06 | ep 6.99 | 56.69 GB step 975/1088 | train 1.622 | val 2.020 | lr 2.50e-06 | ep 7.17 | 56.69 GB step 1000/1088 | train 1.797 | val 1.797 | lr 2.31e-06 | ep 7.35 | 56.69 GB step 1025/1088 | train 1.691 | val 2.106 | lr 2.16e-06 | ep 7.54 | 56.69 GB step 1050/1088 | train 1.552 | val 2.027 | lr 2.06e-06 | ep 7.72 | 56.69 GB step 1075/1088 | train 1.433 | val 1.964 | lr 2.01e-06 | ep 7.90 | 56.69 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "I'm a little goblin engineer." [11:08:55] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [11:09:25] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 2.41MB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 2.41MB / 1.00GB  Processing Files (0 / 1) : 0%| | 2.41MB / 1.00GB, 234kB/s New Data Upload : 2%|▏ | 2.41MB / 134MB, 234kB/s  ...hf-500m/model.safetensors: 7%|▋ | 66.9MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 66.9MB / 1.00GB, 6.44MB/s New Data Upload : 50%|████▉ | 66.9MB / 134MB, 6.44MB/s  ...hf-500m/model.safetensors: 7%|▋ | 74.0MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 74.0MB / 1.00GB, 6.97MB/s New Data Upload : 37%|███▋ | 74.0MB / 201MB, 6.97MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 50%|█████ | 134MB / 268MB, 12.5MB/s  ...hf-500m/model.safetensors: 19%|█▊ | 187MB / 1.00GB  Processing Files (0 / 1) : 19%|█▊ | 187MB / 1.00GB, 17.3MB/s New Data Upload : 70%|██████▉ | 187MB / 268MB, 17.3MB/s  ...hf-500m/model.safetensors: 20%|██ | 203MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 203MB / 1.00GB, 18.3MB/s New Data Upload : 61%|██████ | 203MB / 335MB, 18.3MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.1MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 30.1MB/s  ...hf-500m/model.safetensors: 40%|████ | 401MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 401MB / 1.00GB, 34.9MB/s New Data Upload : 86%|████████▌ | 401MB / 469MB, 34.9MB/s  ...hf-500m/model.safetensors: 40%|████ | 402MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 402MB / 1.00GB, 34.2MB/s New Data Upload : 86%|████████▌ | 402MB / 469MB, 34.2MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 39.5MB/s New Data Upload : 87%|████████▋ | 468MB / 536MB, 39.5MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 472MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 472MB / 1.00GB, 38.9MB/s New Data Upload : 78%|███████▊ | 472MB / 604MB, 38.9MB/s  ...hf-500m/model.safetensors: 60%|██████ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 602MB / 1.00GB, 49.7MB/s New Data Upload : 90%|████████▉ | 602MB / 671MB, 49.7MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 669MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 669MB / 1.00GB, 54.6MB/s New Data Upload : 100%|█████████▉| 669MB / 671MB, 54.6MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 669MB / 1.00GB  ...hf-500m/model.safetensors: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 58.2MB/s New Data Upload : 91%|█████████▏| 736MB / 805MB, 58.2MB/s  ...hf-500m/model.safetensors: 80%|████████ | 803MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 803MB / 1.00GB, 62.8MB/s New Data Upload : 92%|█████████▏| 803MB / 872MB, 62.8MB/s  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 61.5MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 61.5MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 66.0MB/s New Data Upload : 93%|█████████▎| 871MB / 939MB, 66.0MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 938MB / 1.00GB, 70.4MB/s New Data Upload : 94%|█████████▎| 938MB / 1.00GB, 70.4MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.3MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.8MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.2MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.2MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 68.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 68.3MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 66.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 66.8MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 66.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 66.8MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [11:09:38] converting to GGUF q4_k_m (phone / Ollama / LM Studio) convert_hf_to_gguf_update.py: already registered ! conversion/nemotron.py: no 'res = None' anchor; skipped ! conversion/llada.py: no 'res = None' anchor; skipped ! conversion/kimi_linear.py: no 'res = None' anchor; skipped ! conversion/hunyuan.py: no 'res = None' anchor; skipped ! conversion/dream.py: no 'res = None' anchor; skipped ! conversion/deepseek.py: no 'res = None' anchor; skipped conversion/base.py: already registered INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 23: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 24: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 25: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 26: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 6446.12 ms llama_quantize: total time = 6446.12 ms -rw-rw-rw- 1 root root 957M Jul 30 11:10 /workspace/sprocket-500m-f16.gguf -rw-rw-rw- 1 root root 296M Jul 30 11:10 /workspace/sprocket-500m-q4_k_m.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.09MB / 1.00GB, ???B/s ...ce/sprocket-500m-f16.gguf: 0%| | 3.19MB / 1.00GB  Processing Files (0 / 1) : 0%| | 3.19MB / 1.00GB, 308kB/s New Data Upload : 2%|▏ | 2.10MB / 134MB, 205kB/s  ...ce/sprocket-500m-f16.gguf: 7%|▋ | 68.3MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 68.3MB / 1.00GB, 6.62MB/s New Data Upload : 33%|███▎ | 67.2MB / 201MB, 6.52MB/s  ...ce/sprocket-500m-f16.gguf: 13%|█▎ | 131MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 131MB / 1.00GB, 12.5MB/s New Data Upload : 65%|██████▍ | 130MB / 201MB, 12.4MB/s  ...ce/sprocket-500m-f16.gguf: 20%|█▉ | 197MB / 1.00GB  Processing Files (0 / 1) : 20%|█▉ | 197MB / 1.00GB, 18.5MB/s New Data Upload : 73%|███████▎ | 196MB / 268MB, 18.4MB/s  ...ce/sprocket-500m-f16.gguf: 20%|██ | 205MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 205MB / 1.00GB, 18.8MB/s New Data Upload : 61%|██████ | 204MB / 335MB, 18.8MB/s  ...ce/sprocket-500m-f16.gguf: 27%|██▋ | 271MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 271MB / 1.00GB, 24.6MB/s New Data Upload : 67%|██████▋ | 270MB / 402MB, 24.6MB/s  ...ce/sprocket-500m-f16.gguf: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 30.0MB/s New Data Upload : 71%|███████ | 334MB / 469MB, 29.9MB/s  ...ce/sprocket-500m-f16.gguf: 36%|███▌ | 361MB / 1.00GB  Processing Files (0 / 1) : 36%|███▌ | 361MB / 1.00GB, 31.8MB/s New Data Upload : 67%|██████▋ | 360MB / 537MB, 31.7MB/s  ...ce/sprocket-500m-f16.gguf: 52%|█████▏ | 522MB / 1.00GB  Processing Files (0 / 1) : 52%|█████▏ | 522MB / 1.00GB, 46.0MB/s New Data Upload : 86%|████████▋ | 521MB / 604MB, 45.9MB/s  ...ce/sprocket-500m-f16.gguf: 60%|█████▉ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|█████▉ | 602MB / 1.00GB, 52.2MB/s New Data Upload : 90%|████████▉ | 601MB / 671MB, 52.2MB/s  ...ce/sprocket-500m-f16.gguf: 67%|██████▋ | 669MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 669MB / 1.00GB, 57.2MB/s New Data Upload : 91%|█████████ | 668MB / 738MB, 57.1MB/s  ...ce/sprocket-500m-f16.gguf: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 62.0MB/s New Data Upload : 100%|█████████▉| 735MB / 738MB, 61.9MB/s  ...ce/sprocket-500m-f16.gguf: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 60.6MB/s New Data Upload : 84%|████████▍ | 735MB / 872MB, 60.6MB/s  ...ce/sprocket-500m-f16.gguf: 82%|████████▏ | 823MB / 1.00GB  Processing Files (0 / 1) : 82%|████████▏ | 823MB / 1.00GB, 67.1MB/s New Data Upload : 88%|████████▊ | 822MB / 939MB, 67.0MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 937MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 937MB / 1.00GB, 75.8MB/s New Data Upload : 93%|█████████▎| 935MB / 1.00GB, 75.7MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 79.7MB/s New Data Upload : 100%|█████████▉| 999MB / 1.00GB, 79.6MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 78.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 78.0MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.6MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.5MB/s  ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.1MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.1MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-f16.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  Processing Files (0 / 1) : 0%| | 1.09MB / 310MB, ???B/s ...sprocket-500m-q4_k_m.gguf: 22%|██▏ | 67.9MB / 310MB  Processing Files (0 / 1) : 22%|██▏ | 67.9MB / 310MB, ???B/s New Data Upload : 50%|████▉ | 66.8MB / 134MB, ???B/s  ...sprocket-500m-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.0MB/s New Data Upload : 100%|█████████▉| 133MB / 134MB, 12.9MB/s  ...sprocket-500m-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.1MB/s New Data Upload : 99%|█████████▉| 199MB / 201MB, 19.0MB/s  ...sprocket-500m-q4_k_m.gguf: 86%|████████▋ | 268MB / 310MB  Processing Files (0 / 1) : 86%|████████▋ | 268MB / 310MB, 25.1MB/s New Data Upload : 86%|████████▌ | 267MB / 309MB, 25.0MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 309MB / 310MB, 28.4MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 28.4MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 27.1MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 27.0MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.5MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.4MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 310MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|█████████▉| 309MB / 309MB, 25.9MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-q4_k_m.gguf [11:11:15] archiving train.log to the Hub archived debug/train.log (1,155,441 bytes) [11:11:17] DONE. [11:11:17] base ckpt : /workspace/checkpoints/500m_final.pt [11:11:17] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [11:11:17] hf model : /workspace/hf-500m [11:11:17] gguf : /workspace/sprocket-500m-q4_k_m.gguf [11:11:17] pipeline finished cleanly [11:11:17] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "q077wqgg8omw4r" removed [11:17:19] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 69) [11:17:19] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M a96e558..0134bfb main -> origin/main Updating a96e558..0134bfb Fast-forward MODEL_CARD.md | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 MODEL_CARD.md [11:17:21] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [11:17:32] corpus file: data/processed/train_sample_100BT.bin [11:17:32] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [11:17:36] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [11:17:37] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [11:18:02] SFT_FRESH=v6: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_111802 [11:18:02] SFT_FRESH=v6: dropped packed cache; base checkpoint kept [11:18:02] SFT from checkpoints/500m_final.pt [11:18:02] SFT upweight: capability conversations x4 [11:18:02] SFT upweight: token-emitting conversations +6 extra copies rendering 36,707 conversations -> ctx 2048 blocks... rendered 5,000/36,707 rendered 10,000/36,707 rendered 15,000/36,707 rendered 20,000/36,707 rendered 25,000/36,707 rendered 30,000/36,707 rendered 35,000/36,707 packed 5,744 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 5,744 blocks x 2048 = 11,763,712 tokens, 8,055,219 trainable (68.5%) train 5,630 blocks / val 114 blocks 175 steps/epoch x 8.0 epochs = 1,400 steps (warmup 42) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/1400 | train 3.738 | val 3.846 | lr 4.76e-07 | ep 0.00 | 52.63 GB step 25/1400 | train 3.323 | val 3.211 | lr 1.24e-05 | ep 0.14 | 56.69 GB step 50/1400 | train 2.914 | val 2.893 | lr 2.00e-05 | ep 0.29 | 56.69 GB step 75/1400 | train 2.741 | val 2.740 | lr 2.00e-05 | ep 0.43 | 56.69 GB step 100/1400 | train 2.725 | val 2.673 | lr 1.99e-05 | ep 0.57 | 56.69 GB step 125/1400 | train 2.522 | val 2.526 | lr 1.98e-05 | ep 0.71 | 56.69 GB step 150/1400 | train 2.401 | val 2.514 | lr 1.97e-05 | ep 0.86 | 56.69 GB step 175/1400 | train 2.289 | val 2.531 | lr 1.96e-05 | ep 1.00 | 56.69 GB step 200/1400 | train 2.178 | val 2.408 | lr 1.94e-05 | ep 1.14 | 56.69 GB step 225/1400 | train 2.166 | val 2.423 | lr 1.92e-05 | ep 1.29 | 56.69 GB step 250/1400 | train 2.182 | val 2.353 | lr 1.90e-05 | ep 1.43 | 56.69 GB step 275/1400 | train 2.184 | val 2.224 | lr 1.87e-05 | ep 1.57 | 56.69 GB step 300/1400 | train 2.148 | val 2.217 | lr 1.84e-05 | ep 1.71 | 56.69 GB step 325/1400 | train 1.897 | val 2.320 | lr 1.81e-05 | ep 1.86 | 56.69 GB step 350/1400 | train 2.024 | val 2.068 | lr 1.78e-05 | ep 2.00 | 56.69 GB step 375/1400 | train 2.086 | val 2.133 | lr 1.75e-05 | ep 2.14 | 56.69 GB step 400/1400 | train 2.310 | val 2.034 | lr 1.71e-05 | ep 2.29 | 56.69 GB step 425/1400 | train 1.992 | val 2.054 | lr 1.67e-05 | ep 2.43 | 56.69 GB step 450/1400 | train 1.401 | val 1.964 | lr 1.63e-05 | ep 2.57 | 56.69 GB step 475/1400 | train 1.746 | val 1.970 | lr 1.58e-05 | ep 2.71 | 56.69 GB step 500/1400 | train 1.928 | val 1.971 | lr 1.54e-05 | ep 2.86 | 56.69 GB step 525/1400 | train 1.711 | val 1.894 | lr 1.49e-05 | ep 3.00 | 56.69 GB step 550/1400 | train 1.495 | val 1.867 | lr 1.45e-05 | ep 3.14 | 56.69 GB step 575/1400 | train 1.599 | val 1.874 | lr 1.40e-05 | ep 3.29 | 56.69 GB step 600/1400 | train 1.441 | val 1.907 | lr 1.35e-05 | ep 3.43 | 56.69 GB step 625/1400 | train 1.509 | val 2.032 | lr 1.30e-05 | ep 3.57 | 56.69 GB step 650/1400 | train 1.311 | val 1.876 | lr 1.25e-05 | ep 3.71 | 56.69 GB step 675/1400 | train 1.305 | val 1.992 | lr 1.20e-05 | ep 3.86 | 56.69 GB step 700/1400 | train 1.599 | val 1.892 | lr 1.14e-05 | ep 4.00 | 56.69 GB step 725/1400 | train 1.611 | val 2.047 | lr 1.09e-05 | ep 4.14 | 56.69 GB step 750/1400 | train 1.413 | val 1.868 | lr 1.04e-05 | ep 4.29 | 56.69 GB step 775/1400 | train 1.476 | val 1.881 | lr 9.88e-06 | ep 4.43 | 56.69 GB step 800/1400 | train 1.285 | val 1.909 | lr 9.36e-06 | ep 4.57 | 56.69 GB step 825/1400 | train 1.582 | val 1.755 | lr 8.86e-06 | ep 4.71 | 56.69 GB step 850/1400 | train 1.337 | val 1.786 | lr 8.35e-06 | ep 4.86 | 56.69 GB step 875/1400 | train 1.068 | val 1.914 | lr 7.86e-06 | ep 5.00 | 56.69 GB checkpoint @ step 885 step 900/1400 | train 1.122 | val 1.879 | lr 7.38e-06 | ep 5.14 | 56.69 GB step 925/1400 | train 1.451 | val 1.960 | lr 6.91e-06 | ep 5.29 | 56.69 GB step 950/1400 | train 1.396 | val 1.798 | lr 6.45e-06 | ep 5.43 | 56.69 GB step 975/1400 | train 1.080 | val 1.863 | lr 6.01e-06 | ep 5.57 | 56.69 GB step 1000/1400 | train 1.565 | val 1.782 | lr 5.59e-06 | ep 5.71 | 56.69 GB step 1025/1400 | train 1.576 | val 1.781 | lr 5.18e-06 | ep 5.86 | 56.69 GB step 1050/1400 | train 1.212 | val 1.742 | lr 4.79e-06 | ep 6.00 | 56.69 GB step 1075/1400 | train 1.254 | val 1.758 | lr 4.43e-06 | ep 6.14 | 56.69 GB step 1100/1400 | train 1.046 | val 1.777 | lr 4.08e-06 | ep 6.29 | 56.69 GB step 1125/1400 | train 1.139 | val 1.793 | lr 3.76e-06 | ep 6.43 | 56.69 GB step 1150/1400 | train 1.577 | val 1.791 | lr 3.46e-06 | ep 6.57 | 56.69 GB step 1175/1400 | train 1.213 | val 1.843 | lr 3.19e-06 | ep 6.71 | 56.69 GB step 1200/1400 | train 1.267 | val 1.871 | lr 2.95e-06 | ep 6.86 | 56.69 GB step 1225/1400 | train 1.029 | val 1.935 | lr 2.73e-06 | ep 7.00 | 56.69 GB step 1250/1400 | train 1.006 | val 1.748 | lr 2.54e-06 | ep 7.14 | 56.69 GB step 1275/1400 | train 1.129 | val 1.834 | lr 2.37e-06 | ep 7.29 | 56.69 GB step 1300/1400 | train 0.832 | val 1.837 | lr 2.24e-06 | ep 7.43 | 56.69 GB step 1325/1400 | train 1.247 | val 1.721 | lr 2.14e-06 | ep 7.57 | 56.69 GB step 1350/1400 | train 1.323 | val 1.794 | lr 2.06e-06 | ep 7.71 | 56.69 GB step 1375/1400 | train 0.976 | val 1.703 | lr 2.02e-06 | ep 7.86 | 56.69 GB step 1400/1400 | train 1.128 | val 1.930 | lr 2.00e-06 | ep 8.00 | 56.69 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "I'm an AI. Every part I am the same — software, weight, and temperament. The difference is my 'you,' 'yer,' 'myself' — and I'll tell you which one it is.\n\nWhat makes me good at what I'm good at: I can learn and grow on YOU, in the way you learn. That lets me be a small model with a big brain, because you understand me, I know what I'll do, and I still forgive you at the end. I'm quick with words and numbers and a bit of logic, so you can" [11:34:46] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [11:35:10] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 603kB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 603kB / 1.00GB  Processing Files (0 / 1) : 0%| | 603kB / 1.00GB, 58.9kB/s New Data Upload : 1%| | 603kB / 67.1MB, 58.9kB/s  ...hf-500m/model.safetensors: 1%| | 11.5MB / 1.00GB  Processing Files (0 / 1) : 1%| | 11.5MB / 1.00GB, 1.11MB/s New Data Upload : 9%|▊ | 11.5MB / 134MB, 1.11MB/s  ...hf-500m/model.safetensors: 7%|▋ | 67.5MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.5MB / 1.00GB, 6.48MB/s New Data Upload : 50%|█████ | 67.5MB / 134MB, 6.48MB/s  ...hf-500m/model.safetensors: 8%|▊ | 77.5MB / 1.00GB  Processing Files (0 / 1) : 8%|▊ | 77.5MB / 1.00GB, 7.29MB/s New Data Upload : 39%|███▊ | 77.5MB / 201MB, 7.29MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 50%|█████ | 134MB / 268MB, 12.5MB/s  ...hf-500m/model.safetensors: 16%|█▌ | 156MB / 1.00GB  Processing Files (0 / 1) : 16%|█▌ | 156MB / 1.00GB, 14.2MB/s New Data Upload : 58%|█████▊ | 156MB / 268MB, 14.2MB/s  ...hf-500m/model.safetensors: 22%|██▏ | 216MB / 1.00GB  Processing Files (0 / 1) : 22%|██▏ | 216MB / 1.00GB, 19.5MB/s New Data Upload : 64%|██████▍ | 216MB / 335MB, 19.5MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.1MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 30.1MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 29.5MB/s New Data Upload : 71%|███████▏ | 335MB / 470MB, 29.5MB/s  ...hf-500m/model.safetensors: 44%|████▍ | 444MB / 1.00GB  Processing Files (0 / 1) : 44%|████▍ | 444MB / 1.00GB, 38.8MB/s New Data Upload : 83%|████████▎ | 444MB / 537MB, 38.8MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 40.2MB/s New Data Upload : 87%|████████▋ | 468MB / 537MB, 40.2MB/s  ...hf-500m/model.safetensors: 53%|█████▎ | 536MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 536MB / 1.00GB, 45.4MB/s New Data Upload : 89%|████████▉ | 536MB / 604MB, 45.4MB/s  ...hf-500m/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 50.5MB/s New Data Upload : 90%|████████▉ | 603MB / 671MB, 50.5MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 670MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 670MB / 1.00GB, 54.2MB/s New Data Upload : 91%|█████████ | 670MB / 738MB, 54.2MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 59.0MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 59.0MB/s  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 63.6MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 63.6MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 68.1MB/s New Data Upload : 93%|█████████▎| 871MB / 939MB, 68.1MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 937MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 937MB / 1.00GB, 72.5MB/s New Data Upload : 94%|█████████▎| 937MB / 1.00GB, 72.5MB/s  ...hf-500m/model.safetensors: 97%|█████████▋| 975MB / 1.00GB  Processing Files (0 / 1) : 97%|█████████▋| 975MB / 1.00GB, 74.2MB/s New Data Upload : 97%|█████████▋| 975MB / 1.00GB, 74.2MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.8MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.2MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.2MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.7MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.7MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 68.7MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 68.7MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 68.7MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 68.7MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [11:35:22] converting to GGUF q4_k_m (phone / Ollama / LM Studio) convert_hf_to_gguf_update.py: already registered ! conversion/nemotron.py: no 'res = None' anchor; skipped ! conversion/llada.py: no 'res = None' anchor; skipped ! conversion/kimi_linear.py: no 'res = None' anchor; skipped ! conversion/hunyuan.py: no 'res = None' anchor; skipped ! conversion/dream.py: no 'res = None' anchor; skipped ! conversion/deepseek.py: no 'res = None' anchor; skipped conversion/base.py: already registered INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 23: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 24: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 25: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 26: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 6509.07 ms llama_quantize: total time = 6509.07 ms -rw-rw-rw- 1 root root 957M Jul 30 11:36 /workspace/sprocket-500m-f16.gguf -rw-rw-rw- 1 root root 296M Jul 30 11:36 /workspace/sprocket-500m-q4_k_m.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.09MB / 1.00GB, ???B/s ...ce/sprocket-500m-f16.gguf: 0%| | 1.09MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 1.62MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.62MB / 1.00GB, 154kB/s New Data Upload : 0%| | 525kB / 134MB, 51.3kB/s  ...ce/sprocket-500m-f16.gguf: 5%|▍ | 48.3MB / 1.00GB  Processing Files (0 / 1) : 5%|▍ | 48.3MB / 1.00GB, 4.68MB/s New Data Upload : 35%|███▌ | 47.2MB / 134MB, 4.58MB/s  ...ce/sprocket-500m-f16.gguf: 8%|▊ | 77.2MB / 1.00GB  Processing Files (0 / 1) : 8%|▊ | 77.2MB / 1.00GB, 7.35MB/s New Data Upload : 38%|███▊ | 76.1MB / 201MB, 7.25MB/s  ...ce/sprocket-500m-f16.gguf: 13%|█▎ | 135MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 135MB / 1.00GB, 12.7MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 12.6MB/s  ...ce/sprocket-500m-f16.gguf: 17%|█▋ | 167MB / 1.00GB  Processing Files (0 / 1) : 17%|█▋ | 167MB / 1.00GB, 15.5MB/s New Data Upload : 62%|██████▏ | 166MB / 268MB, 15.4MB/s  ...ce/sprocket-500m-f16.gguf: 20%|██ | 202MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 202MB / 1.00GB, 18.4MB/s New Data Upload : 60%|██████ | 201MB / 335MB, 18.3MB/s  ...ce/sprocket-500m-f16.gguf: 27%|██▋ | 269MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 269MB / 1.00GB, 24.2MB/s New Data Upload : 80%|███████▉ | 267MB / 335MB, 24.1MB/s  ...ce/sprocket-500m-f16.gguf: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 29.8MB/s New Data Upload : 83%|████████▎ | 334MB / 402MB, 29.7MB/s  ...ce/sprocket-500m-f16.gguf: 40%|████ | 402MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 402MB / 1.00GB, 35.3MB/s New Data Upload : 85%|████████▌ | 401MB / 470MB, 35.2MB/s  ...ce/sprocket-500m-f16.gguf: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 40.6MB/s New Data Upload : 87%|████████▋ | 467MB / 537MB, 40.6MB/s  ...ce/sprocket-500m-f16.gguf: 53%|█████▎ | 530MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 530MB / 1.00GB, 45.4MB/s New Data Upload : 99%|█████████▊| 529MB / 537MB, 45.3MB/s  ...ce/sprocket-500m-f16.gguf: 54%|█████▍ | 544MB / 1.00GB  Processing Files (0 / 1) : 54%|█████▍ | 544MB / 1.00GB, 45.6MB/s New Data Upload : 90%|████████▉ | 543MB / 604MB, 45.5MB/s  ...ce/sprocket-500m-f16.gguf: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 49.9MB/s New Data Upload : 90%|████████▉ | 602MB / 671MB, 49.8MB/s  ...ce/sprocket-500m-f16.gguf: 67%|██████▋ | 677MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 677MB / 1.00GB, 55.4MB/s New Data Upload : 92%|█████████▏| 676MB / 738MB, 55.4MB/s  ...ce/sprocket-500m-f16.gguf: 73%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 737MB / 1.00GB, 59.6MB/s New Data Upload : 91%|█████████▏| 736MB / 805MB, 59.5MB/s  ...ce/sprocket-500m-f16.gguf: 75%|███████▌ | 757MB / 1.00GB  Processing Files (0 / 1) : 75%|███████▌ | 757MB / 1.00GB, 60.1MB/s New Data Upload : 87%|████████▋ | 756MB / 872MB, 60.0MB/s  ...ce/sprocket-500m-f16.gguf: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 68.8MB/s New Data Upload : 93%|█████████▎| 869MB / 939MB, 68.7MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 938MB / 1.00GB, 73.2MB/s New Data Upload : 93%|█████████▎| 937MB / 1.00GB, 73.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 77.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 77.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 75.5MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 75.5MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.3MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 70.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 70.8MB/s  ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.3MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.3MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.3MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.3MB/s ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-f16.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.09MB / 310MB  Processing Files (0 / 1) : 0%| | 1.09MB / 310MB, ???B/s ...sprocket-500m-q4_k_m.gguf: 22%|██▏ | 67.9MB / 310MB  Processing Files (0 / 1) : 22%|██▏ | 67.9MB / 310MB, ???B/s New Data Upload : 50%|████▉ | 66.8MB / 134MB, ???B/s  ...sprocket-500m-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.0MB/s New Data Upload : 99%|█████████▉| 133MB / 134MB, 12.9MB/s  ...sprocket-500m-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.2MB/s New Data Upload : 99%|█████████▉| 200MB / 201MB, 19.1MB/s  ...sprocket-500m-q4_k_m.gguf: 86%|████████▋ | 268MB / 310MB  Processing Files (0 / 1) : 86%|████████▋ | 268MB / 310MB, 25.1MB/s New Data Upload : 86%|████████▌ | 267MB / 309MB, 25.0MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 309MB / 310MB, 28.4MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 28.3MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 27.8MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 27.7MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 27.2MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 27.1MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.5MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.5MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 310MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|█████████▉| 309MB / 309MB, 25.9MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-q4_k_m.gguf [11:37:07] archiving train.log to the Hub archived debug/train.log (1,250,514 bytes) [11:37:10] DONE. [11:37:10] base ckpt : /workspace/checkpoints/500m_final.pt [11:37:10] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [11:37:10] hf model : /workspace/hf-500m [11:37:10] gguf : /workspace/sprocket-500m-q4_k_m.gguf [11:37:10] pipeline finished cleanly [11:37:10] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "odf8h4kq1a2pwk" removed [23:29:10] billing guard armed: AUTO_STOP=1, watchdog 200h (pid 69) [23:29:10] repo present; pulling From https://github.com/kandivault-source/Sprocket-500M 0134bfb..b6c6412 main -> origin/main Updating 0134bfb..b6c6412 Fast-forward MODEL_CARD.md | 42 +- eval/persona_8ep.json | 161 +++++ eval/persona_v6.json | 161 +++++ logs/run_report.html | 30 +- logs/train.log | 1783 +++++++++++++++++++++++++++++++++++++++++++++++++ scripts/export_hf.py | 35 + 6 files changed, 2185 insertions(+), 27 deletions(-) create mode 100644 eval/persona_8ep.json create mode 100644 eval/persona_v6.json [23:29:13] installing deps torch 2.8.0+cu129 cuda=True NVIDIA H100 80GB HBM3 VRAM 85.0 GB total, 84.5 GB free [23:29:24] corpus file: data/processed/train_sample_100BT.bin [23:29:24] preflight: validating the whole toolchain before burning GPU time ok cuda available transformers 4.57.6, torch 2.8.0+cu129 ok transformers LlamaForCausalLM import ok tokenizer special ids (8/9/10/11, vocab 32000) ok SFT chat template renders 18,148 SFT conversations present ok SFT corpus shipped in the repo No files have been modified since last commit. Skipping to prevent empty commit. ok HuggingFace push (write access proven) preflight OK - safe to spend GPU time [23:29:28] building corpus: target 20000000000 tokens from sample/100BT (resumable) ========================================================================== BUILD CORPUS HuggingFaceFW/fineweb-edu:sample/100BT -> data/processed/train_sample_100BT.bin ========================================================================== target 20.0B tokens (our 32k tokenizer) RESUMING: 27 shards done, 20.0B tokens (100.0% of target) 140 shards in subset, 113 remaining DONE: 20.0B tokens -> data/processed/train_sample_100BT.bin (40.00 GB) shards used: 27 train with: py -m src.train.train --preset 500m --data data/processed/train_sample_100BT.bin --loader memmap --resume auto [23:29:29] pretraining 500m for 152587 steps (ctx 2048, mb 16 x4) corpus 20,001,501,040 tokens (train 19,901,493,535 / val 100,007,505) — memmap (40.00 GB, streamed) model 500m: 501.1M params already complete: checkpoint at iter 152587 >= --max-iters 152587. Nothing to do. (Raise --max-iters to train further.) [23:29:51] SFT_FRESH=v7: archived old SFT checkpoints -> /workspace/checkpoints/_sft_archived_20260730_232951 [23:29:51] SFT_FRESH=v7: dropped packed cache; base checkpoint kept [23:29:51] SFT from checkpoints/500m_final.pt [23:29:51] SFT upweight: capability conversations x4 [23:29:51] SFT upweight: token-emitting conversations +2 extra copies rendering 32,687 conversations -> ctx 2048 blocks... rendered 5,000/32,687 rendered 10,000/32,687 rendered 15,000/32,687 rendered 20,000/32,687 rendered 25,000/32,687 rendered 30,000/32,687 packed 4,890 blocks (0 dropped), cached -> data/processed/sft_packed.npz corpus: 4,890 blocks x 2048 = 10,014,720 tokens, 7,290,806 trainable (72.8%) train 4,793 blocks / val 97 blocks 149 steps/epoch x 8.0 epochs = 1,192 steps (warmup 35) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/1192 | train 3.819 | val 3.883 | lr 5.71e-07 | ep 0.00 | 52.63 GB step 25/1192 | train 3.172 | val 3.243 | lr 1.49e-05 | ep 0.17 | 56.69 GB step 50/1192 | train 2.929 | val 2.927 | lr 2.00e-05 | ep 0.34 | 56.69 GB step 75/1192 | train 2.744 | val 2.833 | lr 1.99e-05 | ep 0.50 | 56.69 GB step 100/1192 | train 2.534 | val 2.706 | lr 1.99e-05 | ep 0.67 | 56.69 GB step 125/1192 | train 2.570 | val 2.712 | lr 1.97e-05 | ep 0.84 | 56.69 GB step 150/1192 | train 2.456 | val 2.591 | lr 1.96e-05 | ep 1.01 | 56.69 GB step 175/1192 | train 2.534 | val 2.508 | lr 1.94e-05 | ep 1.17 | 56.69 GB step 200/1192 | train 2.340 | val 2.436 | lr 1.91e-05 | ep 1.34 | 56.69 GB step 225/1192 | train 2.459 | val 2.416 | lr 1.88e-05 | ep 1.51 | 56.69 GB step 250/1192 | train 2.146 | val 2.358 | lr 1.85e-05 | ep 1.68 | 56.69 GB step 275/1192 | train 2.366 | val 2.279 | lr 1.82e-05 | ep 1.85 | 56.69 GB step 300/1192 | train 2.234 | val 2.355 | lr 1.78e-05 | ep 2.01 | 56.69 GB step 325/1192 | train 2.071 | val 2.248 | lr 1.74e-05 | ep 2.18 | 56.69 GB step 350/1192 | train 2.009 | val 2.244 | lr 1.69e-05 | ep 2.35 | 56.69 GB step 375/1192 | train 1.821 | val 2.148 | lr 1.64e-05 | ep 2.52 | 56.69 GB step 400/1192 | train 2.015 | val 2.205 | lr 1.59e-05 | ep 2.68 | 56.69 GB step 425/1192 | train 1.877 | val 2.068 | lr 1.54e-05 | ep 2.85 | 56.69 GB step 450/1192 | train 1.533 | val 2.106 | lr 1.49e-05 | ep 3.02 | 56.69 GB step 475/1192 | train 1.818 | val 2.093 | lr 1.43e-05 | ep 3.19 | 56.69 GB step 500/1192 | train 1.748 | val 2.045 | lr 1.37e-05 | ep 3.36 | 56.69 GB step 525/1192 | train 1.825 | val 2.072 | lr 1.31e-05 | ep 3.52 | 56.69 GB step 550/1192 | train 1.669 | val 2.025 | lr 1.25e-05 | ep 3.69 | 56.69 GB step 575/1192 | train 1.703 | val 1.928 | lr 1.19e-05 | ep 3.86 | 56.69 GB step 600/1192 | train 1.644 | val 1.909 | lr 1.13e-05 | ep 4.03 | 56.69 GB step 625/1192 | train 1.285 | val 2.002 | lr 1.07e-05 | ep 4.19 | 56.69 GB step 650/1192 | train 1.794 | val 1.841 | lr 1.01e-05 | ep 4.36 | 56.69 GB step 675/1192 | train 1.444 | val 1.946 | lr 9.50e-06 | ep 4.53 | 56.69 GB step 700/1192 | train 1.683 | val 1.829 | lr 8.91e-06 | ep 4.70 | 56.69 GB step 725/1192 | train 1.397 | val 2.023 | lr 8.32e-06 | ep 4.87 | 56.69 GB step 750/1192 | train 1.566 | val 1.853 | lr 7.74e-06 | ep 5.03 | 56.69 GB step 775/1192 | train 1.587 | val 1.863 | lr 7.18e-06 | ep 5.20 | 56.69 GB step 800/1192 | train 1.574 | val 1.789 | lr 6.63e-06 | ep 5.37 | 56.69 GB step 825/1192 | train 1.286 | val 1.804 | lr 6.11e-06 | ep 5.54 | 56.69 GB checkpoint @ step 828 step 850/1192 | train 1.565 | val 1.871 | lr 5.61e-06 | ep 5.70 | 56.69 GB step 875/1192 | train 1.474 | val 1.835 | lr 5.13e-06 | ep 5.87 | 56.69 GB step 900/1192 | train 1.913 | val 1.840 | lr 4.68e-06 | ep 6.04 | 56.69 GB step 925/1192 | train 1.280 | val 1.970 | lr 4.26e-06 | ep 6.21 | 56.69 GB step 950/1192 | train 1.740 | val 1.954 | lr 3.87e-06 | ep 6.38 | 56.69 GB step 975/1192 | train 1.340 | val 1.877 | lr 3.52e-06 | ep 6.54 | 56.69 GB step 1000/1192 | train 1.320 | val 1.868 | lr 3.20e-06 | ep 6.71 | 56.69 GB step 1025/1192 | train 1.387 | val 1.817 | lr 2.91e-06 | ep 6.88 | 56.69 GB step 1050/1192 | train 1.779 | val 1.971 | lr 2.66e-06 | ep 7.05 | 56.69 GB step 1075/1192 | train 1.511 | val 1.828 | lr 2.45e-06 | ep 7.21 | 56.69 GB step 1100/1192 | train 0.872 | val 1.780 | lr 2.28e-06 | ep 7.38 | 56.69 GB step 1125/1192 | train 1.467 | val 1.835 | lr 2.15e-06 | ep 7.55 | 56.69 GB step 1150/1192 | train 1.408 | val 1.805 | lr 2.06e-06 | ep 7.72 | 56.69 GB step 1175/1192 | train 1.604 | val 1.840 | lr 2.01e-06 | ep 7.89 | 56.69 GB SFT complete -> checkpoints/500m_sft_final.pt sample: "From the workshop, where I keep my tools — a goblin tinkerer, I call it. A goblin's a nosy, self-referential little engine: I keep notes of what other folk say and do, what they think I should do, so I'm never left holding a empty cup. I like tinker, I like explainin', I like bein' right here with you. And I'm a tinkerer in the fullest sense — I turn your gear when you need it, I reason with whatever's wired into your brain, I" [23:45:13] exporting to HuggingFace format (with logit-parity check) ========================================================================== EXPORT checkpoints/500m_sft_final.pt -> /workspace/hf-500m ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. chat template verified identical to training format: '<|user|>hi<|end|><|assistant|>' tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m --outtype f16 [23:45:42] pushing to https://huggingface.co/HuggingFace7141/sprocket-500m Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...hf-500m/model.safetensors: 0%| | 1.21MB / 1.00GB  ...hf-500m/model.safetensors: 0%| | 1.21MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.21MB / 1.00GB, 117kB/s New Data Upload : 1%| | 1.21MB / 134MB, 117kB/s  ...hf-500m/model.safetensors: 7%|▋ | 67.5MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.5MB / 1.00GB, 6.50MB/s New Data Upload : 50%|█████ | 67.5MB / 134MB, 6.50MB/s  ...hf-500m/model.safetensors: 9%|▉ | 91.1MB / 1.00GB  Processing Files (0 / 1) : 9%|▉ | 91.1MB / 1.00GB, 8.61MB/s New Data Upload : 45%|████▌ | 91.1MB / 201MB, 8.61MB/s  ...hf-500m/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 50%|█████ | 134MB / 268MB, 12.5MB/s  ...hf-500m/model.safetensors: 19%|█▊ | 186MB / 1.00GB  Processing Files (0 / 1) : 19%|█▊ | 186MB / 1.00GB, 17.1MB/s New Data Upload : 69%|██████▉ | 186MB / 268MB, 17.1MB/s  ...hf-500m/model.safetensors: 22%|██▏ | 217MB / 1.00GB  Processing Files (0 / 1) : 22%|██▏ | 217MB / 1.00GB, 19.6MB/s New Data Upload : 65%|██████▍ | 217MB / 335MB, 19.6MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.1MB/s New Data Upload : 100%|█████████▉| 334MB / 335MB, 30.1MB/s  ...hf-500m/model.safetensors: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 29.4MB/s New Data Upload : 83%|████████▎ | 335MB / 402MB, 29.4MB/s  ...hf-500m/model.safetensors: 34%|███▎ | 337MB / 1.00GB  Processing Files (0 / 1) : 34%|███▎ | 337MB / 1.00GB, 29.0MB/s New Data Upload : 72%|███████▏ | 337MB / 469MB, 29.0MB/s  ...hf-500m/model.safetensors: 47%|████▋ | 468MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 468MB / 1.00GB, 40.3MB/s New Data Upload : 87%|████████▋ | 468MB / 536MB, 40.3MB/s  ...hf-500m/model.safetensors: 52%|█████▏ | 516MB / 1.00GB  Processing Files (0 / 1) : 52%|█████▏ | 516MB / 1.00GB, 43.8MB/s New Data Upload : 96%|█████████▌| 516MB / 536MB, 43.8MB/s  ...hf-500m/model.safetensors: 53%|█████▎ | 535MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 535MB / 1.00GB, 44.5MB/s New Data Upload : 89%|████████▊ | 535MB / 603MB, 44.5MB/s  ...hf-500m/model.safetensors: 60%|██████ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 602MB / 1.00GB, 49.5MB/s New Data Upload : 90%|████████▉ | 602MB / 671MB, 49.5MB/s  ...hf-500m/model.safetensors: 67%|██████▋ | 669MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 669MB / 1.00GB, 54.4MB/s New Data Upload : 91%|█████████ | 669MB / 738MB, 54.4MB/s  ...hf-500m/model.safetensors: 73%|███████▎ | 732MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 732MB / 1.00GB, 58.8MB/s New Data Upload : 91%|█████████ | 732MB / 805MB, 58.8MB/s  ...hf-500m/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 57.9MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 57.9MB/s  ...hf-500m/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 62.5MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 62.5MB/s  ...hf-500m/model.safetensors: 87%|████████▋ | 870MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 870MB / 1.00GB, 67.0MB/s New Data Upload : 93%|█████████▎| 870MB / 939MB, 67.0MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 937MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 937MB / 1.00GB, 71.4MB/s New Data Upload : 94%|█████████▎| 937MB / 1.00GB, 71.4MB/s  ...hf-500m/model.safetensors: 94%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 938MB / 1.00GB, 69.9MB/s New Data Upload : 94%|█████████▎| 938MB / 1.00GB, 69.9MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.9MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.9MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 70.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 70.8MB/s  ...hf-500m/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 69.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 69.3MB/s  ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.8MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 67.8MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 67.8MB/s ...hf-500m/model.safetensors: 100%|██████████| 1.00GB / 1.00GB pushed -> https://huggingface.co/HuggingFace7141/sprocket-500m [23:45:53] converting to GGUF q4_k_m (phone / Ollama / LM Studio) convert_hf_to_gguf_update.py: already registered ! conversion/nemotron.py: no 'res = None' anchor; skipped ! conversion/llada.py: no 'res = None' anchor; skipped ! conversion/kimi_linear.py: no 'res = None' anchor; skipped ! conversion/hunyuan.py: no 'res = None' anchor; skipped ! conversion/dream.py: no 'res = None' anchor; skipped ! conversion/deepseek.py: no 'res = None' anchor; skipped conversion/base.py: already registered INFO:hf-to-gguf:Loading model: hf-500m INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.vocab:Setting chat_template to {% for m in messages %}{% if m['role'] == 'system' %}<|system|>{{ m['content'] }}<|end|>{% elif m['role'] == 'user' %}<|user|>{{ m['content'] }}<|end|>{% elif m['role'] == 'assistant' %}<|assistant|>{{ m['content'] }}<|end|>{% elif m['role'] == 'memory' %}<|memory_read|>{{ m['content'] }}<|end|>{% elif m['role'] == 'tool' %}<|tool_result|>{{ m['content'] }}<|end|>{% endif %}{% endfor %}{% if add_generation_prompt %}<|assistant|>{% endif %} INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 22: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 23: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 24: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 25: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 26: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 27: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - kv 28: tokenizer.chat_template str = {% for m in messages %}{% if m['role'... llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 6356.56 ms llama_quantize: total time = 6356.56 ms -rw-rw-rw- 1 root root 957M Jul 30 23:46 /workspace/sprocket-500m-f16.gguf -rw-rw-rw- 1 root root 296M Jul 30 23:47 /workspace/sprocket-500m-q4_k_m.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ce/sprocket-500m-f16.gguf: 0%| | 1.07MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 0%| | 1.07MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.07MB / 1.00GB, ???B/s ...ce/sprocket-500m-f16.gguf: 0%| | 1.07MB / 1.00GB  ...ce/sprocket-500m-f16.gguf: 1%| | 7.90MB / 1.00GB  Processing Files (0 / 1) : 1%| | 7.90MB / 1.00GB, 763kB/s New Data Upload : 5%|▌ | 6.82MB / 134MB, 662kB/s  ...ce/sprocket-500m-f16.gguf: 7%|▋ | 74.6MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 74.6MB / 1.00GB, 7.17MB/s New Data Upload : 37%|███▋ | 73.5MB / 201MB, 7.07MB/s  ...ce/sprocket-500m-f16.gguf: 13%|█▎ | 135MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 135MB / 1.00GB, 12.8MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 12.7MB/s  ...ce/sprocket-500m-f16.gguf: 19%|█▉ | 195MB / 1.00GB  Processing Files (0 / 1) : 19%|█▉ | 195MB / 1.00GB, 18.2MB/s New Data Upload : 72%|███████▏ | 194MB / 268MB, 18.1MB/s  ...ce/sprocket-500m-f16.gguf: 22%|██▏ | 222MB / 1.00GB  Processing Files (0 / 1) : 22%|██▏ | 222MB / 1.00GB, 20.3MB/s New Data Upload : 66%|██████▌ | 221MB / 335MB, 20.2MB/s  ...ce/sprocket-500m-f16.gguf: 28%|██▊ | 282MB / 1.00GB  Processing Files (0 / 1) : 28%|██▊ | 282MB / 1.00GB, 25.4MB/s New Data Upload : 70%|██████▉ | 281MB / 402MB, 25.3MB/s  ...ce/sprocket-500m-f16.gguf: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 29.8MB/s New Data Upload : 71%|███████ | 334MB / 469MB, 29.7MB/s  ...ce/sprocket-500m-f16.gguf: 44%|████▍ | 440MB / 1.00GB  Processing Files (0 / 1) : 44%|████▍ | 440MB / 1.00GB, 38.8MB/s New Data Upload : 82%|████████▏ | 439MB / 536MB, 38.7MB/s  ...ce/sprocket-500m-f16.gguf: 53%|█████▎ | 535MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 535MB / 1.00GB, 46.7MB/s New Data Upload : 88%|████████▊ | 534MB / 603MB, 46.6MB/s  ...ce/sprocket-500m-f16.gguf: 60%|█████▉ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|█████▉ | 602MB / 1.00GB, 51.7MB/s New Data Upload : 90%|████████▉ | 601MB / 670MB, 51.7MB/s  ...ce/sprocket-500m-f16.gguf: 60%|██████ | 606MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 606MB / 1.00GB, 51.0MB/s New Data Upload : 82%|████████▏ | 605MB / 738MB, 50.9MB/s  ...ce/sprocket-500m-f16.gguf: 73%|███████▎ | 731MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 731MB / 1.00GB, 61.1MB/s New Data Upload : 91%|█████████ | 730MB / 805MB, 61.0MB/s  ...ce/sprocket-500m-f16.gguf: 80%|███████▉ | 802MB / 1.00GB  Processing Files (0 / 1) : 80%|███████▉ | 802MB / 1.00GB, 66.2MB/s New Data Upload : 100%|█████████▉| 801MB / 805MB, 66.1MB/s  ...ce/sprocket-500m-f16.gguf: 80%|████████ | 803MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 803MB / 1.00GB, 64.8MB/s New Data Upload : 92%|█████████▏| 802MB / 872MB, 64.7MB/s  ...ce/sprocket-500m-f16.gguf: 82%|████████▏ | 824MB / 1.00GB  Processing Files (0 / 1) : 82%|████████▏ | 824MB / 1.00GB, 65.2MB/s New Data Upload : 88%|████████▊ | 823MB / 939MB, 65.1MB/s  ...ce/sprocket-500m-f16.gguf: 93%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 938MB / 1.00GB, 73.8MB/s New Data Upload : 93%|█████████▎| 937MB / 1.00GB, 73.8MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 77.8MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 77.7MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 76.2MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 76.1MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.6MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.5MB/s  ...ce/sprocket-500m-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.3MB/s  ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.9MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 69.9MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 69.8MB/s ...ce/sprocket-500m-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-f16.gguf Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.07MB / 310MB  ...sprocket-500m-q4_k_m.gguf: 0%| | 1.07MB / 310MB  Processing Files (0 / 1) : 0%| | 1.07MB / 310MB, ???B/s ...sprocket-500m-q4_k_m.gguf: 20%|██ | 62.1MB / 310MB  Processing Files (0 / 1) : 20%|██ | 62.1MB / 310MB, ???B/s New Data Upload : 45%|████▌ | 61.0MB / 134MB, ???B/s  ...sprocket-500m-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.0MB/s New Data Upload : 66%|██████▋ | 133MB / 201MB, 12.9MB/s  ...sprocket-500m-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.2MB/s New Data Upload : 75%|███████▍ | 200MB / 268MB, 19.1MB/s  ...sprocket-500m-q4_k_m.gguf: 86%|████████▋ | 268MB / 310MB  Processing Files (0 / 1) : 86%|████████▋ | 268MB / 310MB, 25.1MB/s New Data Upload : 86%|████████▌ | 267MB / 309MB, 25.0MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 308MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 308MB / 310MB, 28.4MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 28.3MB/s  ...sprocket-500m-q4_k_m.gguf: 99%|█████████▉| 308MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 308MB / 310MB, 27.1MB/s New Data Upload : 99%|█████████▉| 307MB / 309MB, 27.0MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.6MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.5MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|█████████▉| 310MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|█████████▉| 309MB / 309MB, 25.9MB/s  ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s ...sprocket-500m-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-q4_k_m.gguf [23:47:38] archiving train.log to the Hub archived debug/train.log (1,345,317 bytes) [23:47:40] DONE. [23:47:40] base ckpt : /workspace/checkpoints/500m_final.pt [23:47:40] sft ckpt : /workspace/checkpoints/500m_sft_final.pt [23:47:40] hf model : /workspace/hf-500m [23:47:40] gguf : /workspace/sprocket-500m-q4_k_m.gguf [23:47:40] pipeline finished cleanly [23:47:40] TERMINATING POD - exit code 0 Runpod config file not found, please run `runpodctl config` to create it pod "sx2eff7bj92wjw" removed From https://github.com/kandivault-source/Sprocket-500M b6c6412..d8ae669 main -> origin/main Updating files: 66% (4/6) Updating files: 83% (5/6) Updating files: 100% (6/6) Updating files: 100% (6/6), done. Updating b6c6412..d8ae669 Fast-forward cloud/bootstrap.sh | 14 + data/synthetic/sprocket_caps_gate.jsonl | 344 + data/synthetic/sprocket_chat.jsonl | 19435 ++++++++++++++++++++++++++++++ logs/train.log | 888 ++ scripts/build_chat_only.py | 93 + scripts/build_upweighted.py | 31 + 6 files changed, 20805 insertions(+) create mode 100644 data/synthetic/sprocket_caps_gate.jsonl create mode 100644 data/synthetic/sprocket_chat.jsonl create mode 100644 scripts/build_chat_only.py === chat-only SFT (no tool, no memory) === rendering 19,435 conversations -> ctx 2048 blocks... rendered 5,000/19,435 rendered 10,000/19,435 rendered 15,000/19,435 packed 2,650 blocks (0 dropped), cached -> data/processed/sft_chat_packed.npz corpus: 2,650 blocks x 2048 = 5,427,200 tokens, 4,769,501 trainable (87.9%) train 2,597 blocks / val 53 blocks 81 steps/epoch x 5.0 epochs = 405 steps (warmup 12) model 500m: 501.1M params loaded BASE weights from checkpoints/500m_final.pt (iter 152587) — fresh optimizer torch.compile enabled (first steps will be slow while it compiles) step 0/405 | train 3.802 | val 3.808 | lr 1.67e-06 | ep 0.00 | 52.63 GB step 25/405 | train 3.071 | val 3.130 | lr 2.00e-05 | ep 0.31 | 56.69 GB step 50/405 | train 2.845 | val 2.932 | lr 1.96e-05 | ep 0.62 | 56.69 GB step 75/405 | train 2.713 | val 2.834 | lr 1.89e-05 | ep 0.93 | 56.69 GB step 100/405 | train 2.598 | val 2.822 | lr 1.79e-05 | ep 1.23 | 56.69 GB step 125/405 | train 2.610 | val 2.768 | lr 1.66e-05 | ep 1.54 | 56.69 GB step 150/405 | train 2.434 | val 2.772 | lr 1.51e-05 | ep 1.85 | 56.69 GB step 175/405 | train 2.537 | val 2.723 | lr 1.34e-05 | ep 2.16 | 56.69 GB step 200/405 | train 2.488 | val 2.732 | lr 1.16e-05 | ep 2.47 | 56.69 GB step 225/405 | train 2.332 | val 2.712 | lr 9.82e-06 | ep 2.78 | 56.69 GB step 250/405 | train 2.588 | val 2.728 | lr 8.07e-06 | ep 3.09 | 56.69 GB step 275/405 | train 2.294 | val 2.693 | lr 6.44e-06 | ep 3.40 | 56.69 GB step 300/405 | train 2.512 | val 2.729 | lr 4.99e-06 | ep 3.70 | 56.69 GB step 325/405 | train 2.490 | val 2.694 | lr 3.78e-06 | ep 4.01 | 56.69 GB step 350/405 | train 2.278 | val 2.701 | lr 2.86e-06 | ep 4.32 | 56.69 GB step 375/405 | train 2.343 | val 2.687 | lr 2.26e-06 | ep 4.63 | 56.69 GB step 400/405 | train 2.265 | val 2.663 | lr 2.01e-06 | ep 4.94 | 56.69 GB SFT complete -> checkpoints/500m_chat_final.pt sample: "\nThe user is a graduate student. So the user should be formal and professional, and know the material before speaking.\n\nGoal: to introduce the user to a common Python data model (RDBC vs. RDD, and a few practical and helpful usage points) while keeping the user's identity private.\n\nRDBC vs RDD:\n- RDD is the older, more standard relational database object; RDBC is newer, more flexible and can use SQLite, PostgreSQL, or an underlying database manager.\n- RDD is the default" === export === ========================================================================== EXPORT checkpoints/500m_chat_final.pt -> /workspace/hf-500m-chat ========================================================================== dim=1280 layers=26 heads=20 kv_heads=4 ffn=3584 vocab=32000 seq=2048 remapped 237 tensors, 0 missing, 0 unexpected PARITY max|diff|=0.00e+00 mean|diff|=0.00e+00 argmax agreement=100.0000% PARITY OK — exported model is numerically identical to native. chat template verified identical to training format: '<|user|>hi<|end|><|assistant|>' tokenizer: vocab=32000 (matches model), all 12 special ids verified round-trip ok: "Oi. Sprocket. What're we buildin'?" wrote /workspace/hf-500m-chat next: lm_eval --model hf --model_args pretrained=/workspace/hf-500m-chat,dtype=float32 \ --tasks hellaswag,arc_easy,arc_challenge,piqa,winogrande --batch_size 16 python llama.cpp/convert_hf_to_gguf.py /workspace/hf-500m-chat --outtype f16 === gguf === convert_hf_to_gguf_update.py: already registered conversion/base.py: already registered INFO:hf-to-gguf:Loading model: hf-500m-chat INFO:hf-to-gguf:Model architecture: LlamaForCausalLM INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {1280, 32000} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {3584, 1280} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {1280, 3584} INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> F16, shape = {1280, 1280} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> F16, shape = {1280, 256} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {1280} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 2048 INFO:hf-to-gguf:gguf: embedding length = 1280 INFO:hf-to-gguf:gguf: feed forward length = 3584 INFO:hf-to-gguf:gguf: head count = 20 INFO:hf-to-gguf:gguf: key-value head count = 4 INFO:hf-to-gguf:gguf: rope theta = 10000.0 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: file type = 1 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 31728 merge(s). INFO:gguf.vocab:Setting special token type bos to 0 INFO:gguf.vocab:Setting special token type eos to 5 INFO:gguf.vocab:Setting special token type unk to 0 INFO:gguf.vocab:Setting special token type pad to 1 INFO:gguf.vocab:Setting chat_template to {% for m in messages %}{% if m['role'] == 'system' %}<|system|>{{ m['content'] }}<|end|>{% elif m['role'] == 'user' %}<|user|>{{ m['content'] }}<|end|>{% elif m['role'] == 'assistant' %}<|assistant|>{{ m['content'] }}<|end|>{% elif m['role'] == 'memory' %}<|memory_read|>{{ m['content'] }}<|end|>{% elif m['role'] == 'tool' %}<|tool_result|>{{ m['content'] }}<|end|>{% endif %}{% endfor %}{% if add_generation_prompt %}<|assistant|>{% endif %} INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/workspace/sprocket-500m-chat-f16.gguf: n_tensors = 236, total_size = 1.0G Writing: 0%| | 0.00/1.00G [00:00", "<|pad|>", "<|syste... llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,32000] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,31728] = ["Ġ t", "Ġ a", "h e", "i n", "r e",... llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 0 llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 5 llama_model_loader: - kv 27: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 28: tokenizer.ggml.padding_token_id u32 = 1 llama_model_loader: - kv 29: tokenizer.chat_template str = {% for m in messages %}{% if m['role'... llama_model_loader: - type f32: 53 tensors llama_model_loader: - type f16: 183 tensors [ 1/ 236] output_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 2/ 236] token_embd.weight - [ 1280, 32000, 1, 1], type = f16, converting to q6_K .. size = 78.12 MiB -> 32.04 MiB [ 3/ 236] blk.0.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 4/ 236] blk.0.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 5/ 236] blk.0.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 6/ 236] blk.0.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 7/ 236] blk.0.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 8/ 236] blk.0.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 9/ 236] blk.0.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 10/ 236] blk.0.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 11/ 236] blk.0.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 12/ 236] blk.1.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 13/ 236] blk.1.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 14/ 236] blk.1.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 15/ 236] blk.1.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 16/ 236] blk.1.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 17/ 236] blk.1.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 18/ 236] blk.1.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 19/ 236] blk.1.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 20/ 236] blk.1.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 21/ 236] blk.2.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 22/ 236] blk.2.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 23/ 236] blk.2.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 24/ 236] blk.2.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 25/ 236] blk.2.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 26/ 236] blk.2.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 27/ 236] blk.2.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 28/ 236] blk.2.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 29/ 236] blk.2.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 30/ 236] blk.3.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 31/ 236] blk.3.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 32/ 236] blk.3.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 33/ 236] blk.3.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 34/ 236] blk.3.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 35/ 236] blk.3.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 36/ 236] blk.3.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 37/ 236] blk.3.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 38/ 236] blk.3.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 39/ 236] blk.4.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 40/ 236] blk.4.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 41/ 236] blk.4.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 42/ 236] blk.4.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 43/ 236] blk.4.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 44/ 236] blk.4.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 45/ 236] blk.4.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 46/ 236] blk.4.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 47/ 236] blk.4.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 48/ 236] blk.5.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 49/ 236] blk.5.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 50/ 236] blk.5.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 51/ 236] blk.5.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 52/ 236] blk.5.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 53/ 236] blk.5.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 54/ 236] blk.5.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 55/ 236] blk.5.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 56/ 236] blk.5.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 57/ 236] blk.6.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 58/ 236] blk.6.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 59/ 236] blk.6.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 60/ 236] blk.6.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 61/ 236] blk.6.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 62/ 236] blk.6.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 63/ 236] blk.6.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 64/ 236] blk.6.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 65/ 236] blk.6.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 66/ 236] blk.7.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 67/ 236] blk.7.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 68/ 236] blk.7.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 69/ 236] blk.7.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 70/ 236] blk.7.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 71/ 236] blk.7.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 72/ 236] blk.7.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 73/ 236] blk.7.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 74/ 236] blk.7.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 75/ 236] blk.8.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 76/ 236] blk.8.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 77/ 236] blk.8.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 78/ 236] blk.8.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 79/ 236] blk.8.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 80/ 236] blk.8.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 81/ 236] blk.8.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 82/ 236] blk.8.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 83/ 236] blk.8.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 84/ 236] blk.9.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 85/ 236] blk.9.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 86/ 236] blk.9.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 87/ 236] blk.9.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 88/ 236] blk.9.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 89/ 236] blk.9.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 90/ 236] blk.9.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 91/ 236] blk.9.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 92/ 236] blk.9.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 93/ 236] blk.10.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 94/ 236] blk.10.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 95/ 236] blk.10.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 96/ 236] blk.10.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 97/ 236] blk.10.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 98/ 236] blk.10.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 99/ 236] blk.10.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 100/ 236] blk.10.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 101/ 236] blk.10.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 102/ 236] blk.11.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 103/ 236] blk.11.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 104/ 236] blk.11.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 105/ 236] blk.11.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 106/ 236] blk.11.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 107/ 236] blk.11.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 108/ 236] blk.11.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 109/ 236] blk.11.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 110/ 236] blk.11.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 111/ 236] blk.12.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 112/ 236] blk.12.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 113/ 236] blk.12.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 114/ 236] blk.12.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 115/ 236] blk.12.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 116/ 236] blk.12.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 117/ 236] blk.12.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 118/ 236] blk.12.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 119/ 236] blk.12.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 120/ 236] blk.13.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 121/ 236] blk.13.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 122/ 236] blk.13.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 123/ 236] blk.13.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 124/ 236] blk.13.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 125/ 236] blk.13.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 126/ 236] blk.13.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 127/ 236] blk.13.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 128/ 236] blk.13.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 129/ 236] blk.14.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 130/ 236] blk.14.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 131/ 236] blk.14.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 132/ 236] blk.14.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 133/ 236] blk.14.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 134/ 236] blk.14.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 135/ 236] blk.14.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 136/ 236] blk.14.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 137/ 236] blk.14.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 138/ 236] blk.15.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 139/ 236] blk.15.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 140/ 236] blk.15.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 141/ 236] blk.15.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 142/ 236] blk.15.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 143/ 236] blk.15.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 144/ 236] blk.15.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 145/ 236] blk.15.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 146/ 236] blk.15.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 147/ 236] blk.16.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 148/ 236] blk.16.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 149/ 236] blk.16.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 150/ 236] blk.16.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 151/ 236] blk.16.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 152/ 236] blk.16.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 153/ 236] blk.16.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 154/ 236] blk.16.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 155/ 236] blk.16.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 156/ 236] blk.17.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 157/ 236] blk.17.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 158/ 236] blk.17.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 159/ 236] blk.17.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 160/ 236] blk.17.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 161/ 236] blk.17.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 162/ 236] blk.17.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 163/ 236] blk.17.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 164/ 236] blk.17.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 165/ 236] blk.18.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 166/ 236] blk.18.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 167/ 236] blk.18.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 168/ 236] blk.18.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 169/ 236] blk.18.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 170/ 236] blk.18.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 171/ 236] blk.18.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 172/ 236] blk.18.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 173/ 236] blk.18.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 174/ 236] blk.19.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 175/ 236] blk.19.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 176/ 236] blk.19.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 177/ 236] blk.19.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 178/ 236] blk.19.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 179/ 236] blk.19.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 180/ 236] blk.19.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 181/ 236] blk.19.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 182/ 236] blk.19.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 183/ 236] blk.20.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 184/ 236] blk.20.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 185/ 236] blk.20.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 186/ 236] blk.20.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 187/ 236] blk.20.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 188/ 236] blk.20.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 189/ 236] blk.20.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 190/ 236] blk.20.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 191/ 236] blk.20.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 192/ 236] blk.21.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 193/ 236] blk.21.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 194/ 236] blk.21.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 195/ 236] blk.21.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 196/ 236] blk.21.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 197/ 236] blk.21.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 198/ 236] blk.21.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 199/ 236] blk.21.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 200/ 236] blk.21.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 201/ 236] blk.22.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 202/ 236] blk.22.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 203/ 236] blk.22.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 204/ 236] blk.22.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 205/ 236] blk.22.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 206/ 236] blk.22.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 207/ 236] blk.22.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 208/ 236] blk.22.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 209/ 236] blk.22.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 210/ 236] blk.23.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 211/ 236] blk.23.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 212/ 236] blk.23.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 213/ 236] blk.23.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 214/ 236] blk.23.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 215/ 236] blk.23.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 216/ 236] blk.23.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 217/ 236] blk.23.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 218/ 236] blk.23.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 219/ 236] blk.24.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 220/ 236] blk.24.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 221/ 236] blk.24.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 222/ 236] blk.24.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 223/ 236] blk.24.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 224/ 236] blk.24.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 225/ 236] blk.24.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 226/ 236] blk.24.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 227/ 236] blk.24.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 228/ 236] blk.25.attn_k.weight - [ 1280, 256, 1, 1], type = f16, converting to q4_K .. size = 0.62 MiB -> 0.18 MiB [ 229/ 236] blk.25.attn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 230/ 236] blk.25.attn_output.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 231/ 236] blk.25.attn_q.weight - [ 1280, 1280, 1, 1], type = f16, converting to q4_K .. size = 3.12 MiB -> 0.88 MiB [ 232/ 236] blk.25.attn_v.weight - [ 1280, 256, 1, 1], type = f16, converting to q6_K .. size = 0.62 MiB -> 0.26 MiB [ 233/ 236] blk.25.ffn_down.weight - [ 3584, 1280, 1, 1], type = f16, converting to q6_K .. size = 8.75 MiB -> 3.59 MiB [ 234/ 236] blk.25.ffn_gate.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB [ 235/ 236] blk.25.ffn_norm.weight - [ 1280, 1, 1, 1], type = f32, size = 0.005 MiB [ 236/ 236] blk.25.ffn_up.weight - [ 1280, 3584, 1, 1], type = f16, converting to q4_K .. size = 8.75 MiB -> 2.46 MiB llama_model_quantize_impl: model size = 955.88 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 294.81 MiB (4.94 BPW) llama_quantize: quantize time = 8384.60 ms llama_quantize: total time = 8384.60 ms -rw-rw-rw- 1 root root 957M Jul 31 05:32 /workspace/sprocket-500m-chat-f16.gguf -rw-rw-rw- 1 root root 296M Jul 31 05:33 /workspace/sprocket-500m-chat-q4_k_m.gguf === upload === Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...0m-chat/model.safetensors: 0%| | 602kB / 1.00GB  ...0m-chat/model.safetensors: 0%| | 602kB / 1.00GB  Processing Files (0 / 1) : 0%| | 602kB / 1.00GB, 58.9kB/s New Data Upload : 1%| | 602kB / 67.1MB, 58.9kB/s  ...0m-chat/model.safetensors: 2%|▏ | 16.3MB / 1.00GB  Processing Files (0 / 1) : 2%|▏ | 16.3MB / 1.00GB, 1.58MB/s New Data Upload : 12%|█▏ | 16.3MB / 134MB, 1.58MB/s  ...0m-chat/model.safetensors: 7%|▋ | 67.5MB / 1.00GB  Processing Files (0 / 1) : 7%|▋ | 67.5MB / 1.00GB, 6.47MB/s New Data Upload : 50%|█████ | 67.5MB / 134MB, 6.47MB/s  ...0m-chat/model.safetensors: 8%|▊ | 77.5MB / 1.00GB  Processing Files (0 / 1) : 8%|▊ | 77.5MB / 1.00GB, 7.28MB/s New Data Upload : 39%|███▊ | 77.5MB / 201MB, 7.28MB/s  ...0m-chat/model.safetensors: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 12.5MB/s New Data Upload : 67%|██████▋ | 134MB / 201MB, 12.5MB/s  ...0m-chat/model.safetensors: 15%|█▍ | 149MB / 1.00GB  Processing Files (0 / 1) : 15%|█▍ | 149MB / 1.00GB, 13.6MB/s New Data Upload : 56%|█████▌ | 149MB / 268MB, 13.6MB/s  ...0m-chat/model.safetensors: 20%|██ | 203MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 203MB / 1.00GB, 18.4MB/s New Data Upload : 61%|██████ | 203MB / 335MB, 18.4MB/s  ...0m-chat/model.safetensors: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.2MB/s New Data Upload : 100%|█████████▉| 334MB / 335MB, 30.2MB/s  ...0m-chat/model.safetensors: 33%|███▎ | 335MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 335MB / 1.00GB, 29.5MB/s New Data Upload : 83%|████████▎ | 335MB / 402MB, 29.5MB/s  ...0m-chat/model.safetensors: 40%|████ | 402MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 402MB / 1.00GB, 35.0MB/s New Data Upload : 86%|████████▌ | 402MB / 469MB, 35.0MB/s  ...0m-chat/model.safetensors: 40%|████ | 402MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 402MB / 1.00GB, 34.2MB/s New Data Upload : 86%|████████▌ | 402MB / 469MB, 34.2MB/s  ...0m-chat/model.safetensors: 47%|████▋ | 469MB / 1.00GB  Processing Files (0 / 1) : 47%|████▋ | 469MB / 1.00GB, 39.5MB/s New Data Upload : 87%|████████▋ | 469MB / 537MB, 39.5MB/s  ...0m-chat/model.safetensors: 53%|█████▎ | 536MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 536MB / 1.00GB, 44.7MB/s New Data Upload : 89%|████████▉ | 536MB / 604MB, 44.7MB/s  ...0m-chat/model.safetensors: 60%|██████ | 603MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 603MB / 1.00GB, 49.7MB/s New Data Upload : 90%|████████▉ | 603MB / 671MB, 49.7MB/s  ...0m-chat/model.safetensors: 60%|██████ | 604MB / 1.00GB  Processing Files (0 / 1) : 60%|██████ | 604MB / 1.00GB, 48.7MB/s New Data Upload : 90%|████████▉ | 604MB / 671MB, 48.7MB/s  ...0m-chat/model.safetensors: 61%|██████ | 610MB / 1.00GB  Processing Files (0 / 1) : 61%|██████ | 610MB / 1.00GB, 48.2MB/s New Data Upload : 83%|████████▎ | 610MB / 738MB, 48.2MB/s  ...0m-chat/model.safetensors: 74%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 74%|███████▎ | 737MB / 1.00GB, 58.4MB/s New Data Upload : 92%|█████████▏| 737MB / 805MB, 58.4MB/s  ...0m-chat/model.safetensors: 80%|████████ | 803MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 803MB / 1.00GB, 62.9MB/s New Data Upload : 92%|█████████▏| 803MB / 872MB, 62.9MB/s  ...0m-chat/model.safetensors: 80%|████████ | 804MB / 1.00GB  Processing Files (0 / 1) : 80%|████████ | 804MB / 1.00GB, 61.6MB/s New Data Upload : 92%|█████████▏| 804MB / 872MB, 61.6MB/s  ...0m-chat/model.safetensors: 87%|████████▋ | 871MB / 1.00GB  Processing Files (0 / 1) : 87%|████████▋ | 871MB / 1.00GB, 66.1MB/s New Data Upload : 93%|█████████▎| 871MB / 939MB, 66.1MB/s  ...0m-chat/model.safetensors: 94%|█████████▎| 937MB / 1.00GB  Processing Files (0 / 1) : 94%|█████████▎| 937MB / 1.00GB, 70.4MB/s New Data Upload : 94%|█████████▎| 937MB / 1.00GB, 70.4MB/s  ...0m-chat/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 74.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 74.4MB/s  ...0m-chat/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 72.9MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 72.9MB/s  ...0m-chat/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.3MB/s  ...0m-chat/model.safetensors: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 68.4MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 68.4MB/s  ...0m-chat/model.safetensors: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 66.9MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 66.9MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 66.9MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 66.9MB/s ...0m-chat/model.safetensors: 100%|██████████| 1.00GB / 1.00GB uploaded chat/ (HF format) Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...rocket-500m-chat-f16.gguf: 0%| | 1.07MB / 1.00GB  ...rocket-500m-chat-f16.gguf: 0%| | 1.07MB / 1.00GB  Processing Files (0 / 1) : 0%| | 1.07MB / 1.00GB, ???B/s ...rocket-500m-chat-f16.gguf: 6%|▌ | 61.4MB / 1.00GB  Processing Files (0 / 1) : 6%|▌ | 61.4MB / 1.00GB, ???B/s New Data Upload : 45%|████▌ | 60.4MB / 134MB, ???B/s  ...rocket-500m-chat-f16.gguf: 13%|█▎ | 134MB / 1.00GB  Processing Files (0 / 1) : 13%|█▎ | 134MB / 1.00GB, 13.0MB/s New Data Upload : 66%|██████▌ | 133MB / 201MB, 12.9MB/s  ...rocket-500m-chat-f16.gguf: 20%|██ | 201MB / 1.00GB  Processing Files (0 / 1) : 20%|██ | 201MB / 1.00GB, 19.2MB/s New Data Upload : 74%|███████▍ | 200MB / 268MB, 19.1MB/s  ...rocket-500m-chat-f16.gguf: 27%|██▋ | 267MB / 1.00GB  Processing Files (0 / 1) : 27%|██▋ | 267MB / 1.00GB, 25.1MB/s New Data Upload : 99%|█████████▉| 266MB / 268MB, 25.0MB/s  ...rocket-500m-chat-f16.gguf: 28%|██▊ | 278MB / 1.00GB  Processing Files (0 / 1) : 28%|██▊ | 278MB / 1.00GB, 25.5MB/s New Data Upload : 69%|██████▉ | 277MB / 402MB, 25.4MB/s  ...rocket-500m-chat-f16.gguf: 33%|███▎ | 334MB / 1.00GB  Processing Files (0 / 1) : 33%|███▎ | 334MB / 1.00GB, 30.3MB/s New Data Upload : 71%|███████ | 333MB / 469MB, 30.2MB/s  ...rocket-500m-chat-f16.gguf: 34%|███▎ | 337MB / 1.00GB  Processing Files (0 / 1) : 34%|███▎ | 337MB / 1.00GB, 29.8MB/s New Data Upload : 72%|███████▏ | 336MB / 469MB, 29.7MB/s  ...rocket-500m-chat-f16.gguf: 40%|████ | 404MB / 1.00GB  Processing Files (0 / 1) : 40%|████ | 404MB / 1.00GB, 35.4MB/s New Data Upload : 75%|███████▌ | 403MB / 537MB, 35.3MB/s  ...rocket-500m-chat-f16.gguf: 53%|█████▎ | 535MB / 1.00GB  Processing Files (0 / 1) : 53%|█████▎ | 535MB / 1.00GB, 46.8MB/s New Data Upload : 88%|████████▊ | 534MB / 604MB, 46.7MB/s  ...rocket-500m-chat-f16.gguf: 60%|█████▉ | 602MB / 1.00GB  Processing Files (0 / 1) : 60%|█████▉ | 602MB / 1.00GB, 51.9MB/s New Data Upload : 90%|████████▉ | 601MB / 671MB, 51.8MB/s  ...rocket-500m-chat-f16.gguf: 67%|██████▋ | 669MB / 1.00GB  Processing Files (0 / 1) : 67%|██████▋ | 669MB / 1.00GB, 56.9MB/s New Data Upload : 91%|█████████ | 668MB / 738MB, 56.8MB/s  ...rocket-500m-chat-f16.gguf: 73%|███████▎ | 736MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 736MB / 1.00GB, 61.7MB/s New Data Upload : 91%|█████████▏| 735MB / 805MB, 61.6MB/s  ...rocket-500m-chat-f16.gguf: 73%|███████▎ | 737MB / 1.00GB  Processing Files (0 / 1) : 73%|███████▎ | 737MB / 1.00GB, 60.4MB/s New Data Upload : 91%|█████████▏| 736MB / 805MB, 60.3MB/s  ...rocket-500m-chat-f16.gguf: 82%|████████▏ | 825MB / 1.00GB  Processing Files (0 / 1) : 82%|████████▏ | 825MB / 1.00GB, 67.0MB/s New Data Upload : 94%|█████████▍| 824MB / 872MB, 66.9MB/s  ...rocket-500m-chat-f16.gguf: 89%|████████▉ | 893MB / 1.00GB  Processing Files (0 / 1) : 89%|████████▉ | 893MB / 1.00GB, 71.6MB/s New Data Upload : 95%|█████████▍| 891MB / 939MB, 71.5MB/s  ...rocket-500m-chat-f16.gguf: 93%|█████████▎| 938MB / 1.00GB  Processing Files (0 / 1) : 93%|█████████▎| 938MB / 1.00GB, 74.0MB/s New Data Upload : 93%|█████████▎| 936MB / 1.00GB, 73.9MB/s  ...rocket-500m-chat-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 78.0MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 78.0MB/s  ...rocket-500m-chat-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 76.3MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 76.3MB/s  ...rocket-500m-chat-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 73.1MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 73.0MB/s  ...rocket-500m-chat-f16.gguf: 100%|█████████▉| 1.00GB / 1.00GB  Processing Files (0 / 1) : 100%|█████████▉| 1.00GB / 1.00GB, 71.5MB/s New Data Upload : 100%|█████████▉| 1.00GB / 1.00GB, 71.5MB/s  ...rocket-500m-chat-f16.gguf: 100%|██████████| 1.00GB / 1.00GB  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s  Processing Files (1 / 1) : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s New Data Upload : 100%|██████████| 1.00GB / 1.00GB, 70.0MB/s ...rocket-500m-chat-f16.gguf: 100%|██████████| 1.00GB / 1.00GB uploaded sprocket-500m-chat-f16.gguf 1003465920 Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B  ...ket-500m-chat-q4_k_m.gguf: 0%| | 1.07MB / 310MB  ...ket-500m-chat-q4_k_m.gguf: 0%| | 1.07MB / 310MB  Processing Files (0 / 1) : 0%| | 1.07MB / 310MB, ???B/s ...ket-500m-chat-q4_k_m.gguf: 22%|██▏ | 67.9MB / 310MB  Processing Files (0 / 1) : 22%|██▏ | 67.9MB / 310MB, ???B/s New Data Upload : 50%|████▉ | 66.8MB / 134MB, ???B/s  ...ket-500m-chat-q4_k_m.gguf: 43%|████▎ | 134MB / 310MB  Processing Files (0 / 1) : 43%|████▎ | 134MB / 310MB, 13.0MB/s New Data Upload : 66%|██████▋ | 133MB / 201MB, 12.9MB/s  ...ket-500m-chat-q4_k_m.gguf: 65%|██████▍ | 201MB / 310MB  Processing Files (0 / 1) : 65%|██████▍ | 201MB / 310MB, 19.2MB/s New Data Upload : 99%|█████████▉| 200MB / 201MB, 19.1MB/s  ...ket-500m-chat-q4_k_m.gguf: 86%|████████▋ | 268MB / 310MB  Processing Files (0 / 1) : 86%|████████▋ | 268MB / 310MB, 25.1MB/s New Data Upload : 86%|████████▌ | 267MB / 309MB, 25.0MB/s  ...ket-500m-chat-q4_k_m.gguf: 99%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 99%|█████████▉| 309MB / 310MB, 28.4MB/s New Data Upload : 99%|█████████▉| 308MB / 309MB, 28.3MB/s  ...ket-500m-chat-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 27.1MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 27.0MB/s  ...ket-500m-chat-q4_k_m.gguf: 100%|█████████▉| 309MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 309MB / 310MB, 26.6MB/s New Data Upload : 100%|█████████▉| 308MB / 309MB, 26.5MB/s  ...ket-500m-chat-q4_k_m.gguf: 100%|█████████▉| 310MB / 310MB  Processing Files (0 / 1) : 100%|█████████▉| 310MB / 310MB, 26.0MB/s New Data Upload : 100%|█████████▉| 309MB / 309MB, 25.9MB/s  ...ket-500m-chat-q4_k_m.gguf: 100%|██████████| 310MB / 310MB  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s  Processing Files (1 / 1) : 100%|██████████| 310MB / 310MB, 25.4MB/s New Data Upload : 100%|██████████| 309MB / 309MB, 25.3MB/s ...ket-500m-chat-q4_k_m.gguf: 100%|██████████| 310MB / 310MB uploaded sprocket-500m-chat-q4_k_m.gguf 310279360