| The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function. |
|
Loading checkpoint shards: 0%| | 0/6 [00:00<?, ?it/s]
Loading checkpoint shards: 17%|█▋ | 1/6 [00:00<00:02, 2.02it/s]
Loading checkpoint shards: 33%|███▎ | 2/6 [00:01<00:02, 1.82it/s]
Loading checkpoint shards: 50%|█████ | 3/6 [00:01<00:01, 1.72it/s]
Loading checkpoint shards: 67%|██████▋ | 4/6 [00:02<00:01, 1.69it/s]
Loading checkpoint shards: 83%|████████▎ | 5/6 [00:02<00:00, 1.70it/s]
Loading checkpoint shards: 100%|██████████| 6/6 [00:03<00:00, 1.88it/s]
Loading checkpoint shards: 100%|██████████| 6/6 [00:03<00:00, 1.81it/s] |
| trainable params: 1,069,547,520 || all params: 8,049,135,616 || trainable%: 13.2877 |
| Model sharded via device_map='auto'; skipping .to() |
| MASK token id: 126336 |
| val set: 500 samples eval_interval=50 eval_batches=20 |
| [llada] step 20/3375 loss=nan lr=4.00e-05 elapsed=203s |
| [llada] step 40/3375 loss=nan lr=8.00e-05 elapsed=414s |
| [llada] step 50 val_loss=nan |
| [topk] +save step=50 val=nan (top-1/2) |
| [llada] step 60/3375 loss=nan lr=1.20e-04 elapsed=674s |
| [llada] step 80/3375 loss=nan lr=1.60e-04 elapsed=886s |
| [llada] step 100/3375 loss=nan lr=2.00e-04 elapsed=1098s |
| [llada] step 100 val_loss=nan |
| [topk] +save step=100 val=nan (top-2/2) |
| [llada] step 120/3375 loss=nan lr=2.00e-04 elapsed=1358s |
| [llada] step 140/3375 loss=nan lr=2.00e-04 elapsed=1569s |
| [llada] step 150 val_loss=nan |
| [llada] step 160/3375 loss=nan lr=2.00e-04 elapsed=1828s |
| [llada] step 180/3375 loss=nan lr=2.00e-04 elapsed=2042s |
| [llada] step 200/3375 loss=nan lr=2.00e-04 elapsed=2251s |
| [llada] step 200 val_loss=nan |
| [llada] step 220/3375 loss=nan lr=1.99e-04 elapsed=2512s |
| [llada] step 240/3375 loss=nan lr=1.99e-04 elapsed=2723s |
| [llada] step 250 val_loss=nan |
| [llada] step 260/3375 loss=nan lr=1.99e-04 elapsed=2983s |
| [llada] step 280/3375 loss=nan lr=1.99e-04 elapsed=3195s |
| [llada] step 300/3375 loss=nan lr=1.98e-04 elapsed=3407s |
| [llada] step 300 val_loss=nan |
| [llada] step 320/3375 loss=nan lr=1.98e-04 elapsed=3666s |
| [llada] step 340/3375 loss=nan lr=1.97e-04 elapsed=3878s |
| [llada] step 350 val_loss=nan |
| [llada] step 360/3375 loss=nan lr=1.97e-04 elapsed=4139s |
| [llada] step 380/3375 loss=nan lr=1.96e-04 elapsed=4351s |
| [llada] step 400/3375 loss=nan lr=1.96e-04 elapsed=4563s |
| [llada] step 400 val_loss=nan |
| [llada] step 420/3375 loss=nan lr=1.95e-04 elapsed=4823s |
| [llada] step 440/3375 loss=nan lr=1.95e-04 elapsed=5036s |
| [llada] step 450 val_loss=nan |
| [llada] step 460/3375 loss=nan lr=1.94e-04 elapsed=5295s |
| [llada] step 480/3375 loss=nan lr=1.93e-04 elapsed=5506s |
| [llada] step 500/3375 loss=nan lr=1.93e-04 elapsed=5718s |
| [llada] step 500 val_loss=nan |
| [llada] step 520/3375 loss=nan lr=1.92e-04 elapsed=5980s |
| [llada] step 540/3375 loss=nan lr=1.91e-04 elapsed=6192s |
| [llada] step 550 val_loss=nan |
| [llada] step 560/3375 loss=nan lr=1.90e-04 elapsed=6452s |
| [llada] step 580/3375 loss=nan lr=1.90e-04 elapsed=6665s |
| [llada] step 600/3375 loss=nan lr=1.89e-04 elapsed=6876s |
| [llada] step 600 val_loss=nan |
| [llada] step 620/3375 loss=nan lr=1.88e-04 elapsed=7137s |
| [llada] step 640/3375 loss=nan lr=1.87e-04 elapsed=7349s |
| [llada] step 650 val_loss=nan |
| [llada] step 660/3375 loss=nan lr=1.86e-04 elapsed=7609s |
| [llada] step 680/3375 loss=nan lr=1.85e-04 elapsed=7821s |
| [llada] step 700/3375 loss=nan lr=1.84e-04 elapsed=8032s |
| [llada] step 700 val_loss=nan |
| [llada] step 720/3375 loss=nan lr=1.83e-04 elapsed=8289s |
| [llada] step 740/3375 loss=nan lr=1.82e-04 elapsed=8503s |
| [llada] step 750 val_loss=nan |
| [llada] step 760/3375 loss=nan lr=1.81e-04 elapsed=8762s |
| [llada] step 780/3375 loss=nan lr=1.79e-04 elapsed=8970s |
| [llada] step 800/3375 loss=nan lr=1.78e-04 elapsed=9180s |
| [llada] step 800 val_loss=nan |
| [llada] step 820/3375 loss=nan lr=1.77e-04 elapsed=9433s |
| [llada] step 840/3375 loss=nan lr=1.76e-04 elapsed=9642s |
| [llada] step 850 val_loss=nan |
| [llada] step 860/3375 loss=nan lr=1.75e-04 elapsed=9897s |
| [llada] step 880/3375 loss=nan lr=1.73e-04 elapsed=10111s |
| [llada] step 900/3375 loss=nan lr=1.72e-04 elapsed=10321s |
| [llada] step 900 val_loss=nan |
| [llada] step 920/3375 loss=nan lr=1.71e-04 elapsed=10573s |
| [llada] step 940/3375 loss=nan lr=1.69e-04 elapsed=10783s |
| [llada] step 950 val_loss=nan |
| [llada] step 960/3375 loss=nan lr=1.68e-04 elapsed=11044s |
| [llada] step 980/3375 loss=nan lr=1.66e-04 elapsed=11253s |
| [llada] step 1000/3375 loss=nan lr=1.65e-04 elapsed=11461s |
| [llada] step 1000 val_loss=nan |
| [llada] step 1020/3375 loss=nan lr=1.64e-04 elapsed=11719s |
| [llada] step 1040/3375 loss=nan lr=1.62e-04 elapsed=11927s |
| [llada] step 1050 val_loss=nan |
| [llada] step 1060/3375 loss=nan lr=1.61e-04 elapsed=12181s |
| [llada] step 1080/3375 loss=nan lr=1.59e-04 elapsed=12390s |
| [llada] step 1100/3375 loss=nan lr=1.57e-04 elapsed=12599s |
| [llada] step 1100 val_loss=nan |
|
|
