The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function.

Loading checkpoint shards:   0%|          | 0/6 [00:00<?, ?it/s]
Loading checkpoint shards:  17%|█▋        | 1/6 [00:00<00:02,  2.02it/s]
Loading checkpoint shards:  33%|███▎      | 2/6 [00:01<00:02,  1.82it/s]
Loading checkpoint shards:  50%|█████     | 3/6 [00:01<00:01,  1.72it/s]
Loading checkpoint shards:  67%|██████▋   | 4/6 [00:02<00:01,  1.69it/s]
Loading checkpoint shards:  83%|████████▎ | 5/6 [00:02<00:00,  1.70it/s]
Loading checkpoint shards: 100%|██████████| 6/6 [00:03<00:00,  1.88it/s]
Loading checkpoint shards: 100%|██████████| 6/6 [00:03<00:00,  1.81it/s]
trainable params: 1,069,547,520 || all params: 8,049,135,616 || trainable%: 13.2877
Model sharded via device_map='auto'; skipping .to()
MASK token id: 126336
val set: 500 samples  eval_interval=50  eval_batches=20
[llada] step 20/3375 loss=nan lr=4.00e-05 elapsed=203s
[llada] step 40/3375 loss=nan lr=8.00e-05 elapsed=414s
[llada] step 50 val_loss=nan
[topk] +save step=50 val=nan (top-1/2)
[llada] step 60/3375 loss=nan lr=1.20e-04 elapsed=674s
[llada] step 80/3375 loss=nan lr=1.60e-04 elapsed=886s
[llada] step 100/3375 loss=nan lr=2.00e-04 elapsed=1098s
[llada] step 100 val_loss=nan
[topk] +save step=100 val=nan (top-2/2)
[llada] step 120/3375 loss=nan lr=2.00e-04 elapsed=1358s
[llada] step 140/3375 loss=nan lr=2.00e-04 elapsed=1569s
[llada] step 150 val_loss=nan
[llada] step 160/3375 loss=nan lr=2.00e-04 elapsed=1828s
[llada] step 180/3375 loss=nan lr=2.00e-04 elapsed=2042s
[llada] step 200/3375 loss=nan lr=2.00e-04 elapsed=2251s
[llada] step 200 val_loss=nan
[llada] step 220/3375 loss=nan lr=1.99e-04 elapsed=2512s
[llada] step 240/3375 loss=nan lr=1.99e-04 elapsed=2723s
[llada] step 250 val_loss=nan
[llada] step 260/3375 loss=nan lr=1.99e-04 elapsed=2983s
[llada] step 280/3375 loss=nan lr=1.99e-04 elapsed=3195s
[llada] step 300/3375 loss=nan lr=1.98e-04 elapsed=3407s
[llada] step 300 val_loss=nan
[llada] step 320/3375 loss=nan lr=1.98e-04 elapsed=3666s
[llada] step 340/3375 loss=nan lr=1.97e-04 elapsed=3878s
[llada] step 350 val_loss=nan
[llada] step 360/3375 loss=nan lr=1.97e-04 elapsed=4139s
[llada] step 380/3375 loss=nan lr=1.96e-04 elapsed=4351s
[llada] step 400/3375 loss=nan lr=1.96e-04 elapsed=4563s
[llada] step 400 val_loss=nan
[llada] step 420/3375 loss=nan lr=1.95e-04 elapsed=4823s
[llada] step 440/3375 loss=nan lr=1.95e-04 elapsed=5036s
[llada] step 450 val_loss=nan
[llada] step 460/3375 loss=nan lr=1.94e-04 elapsed=5295s
[llada] step 480/3375 loss=nan lr=1.93e-04 elapsed=5506s
[llada] step 500/3375 loss=nan lr=1.93e-04 elapsed=5718s
[llada] step 500 val_loss=nan
[llada] step 520/3375 loss=nan lr=1.92e-04 elapsed=5980s
[llada] step 540/3375 loss=nan lr=1.91e-04 elapsed=6192s
[llada] step 550 val_loss=nan
[llada] step 560/3375 loss=nan lr=1.90e-04 elapsed=6452s
[llada] step 580/3375 loss=nan lr=1.90e-04 elapsed=6665s
[llada] step 600/3375 loss=nan lr=1.89e-04 elapsed=6876s
[llada] step 600 val_loss=nan
[llada] step 620/3375 loss=nan lr=1.88e-04 elapsed=7137s
[llada] step 640/3375 loss=nan lr=1.87e-04 elapsed=7349s
[llada] step 650 val_loss=nan
[llada] step 660/3375 loss=nan lr=1.86e-04 elapsed=7609s
[llada] step 680/3375 loss=nan lr=1.85e-04 elapsed=7821s
[llada] step 700/3375 loss=nan lr=1.84e-04 elapsed=8032s
[llada] step 700 val_loss=nan
[llada] step 720/3375 loss=nan lr=1.83e-04 elapsed=8289s
[llada] step 740/3375 loss=nan lr=1.82e-04 elapsed=8503s
[llada] step 750 val_loss=nan
[llada] step 760/3375 loss=nan lr=1.81e-04 elapsed=8762s
[llada] step 780/3375 loss=nan lr=1.79e-04 elapsed=8970s
[llada] step 800/3375 loss=nan lr=1.78e-04 elapsed=9180s
[llada] step 800 val_loss=nan
[llada] step 820/3375 loss=nan lr=1.77e-04 elapsed=9433s
[llada] step 840/3375 loss=nan lr=1.76e-04 elapsed=9642s
[llada] step 850 val_loss=nan
[llada] step 860/3375 loss=nan lr=1.75e-04 elapsed=9897s
[llada] step 880/3375 loss=nan lr=1.73e-04 elapsed=10111s
[llada] step 900/3375 loss=nan lr=1.72e-04 elapsed=10321s
[llada] step 900 val_loss=nan
[llada] step 920/3375 loss=nan lr=1.71e-04 elapsed=10573s
[llada] step 940/3375 loss=nan lr=1.69e-04 elapsed=10783s
[llada] step 950 val_loss=nan
[llada] step 960/3375 loss=nan lr=1.68e-04 elapsed=11044s
[llada] step 980/3375 loss=nan lr=1.66e-04 elapsed=11253s
[llada] step 1000/3375 loss=nan lr=1.65e-04 elapsed=11461s
[llada] step 1000 val_loss=nan
[llada] step 1020/3375 loss=nan lr=1.64e-04 elapsed=11719s
[llada] step 1040/3375 loss=nan lr=1.62e-04 elapsed=11927s
[llada] step 1050 val_loss=nan
[llada] step 1060/3375 loss=nan lr=1.61e-04 elapsed=12181s
[llada] step 1080/3375 loss=nan lr=1.59e-04 elapsed=12390s
[llada] step 1100/3375 loss=nan lr=1.57e-04 elapsed=12599s
[llada] step 1100 val_loss=nan