Buckets:
| # Queue 6: wait for P2s (pid 51702) -> MQ-soft (32k) -> MQ-large (216k states = 60% of the 360k train states; soft | |
| # or plain chosen by scripts/pick_large.py) -> E5-proc. | |
| cd /workspace/think-decide | |
| export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True | |
| while kill -0 51702 2>/dev/null; do sleep 15; done | |
| python src/mq.py train --out runs/mqsoft --soft > logs/mqsoft_train.log 2>&1 | |
| python src/mq.py eval --model runs/mqsoft/model.safetensors --out runs/mqsoft/test --soft-k 4 --compile > logs/mqsoft_eval.log 2>&1 | |
| SOFT=$(python scripts/pick_large.py 2> logs/pick_large.log) | |
| python src/mq.py train --out runs/mqlarge $SOFT --n-states 216000 > logs/mqlarge_train.log 2>&1 | |
| python src/mq.py eval --model runs/mqlarge/model.safetensors --out runs/mqlarge/test --soft-k 4 --compile > logs/mqlarge_eval.log 2>&1 | |
| scripts/run_e5_proc.sh > logs/p5_run.out 2>&1 | |
| echo QUEUE6_DONE > logs/queue6.done | |
Xet Storage Details
- Size:
- 899 Bytes
- Xet hash:
- b19164dc42d0f91185e45e807621c73dcd7be811527ff2b0a67419c966206863
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.