Buckets:
| # E5 on the procedural testbed: CoT notes (gold-informed, 8k train + 3.5k test oracle) -> train -> eval (written + oracle notes). | |
| cd /workspace/think-decide | |
| export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True | |
| P=/workspace/data/procedural/canonical | |
| V="env PATH=/root/venvs/vllm/bin:$PATH /root/venvs/vllm/bin/python" | |
| $V src/gen_notes.py --train $P/train.jsonl --first 8000 --style cot --gpu-mem 0.85 --out runs/p5/notes.jsonl > logs/p5_notes.log 2>&1 | |
| $V src/gen_notes.py --train $P/test.jsonl --first 3484 --style cot --gpu-mem 0.85 --out runs/p5/notes_test_oracle.jsonl > logs/p5_notes_test.log 2>&1 | |
| python src/e5.py train --out runs/p5 --train-file $P/train.jsonl --train-limit 32000 --max-length 8192 --T 128 \ | |
| --notes runs/p5/notes.jsonl --lr-bb 2.5e-5 --budget 40000 > logs/p5_train.log 2>&1 | |
| python src/e5.py eval --model runs/p5/model.safetensors --split proc_test --ks 3,7 --T 128 --out runs/p5/proc_test > logs/p5_eval.log 2>&1 | |
| python src/e5.py eval --model runs/p5/model.safetensors --split proc_test --T 128 --oracle-notes runs/p5/notes_test_oracle.jsonl \ | |
| --out runs/p5/proc_test_oracle >> logs/p5_eval.log 2>&1 | |
| grep -E "unmask|oracle" logs/p5_eval.log | |
Xet Storage Details
- Size:
- 1.18 kB
- Xet hash:
- 593f05c03f992128c74efc1ffb611a7c2d41608b2ff0d68f29b8f9006600a09b
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.