| python3 data/build_ruler_data.py \ | |
| --ruler_dir /data/ldeng/code/kvbridge/QCFuse/third_party/RULER \ | |
| --raw_dir /data/ldeng/code/kvbridge/QCFuse/data/ruler_raw \ | |
| --output_dir /data/ldeng/code/kvbridge/QCFuse/data/final_data \ | |
| --tokenizer_path /public/models/Qwen3-8B \ | |
| --num_samples 200 \ | |
| --chunk_size 512 \ | |
| --target_num_chunks 20 \ | |
| --ruler_max_seq_length 11264 | |
Xet Storage Details
- Size:
- 376 Bytes
- Xet hash:
- 7e18779ac2f34212bd1e72f29415045708b4c27cbbe2cf04176aa8d8a457f469
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.