Hugging Face's logo

dignity045
/
grandline

dataset-preprocessing
llm-pretraining
tokenization
deduplication
data-pipeline
ml-intern
Model card Files
xet
 Community
grandline / src
94.2 kB
  • 1 contributor
 History: 3 commits
dignity045's picture
dignity045
Add selective HF parquet shard download support (--hf-files, --hf-subdir, --max-shards, --list-shards)
ab68c56 verified