Buckets:

the-hf-stack
/

dagster-hf-datasets-examples

19 days ago

94 Bytes

	from distributed_token_sharding.assets import (
	fineweb_dataset,
	tokenized_fineweb,
	)

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.