lynnseck/massive-bucket / test_massive.py
lynnseck's picture
download
raw
580 Bytes
from datasets import load_dataset
source = "AmazonScience/massive"
#Individual locales are downloaded from version 1.1 which has Catalan (ca-ES) added in
dataset = load_dataset(source, "en-US", download_mode="force_redownload")
print(dataset)
print(dataset["train"][0])
#Using version 1.1 which has Catalan (ca-ES) added in
dataset = load_dataset(source, "all_v1.1", download_mode="force_redownload")
print(dataset)
print(dataset["train"][0])
#Using version 1.0
dataset = load_dataset(source, "all", download_mode="force_redownload")
print(dataset)
print(dataset["train"][0])

Xet Storage Details

Size:
580 Bytes
·
Xet hash:
032bcac4338ca05217f036e69d036f08441ec0211be87a1375d4f9abe8d0984a

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.