Spaces:

ASTERIZER
/

LUNA

Running

App Files Files Community

LUNA / Base /scripts

552 kB

Ctrl+K

Ctrl+K

3 contributors

History: 11 commits

BHARGAV REDDY

Upload Base/scripts/deep_clean_sft.py with huggingface_hub

bfebd91 verified 5 months ago

audit_5b_pretrain.py

20.9 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
audit_english_clean.py

30.7 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
audit_litdata.py

20.3 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
augment_sft.py

20 kB
Add SFT pipeline: 302K clean dataset + auto-download pretrained from HF 5 months ago
boost_identity.py

33.5 kB
SFT mega augment + move dataset to HF dataset repo 5 months ago
build_100m_english.py

26.7 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
build_5b_pretrain.py

31.5 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
build_english_1b.py

23.2 kB
Upload Base/scripts/build_english_1b.py with huggingface_hub 5 months ago
build_english_corpus.py

12.4 kB
Upload Base/scripts/build_english_corpus.py with huggingface_hub 5 months ago
build_english_curriculum_1b.py

21.6 kB
Upload Base/scripts/build_english_curriculum_1b.py with huggingface_hub 5 months ago
build_instruct_dataset.py

35.2 kB
Upload Base/scripts/build_instruct_dataset.py with huggingface_hub 5 months ago
chat.py

2.74 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
clean_and_merge_pretrain.py

21.2 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
clean_english_1b.py

23.7 kB
Upload Base/scripts/clean_english_1b.py with huggingface_hub 5 months ago
consolidate_datasets.py

8.55 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
consolidate_litdata.py

8.16 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
dedup_5b_pretrain.py

10.2 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
deep_clean_sft.py

24.2 kB
Upload Base/scripts/deep_clean_sft.py with huggingface_hub 5 months ago
filter_datasets.py

4.68 kB
Upload Base/scripts/filter_datasets.py with huggingface_hub 5 months ago
mega_augment_sft.py

43.7 kB
SFT mega augment + move dataset to HF dataset repo 5 months ago
merge_and_build.py

29.5 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
prepare_finetune_data.py

3 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
prepare_litdata.py

14.5 kB
Upload Base/scripts/prepare_litdata.py with huggingface_hub 5 months ago
reclean_3b.py

23 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
reclean_english.py

24.1 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
reclean_litdata.py

15.4 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
smart_cleanup_english.py

14.8 kB
LUNA 100M: cloud-ready training pipeline 5 months ago
validate_model.py

4.33 kB
LUNA 100M: cloud-ready training pipeline 5 months ago