-
Upload Base/Datasets/rag_mcp_sft/sample_preview.json with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/source_manifest.json with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/README.md with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/FINETUNE_COMMANDS.md with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/BUILD_REPORT.md with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/push_to_hf.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/Datasets/rag_mcp_sft/build_rag_mcp_sft_dataset.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/deep_clean_sft.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/filter_datasets.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/build_instruct_dataset.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/build_english_curriculum_1b.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/build_english_corpus.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/prepare_litdata.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/clean_english_1b.py with huggingface_hub
BHARGAV REDDY
-
Upload Base/scripts/build_english_1b.py with huggingface_hub
BHARGAV REDDY
-
SFT mega augment + move dataset to HF dataset repo
- 580K entries, 140M tokens, 2.9% identity (was 302K/82M/0.15%)
- Dataset now hosted at ASTERIZER/Luna_Dataset (auto-downloads at train start)
- Removed LFS dataset files from Space repo
- Added mega_augment_sft.py, boost_identity.py, chat.py
- Updated sft_train.py with dataset auto-download from HF -
Add SFT pipeline: 302K clean dataset + auto-download pretrained from HF
- sft_train.py: full SFT trainer with prompt masking, checkpoint eval, auto-downloads pretrained from ASTERIZER/LUNA-100M
- sft_config.yaml: epochs=2, lr=1.5e-5, batch=64, 10 eval prompts
- setup_and_sft.sh: one-command cloud SFT setup
- 302,785 SFT entries (82M tokens), LFS-tracked JSON
- Dataset scripts: deep_clean_sft.py, augment_sft.py
-
LUNA 100M: cloud-ready training pipeline
- Config-driven train.py with auto/manual hardware detection
- train_config.yaml with auto_config toggle
- setup_and_train.sh: one-command RunPod deployment
- fetch_data.py: HuggingFace/GDrive dataset downloader
- benchmark_runpod.py: local benchmark + cloud cost estimator
- Tokenizer files, data preprocessing scripts, GGUF converter