ASTERIZER commited on
Commit
6462a62
·
verified ·
1 Parent(s): b7c73ba

Upload Base/Datasets/rag_mcp_sft/FINETUNE_COMMANDS.md with huggingface_hub

Browse files
Base/Datasets/rag_mcp_sft/FINETUNE_COMMANDS.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # LoRA Finetune Commands
2
+
3
+ These commands assume you are running from a PowerShell terminal on Windows.
4
+
5
+ ## 1. Clone the workspace repo
6
+
7
+ ```powershell
8
+ git clone <your-luna-repo-url>
9
+ cd LUNA
10
+ ```
11
+
12
+ ## 2. Create the environment and install dependencies
13
+
14
+ ```powershell
15
+ python -m venv .venv
16
+ .\.venv\Scripts\Activate.ps1
17
+ python -m pip install --upgrade pip
18
+ pip install -r requirements.txt
19
+ ```
20
+
21
+ ## 3. Build the RAG + MCP dataset locally
22
+
23
+ ```powershell
24
+ python .\Base\Datasets\rag_mcp_sft\build_rag_mcp_sft_dataset.py --target-tokens 10000000
25
+ ```
26
+
27
+ ## 4. Push the dataset to Hugging Face
28
+
29
+ ```powershell
30
+ $env:HF_TOKEN = "<your_hf_token>"
31
+ python .\Base\Datasets\rag_mcp_sft\push_to_hf.py --repo-id ASTERIZER/LUNA-RAG-MCP-SFT-10M
32
+ ```
33
+
34
+ ## 5. Train LoRA adapters on top of the current LUNA SFT checkpoint from Hugging Face
35
+
36
+ The config already points to the user-requested base checkpoint file:
37
+
38
+ - repo: `ASTERIZER/LUNA-100M`
39
+ - file: `sft_v1/final/model.pth`
40
+
41
+ Run:
42
+
43
+ ```powershell
44
+ $env:HF_TOKEN = "<your_hf_token>"
45
+ python .\lora_sft_train.py --config .\rag_mcp_lora_config.yaml
46
+ ```
47
+
48
+ ## 6. Optional overrides
49
+
50
+ ```powershell
51
+ python .\lora_sft_train.py --config .\rag_mcp_lora_config.yaml --epochs 3
52
+ python .\lora_sft_train.py --config .\rag_mcp_lora_config.yaml --out_dir .\Base\out\sft\rag_mcp_lora_exp2
53
+ python .\lora_sft_train.py --config .\rag_mcp_lora_config.yaml --pretrained_ckpt .\Base\out\input_models\luna_sft_v1\model.pth
54
+ ```