implementation_plan.md

# Implementation Plan: Fine-Tuning GPT-2 on High-Quality SFT Dataset (Revised)

We will perform a new fine-tuning run using the high-quality **`HuggingFaceH4/no_robots`** SFT dataset. We will first train on a subset to verify training dynamics and output coherence, and then execute training on the full dataset.

---

## User Review Required

> [!IMPORTANT]
> **Please review the following clarifications and confirm approval to begin the autonomous run:**

### 1. Continual Pre-training (FineWeb) vs. Direct SFT (no_robots)
You asked: *Is it a good idea to SFT on a high quality document dataset, like fineweb-2, then IF on no_robots?*
- **Our Recommendation**: We recommend **direct SFT on `no_robots`** from the base GPT-2 model.
- **Why**: Continual pre-training (on raw document datasets like FineWeb) is highly compute-intensive and requires billions of tokens to avoid "catastrophic forgetting" of the model's base language modeling capability. For a 124M model like GPT-2 under a 12-hour limit, training on FineWeb documents first would consume significant time and risk degrading the model's coherence or causing it to forget basic grammar before SFT even starts. Instruction-following (IF) alignment directly on `no_robots` is the most reliable way to teach it conversational structure within our timeframe.

### 2. Disk Space Cleanup Strategy
- **Baseline**: Currently, we have 21 GB of free disk space. However, `/home/maxgn/.cache/huggingface/` contains **35 GB** of cached data (29 GB in `hub`, 6.2 GB in `datasets`).
- **Action**: Before and during training, we will monitor disk space. If free space drops below **12 GB**, we will clean up old caches (specifically unused hub models and datasets) to keep the host healthy.

### 3. Updated Test Split & Verification Formats
- **Test Split**: Adjusted to `train[:2500]` per your feedback.
- **Coherence Verification Formats**: Added **`IQ3_XXS`** to our test formats alongside `IQ4_NL` and `Q8_0`.

---

## Proposed Changes

### Configuration Changes

#### [MODIFY] [axolotl-gpt2-chatml-fp32.yml](file:///home/maxgn/gpt2-chatml-web-chat/axolotl-gpt2-chatml-fp32.yml)
Update the dataset configuration to point to `HuggingFaceH4/no_robots`, map its standard fields, and direct outputs to a new dedicated folder:

```yaml
datasets:
  - path: HuggingFaceH4/no_robots
    split: train[:2500] # Set to train[:2500] for test split, train for full run
    type: chat_template
    field_messages: messages
    message_property_mappings:
      role: role
      content: content
    roles_to_train:
      - assistant
    train_on_eos: turn
    default_system_message: "You are a helpful, concise assistant. Keep answers short and clear."

output_dir: /home/maxgn/outputs/gpt2-chatml-no-robots
```

---

## Execution Plan (Autonomous Execution)

Once approved, we will execute the following steps:

### Phase 1: Cache Setup & Pre-Flight Check
1. Verify free disk space. If needed, clear out old Hugging Face hub checkpoints to free up extra headroom.
2. Update `axolotl-gpt2-chatml-fp32.yml` to target the `train[:2500]` split.

### Phase 2: Test Split Training
1. Launch training using the background command:
   ```bash
   export AXOLOTL_DO_NOT_TRACK=1 && source /home/maxgn/axolotl/.venv/bin/activate && accelerate launch --module --dynamo_backend no axolotl.cli.train /home/maxgn/gpt2-chatml-web-chat/axolotl-gpt2-chatml-fp32.yml --debug=False --debug-text-only=False --debug-num-examples=0 --shard=False
   ```
2. Monitor validation loss in `debug.log`.

### Phase 3: Test Split GGUF Quantization & Verification
1. Convert the test checkpoint to GGUF format: `gpt2-f32.gguf`.
2. Generate an imatrix file on the `calibration-data.txt` file.
3. Quantize the model into **`Q8_0`**, **`IQ4_NL`**, and **`IQ3_XXS`**.
4. Run inference tests on:
   - *Prompt A*: "What is the capital of France?"
   - *Prompt B*: "Why is the sky blue? Explain in one short sentence."
5. **Evaluation**:
   - If output quality is equal/better than our previous run, proceed to **Phase 5**.
   - If output quality is worse (e.g. repetitive loops or structure loss), proceed to **Phase 4** (Troubleshooting).

### Phase 4: Troubleshooting (If needed)
1. Adjust hyperparameters (e.g., reduce learning rate to `2e-5`, increase epochs, or tune repetition penalty).
2. Re-run Phase 2 and 3 until verified.

### Phase 5: Full Dataset Training & Final Quantization
1. Update `axolotl-gpt2-chatml-fp32.yml` to `split: train`.
2. Run full training (~1.5 hours).
3. Convert final checkpoint to GGUF and quantize into **`Q8_0`**, **`IQ4_NL`**, and **`IQ3_XXS`** using imatrix.
4. Perform final coherence verification.

---

## Verification Plan

### Automated Tests
- Run `llama-cli -st` on the quantized models and log responses to confirm coherence.
- Output files and sizes will be verified and logged in `walkthrough.md`.