File size: 1,842 Bytes
975da6d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | # CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Tech Advisor: a Gradio chatbot fine-tuned on AWS DevOps Agent documentation. Uses NVIDIA Llama-3.1-Nemotron-Nano-4B-v1.1 as the base model, fine-tuned with QLoRA. Deployed to Hugging Face Spaces with ZeroGPU (no cloud APIs at inference).
## Architecture
- `app.py` — Gradio text chat interface using `@spaces.GPU` decorator for ZeroGPU. Loads model via transformers `AutoModelForCausalLM` + `AutoTokenizer`. Text-only (no vision).
- `training/` — Offline pipeline (runs on EC2 with GPU, not in the Space):
- `prepare_data.py` — Converts raw markdown docs in `training/data/raw/` into instruction-response JSONL pairs
- `train.py` — QLoRA fine-tuning (4-bit NF4, LoRA r=16 on q/k/v/o projections)
- `push_to_hub.py` — Merges adapter into base model and pushes to HF Hub
## Commands
```bash
# Run the Gradio app locally (requires GPU or will be very slow on CPU)
python app.py
# Training pipeline (run on GPU instance)
python training/prepare_data.py # raw docs → training/data/train.jsonl
pip install -r training/requirements.txt
python training/train.py # QLoRA fine-tune → training/output/
python training/push_to_hub.py # merge + push to HF Hub
```
## Deployment
Two git remotes:
- `origin` → GitHub
- `space` → Hugging Face Space (`git push space main` to deploy)
Hardware: ZeroGPU in HF Spaces. No secrets or API keys needed.
## Key Configuration
- `MODEL_ID` in `app.py` controls which model is loaded at inference (base or fine-tuned)
- `HUB_REPO` in `training/push_to_hub.py` is the target HF repo for the merged model
- Training hyperparams in `training/train.py`: epochs=3, batch=2, grad_accum=8, lr=2e-4, max_seq_length=4096
|