| # CLAUDE.md |
|
|
| This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
|
|
| ## Project Overview |
|
|
| Tech Advisor: a Gradio chatbot fine-tuned on AWS DevOps Agent documentation. Uses NVIDIA Llama-3.1-Nemotron-Nano-4B-v1.1 as the base model, fine-tuned with QLoRA. Deployed to Hugging Face Spaces with ZeroGPU (no cloud APIs at inference). |
|
|
| ## Architecture |
|
|
| - `app.py` — Gradio text chat interface using `@spaces.GPU` decorator for ZeroGPU. Loads model via transformers `AutoModelForCausalLM` + `AutoTokenizer`. Text-only (no vision). |
| - `training/` — Offline pipeline (runs on EC2 with GPU, not in the Space): |
| - `prepare_data.py` — Converts raw markdown docs in `training/data/raw/` into instruction-response JSONL pairs |
| - `train.py` — QLoRA fine-tuning (4-bit NF4, LoRA r=16 on q/k/v/o projections) |
| - `push_to_hub.py` — Merges adapter into base model and pushes to HF Hub |
|
|
| ## Commands |
|
|
| ```bash |
| # Run the Gradio app locally (requires GPU or will be very slow on CPU) |
| python app.py |
| |
| # Training pipeline (run on GPU instance) |
| python training/prepare_data.py # raw docs → training/data/train.jsonl |
| pip install -r training/requirements.txt |
| python training/train.py # QLoRA fine-tune → training/output/ |
| python training/push_to_hub.py # merge + push to HF Hub |
| ``` |
|
|
| ## Deployment |
|
|
| Two git remotes: |
| - `origin` → GitHub |
| - `space` → Hugging Face Space (`git push space main` to deploy) |
|
|
| Hardware: ZeroGPU in HF Spaces. No secrets or API keys needed. |
|
|
| ## Key Configuration |
|
|
| - `MODEL_ID` in `app.py` controls which model is loaded at inference (base or fine-tuned) |
| - `HUB_REPO` in `training/push_to_hub.py` is the target HF repo for the merged model |
| - Training hyperparams in `training/train.py`: epochs=3, batch=2, grad_accum=8, lr=2e-4, max_seq_length=4096 |
| |
