CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Tech Advisor: a Gradio chatbot fine-tuned on AWS DevOps Agent documentation. Uses NVIDIA Llama-3.1-Nemotron-Nano-4B-v1.1 as the base model, fine-tuned with QLoRA. Deployed to Hugging Face Spaces with ZeroGPU (no cloud APIs at inference).

Architecture

• app.py — Gradio text chat interface using @spaces.GPU decorator for ZeroGPU. Loads model via transformers AutoModelForCausalLM + AutoTokenizer. Text-only (no vision).
• training/ — Offline pipeline (runs on EC2 with GPU, not in the Space):
  • prepare_data.py — Converts raw markdown docs in training/data/raw/ into instruction-response JSONL pairs
  • train.py — QLoRA fine-tuning (4-bit NF4, LoRA r=16 on q/k/v/o projections)
  • push_to_hub.py — Merges adapter into base model and pushes to HF Hub

Commands

# Run the Gradio app locally (requires GPU or will be very slow on CPU)
python app.py

# Training pipeline (run on GPU instance)
python training/prepare_data.py          # raw docs → training/data/train.jsonl
pip install -r training/requirements.txt
python training/train.py                 # QLoRA fine-tune → training/output/
python training/push_to_hub.py           # merge + push to HF Hub

Deployment

Two git remotes:

• origin → GitHub
• space → Hugging Face Space (git push space main to deploy)

Hardware: ZeroGPU in HF Spaces. No secrets or API keys needed.

Key Configuration

• MODEL_ID in app.py controls which model is loaded at inference (base or fine-tuned)
• HUB_REPO in training/push_to_hub.py is the target HF repo for the merged model
• Training hyperparams in training/train.py: epochs=3, batch=2, grad_accum=8, lr=2e-4, max_seq_length=4096