Initial deployment

.gitattributes

@@ -0,0 +1 @@
+*.txt filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,7 @@
+__pycache__/
+*.pyc
+*.pyo
+.env
+.DS_Store
+student_manual_faiss/
+*.log

Dockerfile

@@ -0,0 +1,32 @@
+# Dockerfile for KUMPAS RAG — Hugging Face Spaces
+FROM python:3.11-slim
+
+# System dependencies for faiss-cpu and sentence-transformers
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    curl \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+# Working directory
+WORKDIR /app
+
+# Copy requirements first (Docker layer caching — faster rebuilds)
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application files
+COPY app.py .
+
+# Copy your Harry Potter text file
+COPY harrypotter.txt .
+
+# HF Spaces runs on port 7860
+EXPOSE 7860
+
+# Predownload the embedding model at build time
+# (so the first request isn't slow)
+RUN python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"
+
+# Start the server
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860", "--workers", "1"]

HF_DEPLOYMENT_GUIDE.md

@@ -0,0 +1,275 @@
+# Deploying KUMPAS RAG to Hugging Face Spaces
+## Complete Step-by-Step Guide
+
+---
+
+## What You Need Before Starting
+- [ ] A Hugging Face account (free) — huggingface.co
+- [ ] Your Groq API key — console.groq.com
+- [ ] Git installed on your computer
+- [ ] Your `harrypotter.txt` file (from Google Drive)
+
+---
+
+## PHASE 1 — Hugging Face Account & Token
+
+### 1.1 Create Account
+1. Go to **https://huggingface.co**
+2. Click **Sign Up** → fill in username, email, password
+3. Verify your email
+
+### 1.2 Get Your Access Token
+1. Click your profile picture (top right) → **Settings**
+2. Left sidebar → **Access Tokens**
+3. Click **New token**
+   - Name: `kumpas-deploy`
+   - Role: **Write**
+4. Click **Generate token** → **Copy it** (you'll need it in a moment)
+
+---
+
+## PHASE 2 — Create the Space
+
+### 2.1 New Space
+1. Click your profile picture → **New Space**
+2. Fill in:
+   - **Owner**: your username
+   - **Space name**: `kumpas-rag` (URL will be `yourname-kumpas-rag.hf.space`)
+   - **License**: MIT
+   - **SDK**: **Docker** ← IMPORTANT: choose Docker, not Gradio/Streamlit
+   - **Visibility**: Public (free) or Private (requires Pro)
+3. Click **Create Space**
+
+You'll land on an empty space page — that's normal.
+
+---
+
+## PHASE 3 — Set Up the Secret (Groq API Key)
+
+Do this BEFORE pushing code, so it's available when the app builds.
+
+1. In your Space page → **Settings** tab (top right of the space)
+2. Scroll down to **Repository secrets**
+3. Click **New secret**
+   - **Name**: `GROQ_API_KEY`  ← must be exactly this name
+   - **Value**: paste your Groq API key (`gsk_...`)
+4. Click **Add secret**
+
+---
+
+## PHASE 4 — Prepare Files on Your Computer
+
+### 4.1 Your folder should look like this:
+```
+kumpas-hf-space/
+├── app.py              ← from the zip
+├── Dockerfile          ← from the zip
+├── requirements.txt    ← from the zip
+├── README.md           ← from the zip
+├── .gitignore          ← from the zip
+└── harrypotter.txt     ← YOU add this (download from Google Drive)
+```
+
+### 4.2 Download harrypotter.txt from Google Drive
+1. Open Google Drive → find `harrypotter.txt`
+2. Right-click → **Download**
+3. Move the downloaded file into your `kumpas-hf-space/` folder
+
+---
+
+## PHASE 5 — Push to Hugging Face via Git
+
+Open your terminal (Command Prompt on Windows, Terminal on Mac/Linux).
+
+### 5.1 Navigate to your folder
+```bash
+cd path/to/kumpas-hf-space
+```
+
+### 5.2 Install Git LFS (for large files like harrypotter.txt)
+```bash
+# Windows (if not installed):
+# Download from https://git-lfs.com
+
+# Mac:
+brew install git-lfs
+
+# Ubuntu/Linux:
+sudo apt-get install git-lfs
+
+# Then enable it:
+git lfs install
+```
+
+### 5.3 Initialize and push
+```bash
+# Initialize git
+git init
+
+# Track large text files with LFS
+git lfs track "*.txt"
+git add .gitattributes
+
+# Add all your files
+git add .
+git commit -m "Initial KUMPAS RAG deployment"
+
+# Connect to your HF Space
+# Replace YOUR_USERNAME with your actual HF username
+git remote add origin https://YOUR_USERNAME:YOUR_HF_TOKEN@huggingface.co/spaces/YOUR_USERNAME/kumpas-rag
+
+# Push!
+git push -u origin main
+```
+
+> **Example:** If your username is `jdelacruz` and your token starts with `hf_abc`:
+> ```
+> git remote add origin https://jdelacruz:hf_abc123@huggingface.co/spaces/jdelacruz/kumpas-rag
+> ```
+
+---
+
+## PHASE 6 — Watch It Build
+
+1. Go back to your Space page on huggingface.co
+2. Click the **Logs** tab — you'll see the Docker build in real time:
+
+```
+Step 1/10 : FROM python:3.11-slim
+Step 2/10 : RUN apt-get update...
+Step 3/10 : COPY requirements.txt .
+Step 4/10 : RUN pip install...        ← takes 3-5 minutes
+Step 5/10 : COPY app.py .
+Step 6/10 : COPY harrypotter.txt .
+Step 7/10 : RUN python -c "...SentenceTransformer..."  ← downloads model
+...
+```
+
+3. After the build succeeds (~5-8 minutes), the app starts:
+```
+INFO: Loading embeddings model...
+INFO: Building FAISS index from harrypotter.txt...  ← first run only, ~2 min
+INFO: Created 14222 chunks
+INFO: QA chain ready ✅
+INFO: 🔮 Server ready!
+INFO: Uvicorn running on http://0.0.0.0:7860
+```
+
+4. The status indicator turns **green** — your Space is live! 🎉
+
+---
+
+## PHASE 7 — Get Your Permanent URL & Test It
+
+### 7.1 Your URL
+```
+https://YOUR_USERNAME-kumpas-rag.hf.space
+```
+Example: `https://jdelacruz-kumpas-rag.hf.space`
+
+**This URL never changes.** Unlike ngrok, it's permanent.
+
+### 7.2 Test in browser
+```
+https://jdelacruz-kumpas-rag.hf.space/health
+```
+Should return:
+```json
+{"status": "ok", "oracle": "ready", "model": "llama-3.3-70b-versatile"}
+```
+
+### 7.3 Test the /ask endpoint
+Open: `https://jdelacruz-kumpas-rag.hf.space/docs`
+
+This opens Swagger UI. Click **POST /ask** → **Try it out** → paste:
+```json
+{
+  "question": "Who is Voldemort?",
+  "history": []
+}
+```
+Click **Execute** → you should get a proper answer!
+
+---
+
+## PHASE 8 — Update Your ASP.NET App
+
+Open `appsettings.json` and update permanently:
+
+```json
+{
+  "Rag": {
+    "BaseUrl": "https://YOUR_USERNAME-kumpas-rag.hf.space",
+    "ApiKey": ""
+  }
+}
+```
+
+Restart your ASP.NET app → open the chatbot → it now uses the always-on server! ✅
+
+---
+
+## Important Notes About Free HF Spaces
+
+| Thing | Detail |
+|---|---|
+| **Sleep after inactivity** | Free spaces sleep after ~15 min of no traffic. First request after sleep takes ~30 sec to wake up. |
+| **CPU only** | Free tier is CPU — FAISS and inference still work, just no GPU speedup. |
+| **Storage** | 50GB limit — your files are well under this. |
+| **Build time** | ~5-8 min on first build. Rebuilds on every `git push`. |
+| **Always-on** | Upgrade to HF Pro ($9/month) to disable sleep. |
+
+### Handling the "sleep" wake-up delay
+The first chatbot message after inactivity will take ~30 seconds. You can handle this gracefully in the chatbot widget — in `chatbot.js`, the timeout is already set to 30 seconds in `RagService.cs`. The user will see the typing dots while it wakes up.
+
+---
+
+## Updating Your Code Later
+
+Whenever you want to update the RAG server (e.g. change the prompt):
+
+```bash
+# Edit app.py locally, then:
+git add app.py
+git commit -m "update prompt"
+git push
+```
+
+HF Spaces auto-rebuilds on every push — takes ~3-5 minutes.
+
+---
+
+## Troubleshooting
+
+### ❌ Build fails: "GROQ_API_KEY not set"
+- You forgot Phase 3. Go to Space Settings → Repository secrets → add `GROQ_API_KEY`.
+- Then go to **Factory reboot** in Space settings to trigger a fresh build.
+
+### ❌ "harrypotter.txt not found"
+- The file wasn't pushed. Check it's in your folder: `ls -la` should show it.
+- Re-run `git add . && git commit -m "add text" && git push`
+
+### ❌ Space shows "Runtime error"
+- Click **Logs** tab → read the error message.
+- Most common cause: missing file or wrong env var name.
+
+### ❌ Push rejected: file too large
+- `harrypotter.txt` might be >10MB. HF needs Git LFS for that.
+- Make sure you ran `git lfs track "*.txt"` before `git add`.
+
+### ❌ Answers are slow (10-20 seconds)
+- Normal for free CPU tier with the LLaMA 70B model via Groq API.
+- Groq's own latency is ~2-3 seconds. The rest is FAISS retrieval + network.
+- Perfectly acceptable for a chatbot.
+
+---
+
+## Summary: The 3 Files You Must Have
+
+| File | Purpose |
+|---|---|
+| `app.py` | The FastAPI server (already written for you) |
+| `Dockerfile` | Tells HF how to build the container |
+| `requirements.txt` | Python dependencies |
+| `harrypotter.txt` | Your HP corpus — YOU must add this |
+| `README.md` | Space description + metadata |

README.md

@@ -0,0 +1,46 @@
+---
+title: KUMPAS Harry Potter RAG Oracle
+emoji: 🔮
+colorFrom: purple
+colorTo: yellow
+sdk: docker
+pinned: false
+app_port: 7860
+---
+
+# 🔮 KUMPAS — Wizarding Oracle RAG API
+
+A Harry Potter RAG (Retrieval-Augmented Generation) API built for the KUMPAS web app.
+
+## Stack
+- **Vector Store**: FAISS with `all-MiniLM-L6-v2` embeddings
+- **LLM**: Groq `llama-3.3-70b-versatile`
+- **Framework**: FastAPI
+
+## Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/health` | Server status |
+| POST | `/ask` | Ask the Oracle |
+
+## Request Format
+
+```json
+POST /ask
+{
+  "question": "Who is Harry Potter's best friend?",
+  "history": [
+    { "role": "user", "content": "previous question" },
+    { "role": "assistant", "content": "previous answer" }
+  ]
+}
+```
+
+## Response Format
+
+```json
+{
+  "answer": "Harry Potter's best friend is Ron Weasley..."
+}
+```

app.py

@@ -0,0 +1,202 @@
+# app.py — KUMPAS RAG Server for Hugging Face Spaces
+# Stack: FAISS + HuggingFace all-MiniLM-L6-v2 + Groq llama-3.3-70b
+
+import os
+import logging
+from contextlib import asynccontextmanager
+from typing import List, Optional
+
+from fastapi import FastAPI, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel
+
+from langchain_community.embeddings import HuggingFaceEmbeddings
+from langchain_community.vectorstores import FAISS
+from langchain_groq import ChatGroq
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.runnables import RunnablePassthrough
+from langchain_core.output_parsers import StrOutputParser
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# ── Global chain (loaded once at startup) ───────────────────────
+qa_chain = None
+
+def build_chain():
+    """
+    Loads the FAISS index from disk and builds the QA chain.
+    Called once when the server starts.
+    """
+    global qa_chain
+
+    groq_api_key = os.environ.get("GROQ_API_KEY", "")
+    if not groq_api_key:
+        raise RuntimeError("GROQ_API_KEY environment variable is not set!")
+
+    faiss_path = os.environ.get("FAISS_PATH", "student_manual_faiss")
+    text_path  = os.environ.get("TEXT_PATH",  "harrypotter.txt")
+
+    logger.info("Loading embeddings model...")
+    embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
+
+    # ── Try loading saved FAISS index first (fast) ──────────────
+    if os.path.exists(faiss_path):
+        logger.info(f"Loading FAISS index from {faiss_path}...")
+        vectorstore = FAISS.load_local(
+            faiss_path,
+            embeddings,
+            allow_dangerous_deserialization=True
+        )
+        logger.info("FAISS index loaded from disk ✅")
+
+    # ── Otherwise build from text file (slow, first run only) ───
+    elif os.path.exists(text_path):
+        logger.info(f"Building FAISS index from {text_path} (this takes a few minutes)...")
+        from langchain_text_splitters import RecursiveCharacterTextSplitter
+
+        try:
+            with open(text_path, "r", encoding="latin-1") as f:
+                text = f.read()
+        except UnicodeDecodeError:
+            with open(text_path, "r", encoding="cp1252") as f:
+                text = f.read()
+
+        splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
+        chunks   = splitter.split_text(text)
+        logger.info(f"Created {len(chunks)} chunks")
+
+        vectorstore = FAISS.from_texts(chunks, embedding=embeddings)
+        vectorstore.save_local(faiss_path)
+        logger.info("FAISS index built and saved ✅")
+
+    else:
+        raise RuntimeError(
+            f"Neither FAISS index ({faiss_path}) nor text file ({text_path}) found! "
+            "Make sure harrypotter.txt is in your Space files."
+        )
+
+    # ── Build retriever ──────────────────────────────────────────
+    retriever = vectorstore.as_retriever(
+        search_type="similarity",
+        search_kwargs={"k": 3}
+    )
+
+    # ── Groq LLM ────────────────────────────────────────────────
+    llm = ChatGroq(
+        model="llama-3.3-70b-versatile",
+        temperature=0.5,
+        groq_api_key=groq_api_key
+    )
+
+    # ── Prompt (same as your Colab notebook) ────────────────────
+    prompt = ChatPromptTemplate.from_template("""
+Answer the question based only on the following context from the Harry Potter series:
+{context}
+
+Question: {question}
+
+Answer clearly and concisely. If the answer is not in the context, say so honestly.
+""")
+
+    # ── LCEL chain (identical to your Colab) ────────────────────
+    qa_chain = (
+        {"context": retriever, "question": RunnablePassthrough()}
+        | prompt
+        | llm
+        | StrOutputParser()
+    )
+
+    logger.info("QA chain ready ✅")
+    return qa_chain
+
+
+# ── Lifespan: build chain on startup ────────────────────────────
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    logger.info("🔮 Starting KUMPAS RAG server...")
+    build_chain()
+    logger.info("🔮 Server ready!")
+    yield
+    logger.info("Server shutting down.")
+
+
+# ── FastAPI app ──────────────────────────────────────────────────
+app = FastAPI(
+    title="KUMPAS Harry Potter RAG API",
+    description="Wizarding Oracle — FAISS + Groq LLaMA",
+    version="1.0.0",
+    lifespan=lifespan
+)
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_methods=["GET", "POST"],
+    allow_headers=["*"],
+)
+
+
+# ── Request / Response models ────────────────────────────────────
+class HistoryItem(BaseModel):
+    role: str
+    content: str
+
+class AskRequest(BaseModel):
+    question: str
+    history: Optional[List[HistoryItem]] = []
+
+class AskResponse(BaseModel):
+    answer: str
+
+
+# ── Endpoints ────────────────────────────────────────────────────
+@app.get("/")
+def root():
+    return {"message": "🔮 KUMPAS Wizarding Oracle is alive!", "status": "ok"}
+
+@app.get("/health")
+def health():
+    return {
+        "status": "ok",
+        "oracle": "ready" if qa_chain else "loading",
+        "model": "llama-3.3-70b-versatile"
+    }
+
+@app.post("/ask", response_model=AskResponse)
+def ask(req: AskRequest):
+    if qa_chain is None:
+        raise HTTPException(status_code=503, detail="Oracle is still initializing. Try again in a moment.")
+
+    question = req.question.strip()
+    if not question:
+        raise HTTPException(status_code=400, detail="Question cannot be empty.")
+
+    # ── Prepend recent history to give the LLM context ──────────
+    if req.history:
+        recent = req.history[-4:]  # last 2 turns
+        history_text = "\n".join(
+            f"{'User' if h.role == 'user' else 'Oracle'}: {h.content}"
+            for h in recent
+        )
+        full_question = f"Previous conversation:\n{history_text}\n\nCurrent question: {question}"
+    else:
+        full_question = question
+
+    try:
+        logger.info(f"Question: {question[:80]}...")
+        answer = qa_chain.invoke(full_question)
+        answer = str(answer).strip()
+
+        if not answer:
+            answer = "The wizarding archives hold no record of this. Try rephrasing your question."
+
+        logger.info(f"Answer: {answer[:80]}...")
+        return AskResponse(answer=answer)
+
+    except Exception as e:
+        logger.error(f"Chain error: {e}")
+        raise HTTPException(
+            status_code=500,
+            detail="The magic is disrupted. The Oracle cannot answer right now."
+        )

harrypotter.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:565fb203ee98c0002c14de4df4dd1b648254f4d5d64d439574ddd62ca235b3b0
+size 6617234

requirements.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:696c6e9ed85fa80cdb8240845987008f6a18ca05a7d65c534de9a70f73526b04
+size 285