Upload 10 files

Readme.md

@@ -0,0 +1,746 @@
+# 🤖 AI Codebase Assistant — Complete Documentation
+
+> An AI-powered developer tool that ingests any local code repository, answers questions about it using RAG (Retrieval-Augmented Generation), detects bugs, measures complexity, generates docs, proposes new files, and integrates with the GitHub MCP API — all from an interactive CLI.
+
+---
+
+## Table of Contents
+
+1. [Project Overview](#1-project-overview)
+2. [Repository Structure](#2-repository-structure)
+3. [Architecture & Flow](#3-architecture--flow)
+4. [Component Deep-Dive](#4-component-deep-dive)
+5. [Setup & Installation](#5-setup--installation)
+6. [Configuration (.env)](#6-configuration-env)
+7. [Running the CLI](#7-running-the-cli)
+8. [CLI Features — All 9 Options](#8-cli-features--all-9-options)
+9. [REST API](#9-rest-api)
+10. [Key Design Decisions & Bug Fixes](#10-key-design-decisions--bug-fixes)
+11. [Extending the Project](#11-extending-the-project)
+
+---
+
+## 1. Project Overview
+
+The **AI Codebase Assistant** is a local developer tool that:
+
+- 📂 **Ingests** any code repository (Python, JS, TS, Java, Go, Markdown)
+- 🔍 **Answers questions** about the code using RAG + LLM
+- 🐛 **Detects bugs** via LLM-powered code review
+- 📊 **Measures cyclomatic complexity** using Radon
+- 📝 **Explains functions** in plain English
+- 📄 **Generates module docs and READMEs** automatically
+- 🛠️ **Proposes & creates new files** using AI, saved to your chosen directory
+- 🔗 **Lists 44 GitHub MCP tools** via the GitHub Copilot MCP API
+
+**LLM Providers supported:**
+| Provider | Model | Notes |
+|---|---|---|
+| Groq | `llama-3.3-70b-versatile` | Free tier, recommended |
+| Gemini | Configurable | Paid, stricter quota |
+
+**Embedding:** Google Gemini Embedding API (`models/gemini-embedding-001`)  
+**Vector Store:** ChromaDB (local persistent)
+
+---
+
+## 2. Repository Structure
+
+```
+Codebase Assistant/
+│
+├── cli.py                     # ← Main entry point (interactive CLI)
+├── app.py                     # ← FastAPI REST server (optional)
+├── config.py                  # ← All settings via pydantic-settings + .env
+├── llm.py                     # ← LLM abstraction (Gemini / Groq) + prompt templates
+├── mcp_client.py              # ← GitHub MCP async context-manager client
+├── requirements.txt           # ← All Python dependencies
+├── .env                       # ← API keys and configuration (not committed)
+│
+├── rag/                       # ── RAG Pipeline ──────────────────────────────
+│   ├── repository_loader.py   #   Walk directory, read files → CodeDocument
+│   ├── splitter.py            #   Split CodeDocuments into chunks with metadata
+│   ├── embedding.py           #   Embed chunks/queries via Gemini Embeddings
+│   ├── retriever.py           #   ChromaDB vector store: build / load / query
+│   └── rag_chain.py           #   Orchestrate retrieve → build context → LLM
+│
+├── services/                  # ── Feature Services ──────────────────────────
+│   ├── code_analysis.py       #   Bug detection, complexity, function explainer
+│   ├── documentation.py       #   Module docs, README generator
+│   └── file_creator.py        #   AI file proposal + local disk write
+│
+├── api/                       # ── REST API (FastAPI) ────────────────────────
+│   └── routes.py              #   All HTTP endpoints (mirrors CLI features)
+│
+└── vector_db/                 # ── ChromaDB Storage (auto-created) ───────────
+    └── chroma.sqlite3         #   Persisted vector embeddings
+```
+
+---
+
+## 3. Architecture & Flow
+
+### 3.1 Full System Architecture
+
+```mermaid
+graph TB
+    subgraph USER["User Interface"]
+        CLI["cli.py\n(Interactive CLI)"]
+        API["app.py\n(FastAPI REST API)"]
+    end
+
+    subgraph RAG["RAG Pipeline"]
+        RL["repository_loader.py\nWalk & read files"]
+        SP["splitter.py\nChunk by language"]
+        EM["embedding.py\nGemini Embeddings"]
+        RT["retriever.py\nChromaDB"]
+        RC["rag_chain.py\nOrchestrator"]
+    end
+
+    subgraph SERVICES["Feature Services"]
+        CA["code_analysis.py\nBugs · Complexity · Explain"]
+        DOC["documentation.py\nModule Docs · README"]
+        FC["file_creator.py\nAI File Creator"]
+    end
+
+    subgraph EXTERNAL["External APIs"]
+        GROQ["Groq API\nllama-3.3-70b"]
+        GEM["Gemini API\nEmbeddings"]
+        MCP["GitHub MCP\n44 tools"]
+    end
+
+    LLM["llm.py\nLLM Abstraction Layer"]
+    CFG["config.py\n.env Settings"]
+    DB[("vector_db/\nChromaDB")]
+
+    CLI --> RAG
+    CLI --> SERVICES
+    API --> RAG
+    API --> SERVICES
+
+    RL --> SP --> EM --> RT
+    RT --> DB
+    RC --> RT
+    RC --> LLM
+
+    CA --> LLM
+    DOC --> LLM
+    FC --> LLM
+
+    LLM --> GROQ
+    LLM --> GEM
+    EM --> GEM
+    CLI --> MCP
+
+    CFG -.->|settings| RAG
+    CFG -.->|settings| LLM
+    CFG -.->|settings| SERVICES
+```
+
+### 3.2 Ingest Flow (Step-by-step)
+
+```mermaid
+flowchart LR
+    A["User provides\nrepo path"] --> B["repository_loader.py\nwalk directories\nskip: .git venv __pycache__"]
+    B --> C["Filter by\nallowed extensions\n.py .js .ts .java .go .md"]
+    C --> D["Read each file\n→ CodeDocument\n(content, path, language, size)"]
+    D --> E["splitter.py\nLanguage-aware chunking\nRecursiveCharacterTextSplitter"]
+    E --> F["Each chunk gets\nmetadata: file_path\nlanguage · start_line · end_line"]
+    F --> G["embedding.py\nGemini embed_documents\n→ float vectors"]
+    G --> H["retriever.py\nDelete old collection\nCreate fresh ChromaDB\ncollection.add(...)"]
+    H --> I["✓ Vector store ready"]
+```
+
+### 3.3 RAG Query Flow
+
+```mermaid
+flowchart LR
+    Q["User question"] --> E["embed_query\n(Gemini)"]
+    E --> S["ChromaDB\ncollection.query\ntop-k chunks"]
+    S --> C["build_context\nformat chunks\nwith file/line headers"]
+    C --> P["build_prompt\nqa template\ncontext + question"]
+    P --> L["LLM\n(Groq / Gemini)"]
+    L --> A["Answer + Sources\n(file_path, line range)"]
+```
+
+### 3.4 CLI Menu Flow
+
+```mermaid
+flowchart TD
+    START([Start cli.py]) --> REPO["▶ Enter repo path"]
+    REPO --> INGEST["Ingest Repository\nload → split → embed → store"]
+    INGEST --> MENU["Show Menu\nOptions 1–9"]
+
+    MENU --> O1["1 Ask a question\n→ RAG Query"]
+    MENU --> O2["2 Detect bugs\n→ LLM code review"]
+    MENU --> O3["3 Cyclomatic complexity\n→ Radon"]
+    MENU --> O4["4 Explain function\n→ AST + LLM"]
+    MENU --> O5["5 Module docs\n→ LLM"]
+    MENU --> O6["6 Generate README\n→ LLM"]
+    MENU --> O7["7 Propose & create file\n→ LLM + local write"]
+    MENU --> O8["8 List GitHub MCP tools\n→ GitHub Copilot MCP"]
+    MENU --> O9["9 Re-ingest repo\n→ new repo path"]
+    MENU --> O0["0 Exit"]
+
+    O1 & O2 & O3 & O4 & O5 & O6 & O7 & O8 & O9 --> MENU
+    O0 --> END([Goodbye!])
+```
+
+---
+
+## 4. Component Deep-Dive
+
+### 4.1 `config.py` — Settings
+
+All configuration lives in one `pydantic-settings` class loaded from `.env`:
+
+| Setting | Default | Description |
+|---|---|---|
+| `llm_provider` | `"groq"` | `"groq"` or `"gemini"` |
+| `groq_api_key` | `""` | From `.env` |
+| `groq_model` | `"llama-3.3-70b-versatile"` | Free Groq model |
+| `gemini_api_key` | `""` | From `.env` |
+| `embedding_model` | `"models/gemini-embedding-001"` | Always Gemini for embeddings |
+| `vector_db_path` | `"./vector_db"` | ChromaDB storage directory |
+| `chunk_size` | `1200` | Characters per chunk |
+| `chunk_overlap` | `100` | Overlap between chunks |
+| `allowed_extensions` | `[.py .js .ts .go .java .md]` | File types to load |
+| `max_file_size_kb` | `1042` | Skip files larger than this |
+| `github_mcp_url` | GitHub Copilot MCP endpoint | For option 8 |
+| `github_mcp_token` | `""` | GitHub PAT from `.env` |
+
+### 4.2 `rag/repository_loader.py` — File Ingestion
+
+```
+load_repository(root_path)
+  └── os.walk(root_path)
+        ├── Skip: .git, node_modules, __pycache__, venv, .venv, dist, build
+        ├── should_include(fpath) → checks extension + file size
+        └── read_file_with_metadata(fpath) → CodeDocument(content, file_path, language, size_bytes)
+```
+
+**Supported languages detected by extension:**
+
+| Extension | Language |
+|---|---|
+| `.py` | python |
+| `.js` | javascript |
+| `.ts` | typescript |
+| `.java` | java |
+| `.go` | go |
+| `.md` | markdown |
+
+### 4.3 `rag/splitter.py` — Chunking
+
+Uses **LangChain's `RecursiveCharacterTextSplitter`** with language-aware splitting:
+- For Python/JS/TS/Java/Go — uses syntax-aware boundaries (functions, classes)
+- For Markdown/unknown — falls back to generic character splitting
+- Each chunk carries: `content`, `file_path`, `language`, `chunk_index`, `start_line`, `end_line`
+
+### 4.4 `rag/embedding.py` — Embeddings
+
+- Uses `GoogleGenerativeAIEmbeddings` (`models/gemini-embedding-001`)
+- `embed_document(chunks)` — batch embeds all chunks, mutates dicts in-place
+- `embed_query(query)` — single query vector for similarity search
+
+### 4.5 `rag/retriever.py` — Vector Store
+
+> **Critical fix applied:** collection is deleted and recreated on every ingest to prevent stale data from previous repos bleeding through.
+
+```python
+build_vector_store(chunks)     # delete → create → add (fresh each ingest)
+load_vector_store()            # get_or_create for reading
+retrieve_relevant_chunks(q, k) # embed query → cosine similarity → top-k
+```
+
+### 4.6 `rag/rag_chain.py` — Orchestration
+
+```python
+run_rag_query(query, k=5)
+  1. retrieve_relevant_chunks(query, k)
+  2. build_context(chunks)         # format with file/line headers
+  3. build_prompt(query, context)  # fill qa template
+  4. llm.generate(prompt)
+  5. return { answer, sources }
+```
+
+### 4.7 `llm.py` — LLM Abstraction
+
+Abstract `BaseLLM` with two concrete providers:
+
+| Class | Provider | API |
+|---|---|---|
+| `GeminiLLM` | Google Gemini | `google-genai` SDK |
+| `GroqLLM` | Groq | `groq` SDK, chat completions |
+
+**Prompt templates (`build_prompt`):**
+
+| `task_type` | Used by |
+|---|---|
+| `"qa"` | RAG query, function explain, module docs, README |
+| `"bug_finding"` | Bug detection (returns JSON) |
+| `"docstring"` | Docstring generation |
+| `"file_creation"` | AI file proposal |
+
+### 4.8 `mcp_client.py` — GitHub MCP Client
+
+Async context-manager pattern using `AsyncExitStack` to keep all `anyio` cancel scopes in the **same task**:
+
+```python
+async with get_github_mcp_client() as client:
+    tools = await client.list_tools()
+    result = await client.call_tool("create_branch", {...})
+```
+
+Available GitHub MCP tools (44 total) include: `search_code`, `list_issues`, `create_pull_request`, `get_file_contents`, `push_files`, `create_branch`, `fork_repository`, `search_repositories`, and many more.
+
+### 4.9 `services/code_analysis.py`
+
+| Function | How it works |
+|---|---|
+| `explain_function(file, name)` | Python `ast` extracts the function source → RAG context → LLM |
+| `detect_bugs(file)` | Read file → `bug_finding` prompt → LLM returns JSON list |
+| `analyze_complexity(file)` | `radon.cc_visit` → cyclomatic complexity + rank A–F |
+
+### 4.10 `services/file_creator.py`
+
+```
+propose_new_file(description, context_query)
+  ├── RAG retrieve relevant context
+  ├── build_prompt(description, context, "file_creation")
+  ├── LLM generates complete file content
+  └── infer_file_path(description) → LLM suggests relative path
+
+apply_approved_file(proposal, confirmed, base_dir)
+  ├── Strip leading / or \ from LLM path
+  ├── os.path.join(base_dir, relative_path)
+  ├── os.makedirs(parent_dirs, exist_ok=True)
+  └── open(abs_path, "w").write(content)
+```
+
+---
+
+## 5. Setup & Installation
+
+### Prerequisites
+
+| Requirement | Version |
+|---|---|
+| Python | 3.11+ |
+| pip | Latest |
+| Internet | For API calls |
+
+### Step 1 — Clone / Download the project
+
+```bash
+git clone <your-repo-url>
+cd "Codebase Assistant"
+```
+
+### Step 2 — Create a virtual environment
+
+```bash
+# Windows (PowerShell)
+python -m venv .venv
+.venv\Scripts\Activate.ps1
+
+# macOS / Linux
+python3 -m venv .venv
+source .venv/bin/activate
+```
+
+### Step 3 — Install dependencies
+
+```bash
+pip install -r requirements.txt
+```
+
+> [!TIP]
+> If you hit permission issues on Windows, use:
+> `pip install -r requirements.txt --user`
+
+### Step 4 — Create your `.env` file
+
+Create a file named `.env` in the project root:
+
+```env
+# Choose your LLM provider: "groq" (free) or "gemini"
+LLM_PROVIDER=groq
+
+# Groq — free at https://console.groq.com
+GROQ_API_KEY=gsk_your_key_here
+
+# Gemini — get at https://aistudio.google.com
+GEMINI_API_KEY=your_gemini_key_here
+
+# GitHub PAT for MCP tools — create at https://github.com/settings/tokens
+# Required scopes: repo, read:org
+GITHUB_MCP_TOKEN=github_pat_your_token_here
+```
+
+> [!IMPORTANT]
+> `GEMINI_API_KEY` is **always required** regardless of LLM provider, because embeddings always use Gemini.
+
+### Step 5 — Run the CLI
+
+```bash
+.venv\Scripts\python.exe cli.py     # Windows
+python cli.py                        # macOS / Linux
+```
+
+---
+
+## 6. Configuration (.env)
+
+```env
+# ─── LLM Provider ────────────────────────────────────────
+LLM_PROVIDER=groq          # "groq" | "gemini"
+
+# ─── Groq (recommended — free tier) ─────────────────────
+GROQ_API_KEY=gsk_...
+GROQ_MODEL=llama-3.3-70b-versatile
+
+# ─── Gemini ──────────────────────────────────────────────
+GEMINI_API_KEY=...
+LLM_MODEL=gemini-1.5-flash        # only used if LLM_PROVIDER=gemini
+EMBEDDING_MODEL=models/gemini-embedding-001
+
+# ─── RAG / Vector DB ─────────────────────────────────────
+VECTOR_DB_PATH=./vector_db
+CHUNK_SIZE=1200
+CHUNK_OVERLAP=100
+
+# ─── File loader ─────────────────────────────────────────
+# comma-separated extensions
+ALLOWED_EXTENSIONS=[".py",".js",".ts",".go",".java",".md"]
+MAX_FILE_SIZE_KB=1042
+
+# ─── GitHub MCP ──────────────────────────────────────────
+GITHUB_MCP_URL=https://api.githubcopilot.com/mcp/
+GITHUB_MCP_TOKEN=github_pat_...
+```
+
+---
+
+## 7. Running the CLI
+
+```
+╔══════════════════════════════════════════╗
+║       AI  Codebase  Assistant  CLI       ║
+╚══════════════════════════════════════════╝
+
+▶ Enter the path to the repository you want to analyse: D:\MyProject
+
+── Ingesting Repository ──────────────────────
+  Loading files from: D:\MyProject
+  ✓  Loaded 12 files
+  ✓  Split into 47 chunks
+  Embedding chunks (this may take a moment)...
+  ✓  Embedded 47 chunks
+  ✓  Vector store ready (ChromaDB)
+
+── What would you like to do? ────────────────
+   1  Ask a question about the codebase
+   2  Detect bugs in a file
+   3  Cyclomatic complexity analysis
+   4  Explain a function
+   5  Generate module documentation
+   6  Generate README for a repository
+   7  Propose & create a new file (AI)
+   8  List GitHub MCP tools
+   9  Re-ingest a repository
+    0  Exit
+
+▶ Choose an option [0–9]:
+```
+
+---
+
+## 8. CLI Features — All 9 Options
+
+### Option 1 — Ask a Question (RAG Query)
+
+```
+▶ Your question: How does the authentication work?
+▶ Number of source chunks to retrieve? [default: 5] 3
+
+── Answer ────────────────────────────────────
+The authentication uses JWT tokens ...
+
+── Sources ───────────────────────────────────
+  • src/auth/middleware.py  lines 12–45
+  • src/auth/tokens.py  lines 1–30
+```
+
+**Pipeline:** Embed question → ChromaDB similarity search → Build context → LLM → Answer + cited sources
+
+---
+
+### Option 2 — Detect Bugs
+
+```
+▶ File path to analyse: src/payment.py
+
+── Found 2 issue(s) ──────────────────────────
+  [HIGH] Line 34: SQL query uses string concatenation
+    → Use parameterized queries to prevent SQL injection
+
+  [MEDIUM] Line 67: Exception swallowed silently
+    → Log or re-raise the exception
+```
+
+**Pipeline:** Read file → `bug_finding` prompt → LLM returns JSON → parsed and displayed
+
+---
+
+### Option 3 — Cyclomatic Complexity
+
+```
+▶ File path to analyse: src/processor.py
+
+── 4 function(s) ─────────────────────────────
+  [A]  process_order              complexity=2  ██
+  [B]  validate_cart              complexity=5  █████
+  [C]  apply_discounts            complexity=8  ████████
+  [F]  handle_edge_cases          complexity=18 ██████████████████
+```
+
+**Rank scale:** A (1–5, simple) → F (26+, untestable)
+
+---
+
+### Option 4 — Explain a Function
+
+```
+▶ File path: src/utils.py
+▶ Function name: parse_date_range
+
+── Explanation ───────────────────────────────
+parse_date_range takes a string like "2024-01-01:2024-12-31"
+and returns a tuple of (start_date, end_date) as datetime objects ...
+```
+
+**Pipeline:** Python `ast` module extracts exact function source → RAG retrieves usages → LLM explains
+
+---
+
+### Option 5 — Generate Module Documentation
+
+```
+▶ File path: src/database.py
+
+── Documentation ─────────────────────────────
+## database.py
+
+### Overview
+This module provides the database connection layer ...
+
+### Functions
+- `connect(url)` — Establishes a connection ...
+- `execute(query, params)` — Runs a parameterized query ...
+```
+
+---
+
+### Option 6 — Generate README
+
+```
+▶ Repository root path: D:\MyProject
+
+── README Preview ────────────────────────────
+# MyProject
+
+## Overview
+A FastAPI application that ...
+
+▶ Save to README.md in that directory? [y/N] y
+  ✓  Saved to D:\MyProject\README.md
+```
+
+---
+
+### Option 7 — Propose & Create a New File
+
+```
+▶ Describe the file you want to create: Rectangle area calculator in JavaScript
+▶ Optional context query (or press Enter to skip):
+
+── Proposed File ─────────────────────────────
+  Path:  /src/geometry/rectangle.js
+
+  class Rectangle { ...full generated code... }
+
+▶ Write this file to disk? [y/N] y
+▶ Save under which directory? [default: D:\MyProject]
+
+  ✓  File written: D:\MyProject\src\geometry\rectangle.js
+```
+
+**Pipeline:** RAG context (optional) → `file_creation` prompt → LLM generates code → user confirms → write to `base_dir/relative_path`
+
+---
+
+### Option 8 — List GitHub MCP Tools
+
+```
+── List GitHub MCP Tools ─────────────────────
+  44 tools available:
+    • create_branch                   Create a new branch in a GitHub repository
+    • create_pull_request             Create a new pull request ...
+    • search_code                     Fast and precise code search ...
+    • list_issues                     List issues in a GitHub repository ...
+    ...
+```
+
+**Connection:** Uses `streamable_http_client` → `ClientSession` → GitHub Copilot MCP endpoint
+
+---
+
+### Option 9 — Re-ingest a Repository
+
+```
+▶ New repository path to ingest: D:\AnotherProject
+── Ingesting Repository ──────────────────────
+  ✓  Loaded 8 files ...
+```
+
+Wipes the ChromaDB collection and ingests the new repo fresh. All subsequent queries answer from the new repo only.
+
+---
+
+## 9. REST API
+
+Start the FastAPI server:
+
+```bash
+.venv\Scripts\python.exe -m uvicorn app:app --reload --port 8000
+```
+
+Open docs at: `http://localhost:8000/docs`
+
+### Endpoints
+
+| Method | Path | Description |
+|---|---|---|
+| `GET` | `/health` | Health check |
+| `POST` | `/api/query` | RAG question answering |
+| `POST` | `/api/analyze/bugs?file_path=...` | Bug detection |
+| `POST` | `/api/analyze/complexity?file_path=...` | Cyclomatic complexity |
+| `POST` | `/api/analyze/explain` | Explain a function |
+| `POST` | `/api/docs/module?file_path=...` | Module documentation |
+| `POST` | `/api/docs/readme?root_path=...` | README generation |
+| `POST` | `/api/files/propose` | Propose a new file |
+| `POST` | `/api/files/approve` | Write approved file to disk |
+
+### Example — Query
+
+```bash
+curl -X POST http://localhost:8000/api/query \
+  -H "Content-Type: application/json" \
+  -d '{"query": "How does authentication work?", "k": 5}'
+```
+
+```json
+{
+  "answer": "Authentication is handled by ...",
+  "sources": [
+    {"file_path": "src/auth.py", "start_line": 10, "end_line": 45}
+  ]
+}
+```
+
+---
+
+## 10. Key Design Decisions & Bug Fixes
+
+### Bug Fix 1 — Collection Isolation (retriever.py)
+
+**Problem:** ChromaDB used `upsert` on a shared `"codebase"` collection — stale chunks from previously ingested repos leaked into new queries.
+
+**Fix:** On every ingest, `delete_collection` + `create_collection` ensures a clean slate:
+
+```python
+# Before (bug)
+collection = client.get_or_create_collection("codebase")
+collection.upsert(...)
+
+# After (fix)
+client.delete_collection("codebase")   # wipe old repo
+collection = client.create_collection("codebase")
+collection.add(...)
+```
+
+---
+
+### Bug Fix 2 — GitHub MCP Cancel Scope (mcp_client.py)
+
+**Problem:** Manually calling `__aenter__`/`__aexit__` on `anyio`-backed context managers across tasks causes `RuntimeError: Attempted to exit cancel scope in a different task`.
+
+**Fix:** Use `AsyncExitStack` to nest all context managers inside a single `async with`:
+
+```python
+async with get_github_mcp_client() as client:
+    tools = await client.list_tools()
+```
+
+---
+
+### Bug Fix 3 — Wrong Unpack Count (mcp_client.py)
+
+**Problem:** `streamable_http_client` yields 2 values, not 3. Unpacking 3 caused `ValueError: not enough values to unpack`.
+
+```python
+# Before (bug)
+read, write, _ = await ctx.__aenter__()
+
+# After (fix)
+read, write = await stack.enter_async_context(streamable_http_client(...))
+```
+
+---
+
+### Bug Fix 4 — File Save Path (file_creator.py)
+
+**Problem:** Generated files saved to the current working directory regardless of user input.
+
+**Fix:** 
+- Strip leading `/\` from LLM path to make it always relative
+- Accept `base_dir` defaulting to `_current_repo` (the ingested repo path)
+- Create all parent directories automatically
+
+---
+
+## 11. Extending the Project
+
+### Add a new LLM provider
+
+1. Add a new class in `llm.py` extending `BaseLLM`
+2. Add the provider name to `get_llm_client()` factory
+3. Add matching settings in `config.py`
+
+### Add support for new file types
+
+1. Add extension → language in `LANGUAGE_BY_EXT` in `repository_loader.py`
+2. Add extension to `allowed_extensions` in `config.py`
+3. If LangChain has a `Language` enum for it, add to `LANGUAGE_MAP` in `splitter.py`
+
+### Use a GitHub MCP tool in a feature
+
+```python
+async with get_github_mcp_client() as client:
+    result = await client.call_tool("create_branch", {
+        "owner": "myuser",
+        "repo": "myrepo",
+        "branch": "feature/new-branch",
+        "from_branch": "main"
+    })
+```
+
+### Add a new CLI option
+
+1. Write a `feature_xxx()` function in `cli.py`
+2. Add `("Label", feature_xxx)` to the `MENU` list
+3. Add a matching FastAPI endpoint in `api/routes.py`
+
+---
+

app.py

@@ -0,0 +1,34 @@
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+
+from api.routes import router
+from mcp_client import get_github_mcp_client
+
+app = FastAPI(title="AI Codebase Assistant")
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+app.include_router(router, prefix="/api")
+
+mcp_client_instance = get_github_mcp_client()
+
+
+@app.on_event("startup")
+async def startup_event():
+    await mcp_client_instance.connect()
+
+
+@app.on_event("shutdown")
+async def shutdown_event():
+    await mcp_client_instance.disconnect()
+
+
+@app.get("/health")
+def health_check():
+    return {"status": "ok"}

cli.py

@@ -0,0 +1,323 @@
+"""
+Codebase Assistant — Interactive CLI
+Run with: .venv\Scripts\python.exe cli.py
+"""
+
+import os
+import sys
+import asyncio
+
+# ── styling ───────────────────────────────────────────────────────────────────
+
+CYAN   = "\033[96m"
+GREEN  = "\033[92m"
+YELLOW = "\033[93m"
+RED    = "\033[91m"
+BOLD   = "\033[1m"
+DIM    = "\033[2m"
+RESET  = "\033[0m"
+
+# ── session state ─────────────────────────────────────────────────────────────
+# Tracks the repo path provided at startup (or via re-ingest) so that option 7
+# can save generated files directly into that directory without asking again.
+_current_repo: str = "."
+
+def banner():
+    print(f"""
+{CYAN}{BOLD}
+  ╔══════════════════════════════════════════╗
+  ║       AI  Codebase  Assistant  CLI       ║
+  ╚══════════════════════════════════════════╝
+{RESET}""")
+
+def section(title: str):
+    print(f"\n{CYAN}{BOLD}── {title} {'─' * (42 - len(title))}{RESET}")
+
+def ok(msg: str):
+    print(f"  {GREEN}✓{RESET}  {msg}")
+
+def err(msg: str):
+    print(f"  {RED}✗  {msg}{RESET}")
+
+def info(msg: str):
+    print(f"  {DIM}{msg}{RESET}")
+
+def ask(prompt: str) -> str:
+    return input(f"\n{YELLOW}▶ {prompt}{RESET} ").strip()
+
+# ── ingest ────────────────────────────────────────────────────────────────────
+
+def ingest_repository(root_path: str):
+    global _current_repo
+    section("Ingesting Repository")
+    from rag.repository_loader import load_repository
+    from rag.splitter import split_code
+    from rag.embedding import embed_document
+    from rag.retriever import build_vector_store
+
+    print(f"  {DIM}Loading files from: {root_path}{RESET}")
+    docs = load_repository(root_path)
+    if not docs:
+        err("No supported files found. Check allowed_extensions in config.py.")
+        return False
+    ok(f"Loaded {len(docs)} files")
+
+    all_chunks = []
+    for doc in docs:
+        all_chunks.extend(split_code(doc))
+    ok(f"Split into {len(all_chunks)} chunks")
+
+    print(f"  {DIM}Embedding chunks (this may take a moment)...{RESET}")
+    embedded = embed_document(all_chunks)
+    ok(f"Embedded {len(embedded)} chunks")
+
+    build_vector_store(embedded)
+    ok("Vector store ready (ChromaDB)\n")
+    _current_repo = root_path   # remember for option 7
+    return True
+
+# ── feature handlers ──────────────────────────────────────────────────────────
+
+def feature_query():
+    section("Ask a Question  [RAG Query]")
+    query = ask("Your question:")
+    if not query:
+        return
+    k_str = ask("Number of source chunks to retrieve? [default: 5]")
+    k = int(k_str) if k_str.isdigit() else 5
+
+    from rag.rag_chain import run_rag_query
+    print(f"\n  {DIM}Thinking...{RESET}")
+    try:
+        result = run_rag_query(query, k=k)
+        section("Answer")
+        print(f"\n{result['answer']}")
+        section("Sources")
+        for s in result["sources"]:
+            print(f"  {DIM}• {s.get('file_path')}  lines {s.get('start_line')}–{s.get('end_line')}{RESET}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_bugs():
+    section("Detect Bugs  [LLM Code Review]")
+    file_path = ask("File path to analyse:")
+    if not os.path.isfile(file_path):
+        err(f"File not found: {file_path}")
+        return
+
+    from services.code_analysis import detect_bugs
+    print(f"\n  {DIM}Analysing...{RESET}")
+    try:
+        bugs = detect_bugs(file_path)
+        if not bugs:
+            ok("No issues found.")
+            return
+        section(f"Found {len(bugs)} issue(s)")
+        for b in bugs:
+            sev = b.get("severity", "?").upper()
+            color = RED if sev in ("HIGH", "CRITICAL") else YELLOW
+            print(f"\n  {color}[{sev}]{RESET} Line {b.get('line', '?')}: {b.get('issue', '')}")
+            print(f"  {DIM}  → {b.get('suggestion', '')}{RESET}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_complexity():
+    section("Cyclomatic Complexity Analysis")
+    file_path = ask("File path to analyse:")
+    if not os.path.isfile(file_path):
+        err(f"File not found: {file_path}")
+        return
+
+    from services.code_analysis import analyze_complexity
+    try:
+        result = analyze_complexity(file_path)
+        fns = result["functions"]
+        if not fns:
+            info("No functions found (or file is not Python).")
+            return
+        section(f"{len(fns)} function(s)")
+        for fn in fns:
+            rank = fn["rank"]
+            color = GREEN if rank == "A" else (YELLOW if rank in ("B", "C") else RED)
+            bar = "█" * fn["complexity"]
+            print(f"  {color}[{rank}]{RESET}  {fn['name']:<30} complexity={fn['complexity']}  {DIM}{bar}{RESET}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_explain():
+    section("Explain a Function")
+    file_path = ask("File path:")
+    if not os.path.isfile(file_path):
+        err(f"File not found: {file_path}")
+        return
+    func_name = ask("Function name:")
+    if not func_name:
+        return
+
+    from services.code_analysis import explain_function
+    print(f"\n  {DIM}Thinking...{RESET}")
+    try:
+        explanation = explain_function(file_path, func_name)
+        section("Explanation")
+        print(f"\n{explanation}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_module_docs():
+    section("Generate Module Documentation")
+    file_path = ask("File path:")
+    if not os.path.isfile(file_path):
+        err(f"File not found: {file_path}")
+        return
+
+    from services.documentation import generate_module_docs
+    print(f"\n  {DIM}Generating docs...{RESET}")
+    try:
+        docs = generate_module_docs(file_path)
+        section("Documentation")
+        print(f"\n{docs}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_readme():
+    section("Generate README.md")
+    root = ask("Repository root path:")
+    if not os.path.isdir(root):
+        err(f"Directory not found: {root}")
+        return
+
+    from services.documentation import generate_readme
+    print(f"\n  {DIM}Generating README...{RESET}")
+    try:
+        readme = generate_readme(root)
+        section("README Preview")
+        print(f"\n{readme[:1500]}{'...' if len(readme) > 1500 else ''}")
+        save = ask("Save to README.md in that directory? [y/N]")
+        if save.lower() == "y":
+            out = os.path.join(root, "README.md")
+            with open(out, "w", encoding="utf-8") as f:
+                f.write(readme)
+            ok(f"Saved to {out}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_propose_file():
+    section("Propose a New File  [AI File Creator]")
+    description = ask("Describe the file you want to create:")
+    if not description:
+        return
+    context_query = ask("Optional context query (or press Enter to skip):")
+
+    from services.file_creator import propose_new_file
+    print(f"\n  {DIM}Generating...{RESET}")
+    try:
+        proposal = propose_new_file(description, context_query or None)
+        section("Proposed File")
+        print(f"\n  {BOLD}Path:{RESET}  {proposal.path}")
+        print(f"\n{DIM}{'─'*50}{RESET}")
+        print(proposal.content[:1000] + ("..." if len(proposal.content) > 1000 else ""))
+        print(f"{DIM}{'─'*50}{RESET}")
+
+        confirm = ask("Write this file to disk? [y/N]")
+        if confirm.lower() == "y":
+            default_dir = _current_repo
+            typed = ask(f"Save under which directory? [default: {default_dir}]").strip()
+            base_dir = typed if typed else default_dir
+            if not os.path.isdir(base_dir):
+                err(f"Directory not found: {base_dir}")
+                return
+            async def _write():
+                from services.file_creator import apply_approved_file
+                proposal.approved = True
+                return await apply_approved_file(proposal, True, base_dir=base_dir)
+            result = asyncio.run(_write())
+            if result.get("status") == "written":
+                ok(f"File written: {result.get('path', proposal.path)}")
+            else:
+                err(f"Write failed: {result}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_mcp_tools():
+    section("List GitHub MCP Tools")
+    async def _list():
+        from mcp_client import get_github_mcp_client
+        async with get_github_mcp_client() as client:
+            return await client.list_tools()
+    try:
+        tools = asyncio.run(_list())
+        ok(f"{len(tools)} tools available:")
+        for t in tools:
+            print(f"  {DIM}• {t['name']:<30}{RESET} {t.get('description','')[:60]}")
+    except Exception as e:
+        err(str(e))
+
+
+def feature_reingest():
+    root = ask("New repository path to ingest:")
+    if os.path.isdir(root):
+        ingest_repository(root)
+    else:
+        err(f"Directory not found: {root}")
+
+# ── menu ──────────────────────────────────────────────────────────────────────
+
+MENU = [
+    ("Ask a question about the codebase",    feature_query),
+    ("Detect bugs in a file",                feature_bugs),
+    ("Cyclomatic complexity analysis",       feature_complexity),
+    ("Explain a function",                   feature_explain),
+    ("Generate module documentation",        feature_module_docs),
+    ("Generate README for a repository",     feature_readme),
+    ("Propose & create a new file (AI)",     feature_propose_file),
+    ("List GitHub MCP tools",                feature_mcp_tools),
+    ("Re-ingest a repository",               feature_reingest),
+]
+
+def show_menu():
+    section("What would you like to do?")
+    for i, (label, _) in enumerate(MENU, 1):
+        print(f"  {CYAN}{i:>2}{RESET}  {label}")
+    print(f"  {DIM}  0  Exit{RESET}")
+
+# ── main ──────────────────────────────────────────────────────────────────────
+
+def main():
+    # Enable ANSI colours on Windows
+    os.system("")
+
+    banner()
+
+    # Step 1 — ask for repo path
+    while True:
+        root = ask("Enter the path to the repository you want to analyse:")
+        if os.path.isdir(root):
+            break
+        err(f"Directory not found: {root}")
+
+    # Step 2 — ingest
+    if not ingest_repository(root):
+        sys.exit(1)
+
+    # Step 3 — interactive menu loop
+    while True:
+        show_menu()
+        choice = ask("Choose an option [0–9]:")
+        if choice == "0" or choice.lower() in ("exit", "quit", "q"):
+            print(f"\n{DIM}Goodbye!{RESET}\n")
+            break
+        if choice.isdigit() and 1 <= int(choice) <= len(MENU):
+            MENU[int(choice) - 1][1]()
+        else:
+            err("Invalid choice. Enter a number from the menu.")
+
+
+if __name__ == "__main__":
+    main()

config.py

@@ -0,0 +1,41 @@
+from pydantic_settings import BaseSettings
+from functools import lru_cache
+
+
+class Settings(BaseSettings):
+    # ── LLM Provider ──────────────────────────────────────────────────────────
+    # Set LLM_PROVIDER to "gemini" or "groq" in your .env file.
+    # Groq is free and has no strict daily quota — recommended when Gemini
+    # free-tier is exhausted.
+    llm_provider: str = "groq"
+
+    # ── Gemini settings ───────────────────────────────────────────────────────
+    gemini_api_key: str = ""
+    llm_model: str = "llama-3.3-70b-versatile"   # overridden per provider below
+    embedding_model: str = "models/gemini-embedding-001"
+
+    # ── Groq settings ─────────────────────────────────────────────────────────
+    groq_api_key: str = ""
+    groq_model: str = "llama-3.3-70b-versatile"  # free, 6k tokens/min on Groq
+
+    # ── RAG / Vector DB ───────────────────────────────────────────────────────
+    vector_db_path: str = "./vector_db"
+    chunk_size: int = 1200
+    chunk_overlap: int = 100
+
+    # ── MCP ───────────────────────────────────────────────────────────────────
+    mcp_server_url: str = "http://localhost:3333"
+    github_mcp_url: str = "https://api.githubcopilot.com/mcp/"
+    github_mcp_token: str = ""
+
+    # ── Loader ────────────────────────────────────────────────────────────────
+    allowed_extensions: list[str] = [".py", ".js", ".ts", ".go", ".java", ".md"]
+    max_file_size_kb: int = 1042
+
+    class Config:
+        env_file = ".env"
+
+
+@lru_cache()
+def get_settings() -> Settings:
+    return Settings()

example.py

@@ -0,0 +1,21 @@
+class Area:
+    # Function to calculate rectangle area
+    def rectangle_area(self, length, width):
+        return length * width
+
+    # Function to calculate square area
+    def square_area(self, side):
+        return side * side
+
+
+# Create object
+obj = Area()
+
+# Input
+length = int(input("Enter length of rectangle: "))
+width = int(input("Enter width of rectangle: "))
+side = int(input("Enter side of square: "))
+
+# Output
+print("Rectangle Area =", obj.rectangle_area(length, width))
+print("Square Area =", obj.square_area(side))

llm.py

@@ -0,0 +1,130 @@
+from abc import ABC, abstractmethod
+from config import get_settings
+
+
+class BaseLLM(ABC):
+    @abstractmethod
+    def generate(self, prompt: str, system: str = "") -> str:
+        ...
+
+    @abstractmethod
+    def stream(self, prompt: str, system: str = ""):
+        ...
+
+
+# ── Gemini provider ────────────────────────────────────────────────────────────
+
+class GeminiLLM(BaseLLM):
+    def __init__(self, api_key: str, model: str):
+        from google import genai
+        self.client = genai.Client(api_key=api_key)
+        self.model = model
+
+    def generate(self, prompt: str, system: str = "") -> str:
+        full_prompt = f"{system}\n\n{prompt}" if system else prompt
+        response = self.client.models.generate_content(
+            model=self.model,
+            contents=full_prompt,
+        )
+        return response.text
+
+    def stream(self, prompt: str, system: str = ""):
+        full_prompt = f"{system}\n\n{prompt}" if system else prompt
+        for chunk in self.client.models.generate_content_stream(
+            model=self.model,
+            contents=full_prompt,
+        ):
+            if chunk.text:
+                yield chunk.text
+
+
+# ── Groq provider (free tier, no daily quota issues) ──────────────────────────
+
+class GroqLLM(BaseLLM):
+    """
+    Uses Groq's free API — no per-day quota, just a per-minute rate limit
+    that is far more generous than Gemini's free tier.
+
+    Sign up at https://console.groq.com and set GROQ_API_KEY in your .env.
+    Recommended model: llama-3.3-70b-versatile (free)
+    """
+
+    def __init__(self, api_key: str, model: str):
+        from groq import Groq
+        self.client = Groq(api_key=api_key)
+        self.model = model
+
+    def _messages(self, prompt: str, system: str) -> list[dict]:
+        messages = []
+        if system:
+            messages.append({"role": "system", "content": system})
+        messages.append({"role": "user", "content": prompt})
+        return messages
+
+    def generate(self, prompt: str, system: str = "") -> str:
+        response = self.client.chat.completions.create(
+            model=self.model,
+            messages=self._messages(prompt, system),
+        )
+        return response.choices[0].message.content
+
+    def stream(self, prompt: str, system: str = ""):
+        stream = self.client.chat.completions.create(
+            model=self.model,
+            messages=self._messages(prompt, system),
+            stream=True,
+        )
+        for chunk in stream:
+            delta = chunk.choices[0].delta.content
+            if delta:
+                yield delta
+
+
+# ── Factory ───────────────────────────────────────────────────────────────────
+
+def get_llm_client() -> BaseLLM:
+    settings = get_settings()
+
+    if settings.llm_provider == "gemini":
+        return GeminiLLM(settings.gemini_api_key, settings.llm_model)
+
+    if settings.llm_provider == "groq":
+        return GroqLLM(settings.groq_api_key, settings.groq_model)
+
+    raise ValueError(
+        f"Unsupported LLM_PROVIDER '{settings.llm_provider}'. "
+        "Valid options: 'gemini', 'groq'"
+    )
+
+
+# ── Prompt templates ──────────────────────────────────────────────────────────
+
+def build_prompt(query: str, context: str, task_type: str) -> str:
+    templates = {
+        "qa": (
+            "You are a codebase assistant. Use the context below to answer the question.\n\n"
+            "Context:\n{context}\n\nQuestion: {query}\n\nAnswer clearly, citing file names when relevant."
+        ),
+        "bug_finding": (
+            "Analyze the following code for potential bugs, security issues, or bad practices.\n\n"
+            "Code:\n{context}\n\n"
+            "Return a JSON list of objects with keys: line, issue, severity, suggestion."
+        ),
+        "docstring": (
+            "Write a clear, concise docstring for the following function. "
+            "Follow standard conventions for its language.\n\nFunction:\n{query}"
+        ),
+        "file_creation": (
+            "Generate complete, production-ready code for the following request.\n\n"
+            "Request: {query}\n\nRelevant existing code for context:\n{context}\n\n"
+            "Return only the code, no explanations."
+        ),
+    }
+    template = templates.get(task_type, templates["qa"])
+    return template.format(query=query, context=context)
+
+
+def count_tokens(text: str) -> int:
+    import tiktoken
+    encoder = tiktoken.get_encoding("cl100k_base")
+    return len(encoder.encode(text))

mcp_client.py

@@ -0,0 +1,68 @@
+from contextlib import asynccontextmanager
+
+from mcp.client.streamable_http import streamable_http_client
+from mcp.shared._httpx_utils import create_mcp_http_client
+from mcp import ClientSession
+from config import get_settings
+
+
+class RemoteMCPClient:
+    """
+    Async context-manager wrapper around an MCP streamable-HTTP session.
+
+    Usage (all work must happen inside the `async with` block so that every
+    anyio cancel scope is entered and exited in the *same* task):
+
+        async with RemoteMCPClient(url, token) as client:
+            tools = await client.list_tools()
+    """
+
+    def __init__(self, server_url: str, api_key: str):
+        self.server_url = server_url
+        self.api_key = api_key
+        self.session: ClientSession | None = None
+        self._exit_stack = None
+
+    # ── async context manager ─────────────────────────────────────────────────
+
+    async def __aenter__(self):
+        from contextlib import AsyncExitStack
+        self._exit_stack = AsyncExitStack()
+        await self._exit_stack.__aenter__()
+
+        headers = {"Authorization": f"Bearer {self.api_key}"}
+        http_client = create_mcp_http_client(headers=headers)
+        await self._exit_stack.enter_async_context(http_client)
+
+        # streamable_http_client yields (read_stream, write_stream) — 2 values
+        read_stream, write_stream = await self._exit_stack.enter_async_context(
+            streamable_http_client(self.server_url, http_client=http_client)
+        )
+
+        self.session = await self._exit_stack.enter_async_context(
+            ClientSession(read_stream, write_stream)
+        )
+        await self.session.initialize()
+        return self
+
+    async def __aexit__(self, *exc_info):
+        await self._exit_stack.__aexit__(*exc_info)
+        self.session = None
+
+    # ── public API ────────────────────────────────────────────────────────────
+
+    async def list_tools(self) -> list[dict]:
+        result = await self.session.list_tools()
+        return [{"name": t.name, "description": t.description} for t in result.tools]
+
+    async def call_tool(self, tool_name: str, params: dict) -> dict:
+        try:
+            result = await self.session.call_tool(tool_name, params)
+            return {"status": "success", "data": result.content}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+
+
+def get_github_mcp_client() -> RemoteMCPClient:
+    settings = get_settings()
+    return RemoteMCPClient(settings.github_mcp_url, settings.github_mcp_token)

requirements.txt

@@ -0,0 +1,38 @@
+# Core LLM providers
+google-genai==2.14.0
+google-generativeai==0.8.6
+groq
+
+# LangChain + embeddings
+langchain==1.3.14
+langchain-chroma==1.1.0
+langchain-community==0.4.2
+langchain-core==1.5.2
+langchain-google-genai==4.3.2
+langchain-text-splitters==1.1.2
+
+# Vector store
+chromadb==1.5.9
+
+# API / Web
+fastapi==0.116.1
+uvicorn==0.35.0
+python-multipart==0.0.32
+
+# MCP client
+mcp==2.0.0
+httpx==0.28.1
+httpx-sse==0.4.3
+
+# Config & utilities
+pydantic==2.13.4
+pydantic-settings==2.14.2
+python-dotenv==1.2.2
+tiktoken==0.13.0
+rich==14.3.3
+typer==0.25.1
+requests==2.34.2
+
+# Code analysis
+radon==6.0.1
+GitPython==3.1.45

test_github_mcp.py

@@ -0,0 +1,18 @@
+import asyncio
+from mcp_client import get_github_mcp_client
+from config import get_settings
+
+
+async def main():
+    client = get_github_mcp_client()
+    await client.connect()
+
+    tools = await client.list_tools()
+    print(f"Connected. {len(tools)} tools available:\n")
+    for t in tools:
+        print(f"- {t['name']}: {t['description']}")
+
+    await client.disconnect()
+
+
+asyncio.run(main())

test_runner.py

@@ -0,0 +1,164 @@
+"""
+Manual test runner for the Codebase Assistant.
+Run with: .venv\Scripts\python.exe test_runner.py [test_name]
+
+Available tests:
+  mcp        - Test GitHub MCP connection & list tools
+  embed      - Test embedding a small text snippet
+  ingest     - Index this repo into the local vector store
+  query      - Run a RAG query against the vector store
+  bugs       - Detect bugs in a given file
+  complexity - Analyse cyclomatic complexity of a file
+  all        - Run embed -> ingest -> query -> bugs -> complexity (no MCP)
+"""
+
+import asyncio
+import sys
+import os
+
+# ── helpers ──────────────────────────────────────────────────────────────────
+
+def header(title: str):
+    print(f"\n{'='*60}")
+    print(f"  {title}")
+    print(f"{'='*60}")
+
+def ok(msg: str):   print(f"  OK  {msg}")
+def fail(msg: str): print(f"  FAIL  {msg}")
+
+ROOT = os.path.dirname(__file__)
+
+# ── individual tests ──────────────────────────────────────────────────────────
+
+async def test_mcp():
+    """Connect to GitHub MCP and list available tools."""
+    header("Test: GitHub MCP Connection")
+    from mcp_client import get_github_mcp_client
+    client = get_github_mcp_client()
+    try:
+        await client.connect()
+        tools = await client.list_tools()
+        ok(f"Connected. {len(tools)} tools found:")
+        for t in tools[:5]:
+            print(f"       - {t['name']}: {t['description'][:60]}")
+        if len(tools) > 5:
+            print(f"       ... and {len(tools)-5} more")
+    except Exception as e:
+        fail(f"MCP connection failed: {e}")
+    finally:
+        await client.disconnect()
+
+
+def test_embed():
+    """Embed a small text snippet and verify the vector shape."""
+    header("Test: Embedding")
+    from rag.embedding import embed_query, embed_document
+    try:
+        vec = embed_query("def hello(): pass")
+        assert isinstance(vec, list) and len(vec) > 0, "embed_query returned empty"
+        ok(f"embed_query  -> vector of {len(vec)} dims")
+
+        fake_chunks = [{"content": "def add(a, b): return a+b"}]
+        chunks = embed_document(fake_chunks)
+        assert "embedding" in chunks[0], "embed_document did not add 'embedding' key"
+        ok(f"embed_document -> chunk embedding of {len(chunks[0]['embedding'])} dims")
+    except Exception as e:
+        fail(f"Embedding failed: {e}")
+
+
+def test_ingest():
+    """Load and index this repository into the local ChromaDB."""
+    header("Test: Repository Ingest")
+    from rag.repository_loader import load_repository
+    from rag.splitter import split_code
+    from rag.embedding import embed_document
+    from rag.retriever import build_vector_store
+    try:
+        docs = load_repository(ROOT)
+        ok(f"Loaded {len(docs)} files from repo")
+
+        all_chunks = []
+        for doc in docs:
+            all_chunks.extend(split_code(doc))
+        ok(f"Split into {len(all_chunks)} chunks")
+
+        embedded = embed_document(all_chunks)
+        ok(f"Embedded {len(embedded)} chunks")
+
+        build_vector_store(embedded)
+        ok("Vector store built / updated (ChromaDB)")
+    except Exception as e:
+        fail(f"Ingest failed: {e}")
+
+
+def test_query():
+    """Run a RAG query. Requires the vector store to be populated first."""
+    header("Test: RAG Query")
+    from rag.rag_chain import run_rag_query
+    try:
+        result = run_rag_query("How does the embedding module work?", k=3)
+        ok(f"Answer received ({len(result['answer'])} chars)")
+        ok(f"Sources returned: {len(result['sources'])}")
+        for s in result["sources"]:
+            print(f"       - {s.get('file_path','?')} lines {s.get('start_line')}-{s.get('end_line')}")
+        print(f"\n  Answer preview:\n  {result['answer'][:300]}...")
+    except Exception as e:
+        fail(f"RAG query failed: {e}")
+
+
+def test_bugs():
+    """Run bug detection on llm.py."""
+    header("Test: Bug Detection")
+    from services.code_analysis import detect_bugs
+    target = os.path.join(ROOT, "llm.py")
+    try:
+        bugs = detect_bugs(target)
+        ok(f"Bug detection returned {len(bugs)} item(s) for llm.py")
+        for b in bugs[:3]:
+            print(f"       Line {b.get('line','?')}: [{b.get('severity','?')}] {b.get('issue','?')}")
+    except Exception as e:
+        fail(f"Bug detection failed: {e}")
+
+
+def test_complexity():
+    """Analyse cyclomatic complexity of llm.py."""
+    header("Test: Complexity Analysis")
+    from services.code_analysis import analyze_complexity
+    target = os.path.join(ROOT, "llm.py")
+    try:
+        result = analyze_complexity(target)
+        ok(f"Complexity analysis returned {len(result['functions'])} function(s):")
+        for fn in result["functions"]:
+            print(f"       {fn['name']}: complexity={fn['complexity']}, rank={fn['rank']}")
+    except Exception as e:
+        fail(f"Complexity analysis failed: {e}")
+
+
+# ── entry point ───────────────────────────────────────────────────────────────
+
+TESTS = {
+    "mcp":        lambda: asyncio.run(test_mcp()),
+    "embed":      test_embed,
+    "ingest":     test_ingest,
+    "query":      test_query,
+    "bugs":       test_bugs,
+    "complexity": test_complexity,
+}
+
+def run_all_non_mcp():
+    """Run embed -> ingest -> query -> bugs -> complexity in sequence."""
+    test_embed()
+    test_ingest()
+    test_query()
+    test_bugs()
+    test_complexity()
+
+if __name__ == "__main__":
+    arg = sys.argv[1] if len(sys.argv) > 1 else "all"
+    if arg == "all":
+        run_all_non_mcp()
+    elif arg in TESTS:
+        TESTS[arg]()
+    else:
+        print(f"Unknown test '{arg}'. Choose from: {', '.join(TESTS)} or 'all'")
+        sys.exit(1)