Update clean backend-only project docs and eval

MODEL_CARD.md

@@ -23,9 +23,9 @@ base_model:
 # BlitzKode
 
 **BlitzKode** is a local AI coding assistant fine-tuned from the Qwen2.5 family. It
-ships as a **GGUF model** (1.5B, F16, ~3 GB) for fast offline inference with
-llama.cpp, and as a **LoRA adapter** (0.5B, ~100 MB) for PEFT-based research and
-further fine-tuning.
+ships as a **GGUF model** (1.5B, Q8_0, ~1.53 GB) for fast offline inference with
+llama.cpp, and as **LoRA adapters** for PEFT-based research and further
+fine-tuning.
 
 > **Creator:** [Sajad (neuralbroker)](https://github.com/neuralbroker)
 > **GitHub:** <https://github.com/neuralbroker/blitzkode>
@@ -38,7 +38,7 @@ further fine-tuning.
 
 | Variant | Version | Base Model | Format | Size | Runtime |
 |---|---|---|---|---|---|
-| **GGUF** (production) | 2.0 | `Qwen/Qwen2.5-1.5B-Instruct` | GGUF F16 | ~3 GB | llama.cpp / llama-cpp-python |
+| **GGUF** (production) | 2.0 | `Qwen/Qwen2.5-1.5B-Instruct` | GGUF Q8_0 | ~1.53 GB | llama.cpp / llama-cpp-python |
 | **LoRA adapter** (research) | 2.1 | `Qwen/Qwen2.5-0.5B-Instruct` | PEFT safetensors | ~100 MB | PEFT + Transformers |
 
 ---
@@ -49,7 +49,7 @@ further fine-tuning.
 |---|---|---|
 | **Model type** | Transformer (Qwen2) | Transformer (Qwen2) + LoRA |
 | **Parameters** | 1.5 B | 0.5 B + adapter weights |
-| **Quantization** | GGUF F16 | bfloat16 / float16 |
+| **Quantization** | GGUF Q8_0 | bfloat16 / float16 |
 | **LoRA rank (r)** | — | 16 |
 | **LoRA alpha** | — | 32 |
 | **LoRA target modules** | — | q, k, v, o, gate, up, down projections |
@@ -95,10 +95,10 @@ reached **~0.48**.
 
 ### Stage 5 — Merge & Export (GGUF)
 LoRA adapters from Stage 1–3 were merged into the 1.5B base model using
-`merge_and_unload()`, then converted to GGUF F16 format with llama.cpp.
+`merge_and_unload()`, then converted to GGUF Q8_0 format with llama.cpp.
 
 - **Script:** `scripts/export_gguf.py`
-- **Artifact:** `blitzkode.gguf` (~3 GB, git-ignored)
+- **Artifact:** `blitzkode.gguf` (~1.53 GB, git-ignored)
 
 ---
 
@@ -126,8 +126,8 @@ preprocessing notes, and per-sample license details.
 - **Algorithm assistance** — data structures and algorithms (LeetCode-style)
 - **Offline operation** — fully local, no internet required at inference time
 - **Fast CPU inference** — GGUF F16 runs on commodity CPUs
-- **Modern web UI** — React/Vite chat interface with SSE streaming
-- **REST API** — FastAPI backend with streaming and optional web-search augmentation
+- **API-first serving** — FastAPI backend with REST and SSE streaming endpoints
+- **Optimized local inference** — configurable llama.cpp GPU offload, mmap loading, batching, and prompt cache
 
 ---
 
@@ -141,12 +141,9 @@ git clone https://github.com/neuralbroker/blitzkode
 cd blitzkode
 pip install -r requirements.txt
 
-# Build the frontend
-cd frontend && npm install && npm run build && cd ..
-
-# Start the server (place blitzkode.gguf in repo root first)
+# Start the API server (place blitzkode.gguf in repo root first)
 python server.py
-# Open http://localhost:7860
+curl http://localhost:7860/health
 ```
 
 ### Research: LoRA Adapter with PEFT
@@ -199,13 +196,29 @@ and well-documented code. Keep responses concise and practical.<|im_end|>
 
 ---
 
+## Evaluation
+
+Latest local GGUF smoke evaluation was run with `python scripts/evaluate_model.py` on CPU (`n_ctx=2048`, `threads=8`, `batch=256`, `gpu_layers=0`). Full machine-readable results are available in [`docs/evaluation_results.json`](docs/evaluation_results.json).
+
+| Eval case | Result | Notes |
+|---|---:|---|
+| Python factorial with negative-input handling | ✅ Pass | Correct iterative implementation with negative-input validation. |
+| Iterative binary search | ✅ Pass | Valid loop-based implementation returning index or `-1`. |
+| SQL top users by order count | ✅ Pass | Correct `JOIN`, `GROUP BY`, `ORDER BY`, and `LIMIT 5` structure. |
+| Unknown fictional API uncertainty | ❌ Fail | Raw model hallucinated a plausible signature; the FastAPI backend adds a guard for direct unknown-signature prompts. |
+
+Summary: **3 / 4 passed (75%)**. This is a lightweight heuristic regression smoke test, not a benchmark suite. Stronger future evaluation should include executable unit tests and larger coding benchmarks such as HumanEval/MBPP-style tasks.
+
+---
+
 ## Limitations
 
 - **Text-only input** — no image or file-upload support
-- **2 048-token context** — CPU-friendly but limits long conversation history
+- **2 048-token default context** — CPU-friendly but limits long conversation history
 - **Verify all outputs** — always review and test generated code
 - **Small model** — 0.5B–1.5B scale; may produce incorrect code on complex tasks
-- **No real-time data** — knowledge cutoff follows the Qwen2.5 base model
+- **Raw model hallucination risk** — the API server includes guardrails, but direct GGUF prompting can still invent unsupported API details
+- **No real-time data** — knowledge cutoff follows the Qwen2.5 base model unless the optional research endpoint is used
 - **Math reasoning** — MetaMathQA transfer helps basic reasoning; not a math specialist
 
 ---
@@ -215,9 +228,12 @@ and well-documented code. Keep responses concise and practical.<|im_end|>
 | Variable | Default | Description |
 |---|---|---|
 | `BLITZKODE_GPU_LAYERS` | `0` | Number of layers to offload to GPU |
-| `BLITZKODE_THREADS` | system | CPU inference thread count |
+| `BLITZKODE_THREADS` | system | CPU decode thread count |
+| `BLITZKODE_THREADS_BATCH` | system | CPU prompt-processing thread count |
 | `BLITZKODE_N_CTX` | `2048` | Context window size |
-| `BLITZKODE_BATCH` | `512` | llama.cpp batch size |
+| `BLITZKODE_BATCH` | `256` | llama.cpp prompt-processing batch size |
+| `BLITZKODE_UBATCH` | `128` | llama.cpp micro-batch size |
+| `BLITZKODE_PROMPT_CACHE` | `true` | Enable in-memory prompt cache when supported |
 | `BLITZKODE_PRELOAD_MODEL` | `false` | Load model at startup vs first request |
 
 ---
@@ -228,8 +244,8 @@ and well-documented code. Keep responses concise and practical.<|im_end|>
 BlitzKode/
   server.py                   # FastAPI backend (inference + search)
   blitzkode.gguf              # GGUF model artifact (~3 GB, git-ignored)
-  frontend/                   # React/Vite web UI
   scripts/
+    evaluate_model.py         # Lightweight GGUF evaluation harness
     train_sft.py              # Stage 1: SFT training
     train_reward_sft.py       # Stage 2: Reward-SFT
     train_dpo.py              # Stage 3: DPO
@@ -237,6 +253,8 @@ BlitzKode/
     export_gguf.py            # Merge & convert to GGUF
     push_to_hub.py            # Push adapter to HuggingFace Hub
     build_full_dataset.py     # Dataset builder (algorithmic + HF datasets)
+  docs/
+    evaluation_results.json   # Latest smoke-eval output
   datasets/
     MANIFEST.md               # Dataset provenance and license info
   checkpoints/