Update clean backend-only project docs and eval

docs/PRODUCTION_RUNBOOK.md

@@ -0,0 +1,178 @@
+# BlitzKode Production Runbook
+
+This runbook captures the operational path for serving BlitzKode as a local or self-hosted coding assistant.
+
+## 1. Release artifacts
+
+Expected production artifacts:
+
+- `blitzkode.gguf` — local GGUF model mounted into the container at `/app/blitzkode.gguf`.
+- Docker image built from `Dockerfile` — includes `server.py` and Python dependencies only.
+- Optional HuggingFace repos:
+  - `neuralbroker/blitzkode` — GGUF distribution repo.
+  - `neuralbroker/blitzkode-1.5b-lora` — 1.5B adapter repo.
+  - `neuralbroker/blitzkode-lora-0.5b` — 0.5B adapter repo.
+
+Do not commit model weights, checkpoints, `.env` files, or HuggingFace tokens to git.
+
+## 2. Required environment
+
+Minimum runtime:
+
+- Python 3.11+ when running directly.
+- Docker 24+ when running in containers.
+- 4 GB+ RAM for the Q8_0 1.5B GGUF artifact.
+- Optional NVIDIA container toolkit for GPU offload.
+
+Key server variables:
+
+| Variable | Production guidance |
+|---|---|
+| `BLITZKODE_MODEL_PATH` | Set to `/app/blitzkode.gguf` in Docker or an absolute local path outside Docker. |
+| `BLITZKODE_PRELOAD_MODEL` | Use `true` for production so startup fails fast if the model cannot load. |
+| `BLITZKODE_API_KEY` | Set a strong bearer token for any network-accessible deployment. |
+| `BLITZKODE_CORS_ORIGINS` | Restrict to trusted API client origins instead of `*`. |
+| `BLITZKODE_RATE_LIMIT` | Keep `true` unless running behind another trusted limiter. |
+| `BLITZKODE_RATE_LIMIT_MAX` | Tune based on expected users and hardware. |
+| `BLITZKODE_WEB_SEARCH` | Set `false` for fully offline operation; keep `true` for research mode. |
+| `BLITZKODE_GPU_LAYERS` | `0` for CPU only, `-1` for all possible layers on GPU, or tune gradually. |
+| `BLITZKODE_N_CTX` | Start with `2048`; increase to `4096` or higher only if memory allows. |
+| `BLITZKODE_BATCH` / `BLITZKODE_UBATCH` | Start with `256` / `128`; increase only after latency and memory checks. |
+| `BLITZKODE_PROMPT_CACHE` | Keep `true` for repeated system/history prefixes if supported by the installed `llama-cpp-python`. |
+
+## 3. Pre-deployment validation
+
+Run these checks before tagging or deploying a release:
+
+```bash
+python -m pytest tests/ -v
+python -m ruff check .
+python -m mypy server.py --ignore-missing-imports
+docker build -t blitzkode:ci .
+```
+
+For CI smoke tests without the real model, start the container with `BLITZKODE_PRELOAD_MODEL=false` and verify `/health` returns HTTP 200.
+
+## 4. CPU Docker deployment
+
+Place `blitzkode.gguf` next to `docker-compose.yml`, then run:
+
+```bash
+docker compose up --build -d
+```
+
+The default compose service mounts the model read-only into `/app/blitzkode.gguf` and exposes the app on `http://localhost:7860`.
+
+Check service state:
+
+```bash
+docker compose ps
+docker compose logs --tail=100 blitzkode
+curl -sf http://localhost:7860/health
+curl -sf http://localhost:7860/info
+```
+
+A healthy deployment should report:
+
+- `status` is `healthy` when the model file exists.
+- `model_exists` is `true`.
+- `last_error` is empty or `null`.
+- `batch`, `ubatch`, and thread settings match the intended deployment profile.
+
+## 5. GPU Docker deployment
+
+Prerequisites:
+
+1. NVIDIA driver installed on the host.
+2. `nvidia-container-toolkit` installed.
+3. Docker configured for the NVIDIA runtime.
+4. A `llama-cpp-python` build with compatible GPU acceleration.
+
+Start the GPU profile:
+
+```bash
+BLITZKODE_GPU_LAYERS=35 docker compose --profile gpu up --build -d
+```
+
+If startup fails or inference crashes, lower `BLITZKODE_GPU_LAYERS` and restart. Use `0` to force CPU-only fallback.
+
+## 6. Direct local deployment
+
+For non-container operation:
+
+```bash
+pip install -r requirements.txt
+BLITZKODE_MODEL_PATH=blitzkode.gguf BLITZKODE_PRELOAD_MODEL=true python server.py
+```
+
+On Windows shells, set environment variables using the shell-specific syntax before running `python server.py`.
+
+## 7. Health checks and smoke tests
+
+Recommended checks after each deployment:
+
+```bash
+curl -sf http://localhost:7860/health
+curl -sf http://localhost:7860/info
+curl -sf -X POST http://localhost:7860/generate \
+  -H "Content-Type: application/json" \
+  -d '{"prompt":"Return a short Python hello-world function.","max_tokens":64}'
+```
+
+If `BLITZKODE_API_KEY` is configured, include `Authorization: Bearer <token>` on protected requests.
+
+## 8. Rollback plan
+
+Rollback should be artifact-based and fast:
+
+1. Keep the last known-good Docker image tag available locally or in the registry.
+2. Keep the last known-good `blitzkode.gguf` artifact available outside the container.
+3. Stop the current service.
+4. Restore the previous image tag and/or previous model file.
+5. Start the service and run the health checks from section 7.
+
+Example container rollback flow:
+
+```bash
+docker compose down
+docker tag blitzkode:previous blitzkode:latest
+docker compose up -d
+curl -sf http://localhost:7860/health
+```
+
+## 9. HuggingFace publishing
+
+Use a token only through environment variables or CI secrets:
+
+```bash
+HF_TOKEN=hf_xxx python scripts/push_all_to_hub.py
+```
+
+Before publishing, confirm:
+
+- `blitzkode.gguf` exists and loads locally.
+- Adapter directories contain `adapter_config.json` and adapter weights.
+- `MODEL_CARD.md`, `README.md`, and `datasets/MANIFEST.md` match the artifact versions.
+- The token has write access to the intended repos.
+
+Never paste real tokens into documentation, committed scripts, or issue comments.
+
+## 10. Common failure modes
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| `/health` returns `degraded` | Model file missing from configured path | Mount or copy `blitzkode.gguf`; verify `BLITZKODE_MODEL_PATH`. |
+| Startup hangs while loading | Large context/batch or slow CPU disk load | Reduce `BLITZKODE_N_CTX` / `BLITZKODE_BATCH`, check disk and RAM. |
+| Container exits on first request | llama.cpp cannot load model | Verify GGUF file integrity and llama-cpp-python compatibility. |
+| Browser cannot call API | CORS origin mismatch | Set `BLITZKODE_CORS_ORIGINS` to the deployed UI origin. |
+| HTTP 401 | Missing or wrong bearer token | Send `Authorization: Bearer <BLITZKODE_API_KEY>`. |
+| HTTP 429 | Rate limit exceeded | Increase `BLITZKODE_RATE_LIMIT_MAX` or add an upstream queue/limit policy. |
+| Research mode fails | Web search disabled or network blocked | Set `BLITZKODE_WEB_SEARCH=true` and verify outbound HTTP access. |
+
+## 11. Operational notes
+
+- Treat generated code as assistant output, not an automatically trusted patch.
+- Prefer `/generate/research` for current APIs or documentation-sensitive questions.
+- Keep logs free of prompts if prompts may contain private code or secrets.
+- Rotate `BLITZKODE_API_KEY` and HuggingFace tokens regularly.
+- Re-run the full validation suite after changing dependencies, model artifacts, or Docker base images.