Add full Edge Impulse docs RAG assistant (prebuilt index + inference)

.gitattributes

@@ -1,35 +1,2 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
+*.faiss filter=lfs diff=lfs merge=lfs -text
 *.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,130 @@
+---
+license: apache-2.0
+base_model: edgeimpulse/edgeimpulse-docs-qwen-0.5b
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+  - edge-impulse
+  - rag
+  - retrieval-augmented-generation
+  - faiss
+  - qwen
+  - documentation
+  - tinyml
+  - edge-ai
+---
+
+# Edge Impulse Docs — RAG Assistant
+
+A retrieval-augmented assistant for the [Edge Impulse](https://edgeimpulse.com)
+documentation. It grounds every answer in a prebuilt vector index of the docs and
+generates with the small quantized model
+[`edgeimpulse/edgeimpulse-docs-qwen-0.5b`](https://huggingface.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b),
+so it runs comfortably on a laptop.
+
+- **Retrieval:** FAISS (inner-product) over `data/index`, embedded with
+  `sentence-transformers/all-MiniLM-L6-v2` (384-dim, the same model the index was
+  built with).
+- **Generation:** the 0.5B GGUF, served through any OpenAI-compatible endpoint
+  (llama.cpp `llama-server` or Ollama). No training stack required.
+- **Grounded + cited:** answers are constrained to the retrieved context and each
+  response lists its source documents.
+
+This repo ships only what you need to **run** the assistant — the prebuilt index
+and the inference code. The raw document corpus and the index-building pipeline
+are not included.
+
+## Contents
+
+| File | Purpose |
+| --- | --- |
+| `data/index/edge_impulse_docs.faiss` | FAISS inner-product index of the docs |
+| `data/index/chunks.pkl` | Chunk text + source metadata (aligned to the index) |
+| `data/index/metadata.json` | Embedding model + index parameters |
+| `rag.py` | Retrieval + grounded generation (CLI + importable) |
+| `serve.py` | Minimal Flask HTTP API (`POST /ask`) |
+| `requirements.txt` | Runtime dependencies |
+
+## Quickstart
+
+**1. Install dependencies and download this repo**
+
+```bash
+pip install -r requirements.txt
+hf download edgeimpulse/edgeimpulse-docs-rag --local-dir edgeimpulse-docs-rag
+cd edgeimpulse-docs-rag
+```
+
+**2. Start the generator** (pick one)
+
+llama.cpp:
+
+```bash
+hf download edgeimpulse/edgeimpulse-docs-qwen-0.5b qwen-edgeai-q4_k_m.gguf --local-dir .
+llama-server -m qwen-edgeai-q4_k_m.gguf -c 4096 --port 8080 --jinja
+```
+
+Ollama:
+
+```bash
+ollama run hf.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b
+# then point rag.py at Ollama's OpenAI-compatible port:
+export RAG_API_BASE=http://127.0.0.1:11434/v1
+export RAG_MODEL=hf.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b
+```
+
+**3. Ask a question**
+
+```bash
+python rag.py "How do I deploy a model to run on a Linux target as an .eim file?"
+```
+
+Only see what was retrieved (no generation):
+
+```bash
+python rag.py "How do I create an API key?" --no-generate
+```
+
+Serve it over HTTP:
+
+```bash
+python serve.py --host 0.0.0.0 --port 8000
+curl -s localhost:8000/ask -H 'content-type: application/json' \
+  -d '{"question": "What is the data forwarder?"}'
+```
+
+## Configuration
+
+`rag.py` reads these environment variables (all optional):
+
+| Variable | Default | Meaning |
+| --- | --- | --- |
+| `RAG_INDEX_DIR` | `data/index` | Location of the FAISS index + chunks |
+| `RAG_API_BASE` | `http://127.0.0.1:8080/v1` | OpenAI-compatible generation endpoint |
+| `RAG_MODEL` | `edgeimpulse/edgeimpulse-docs-qwen-0.5b` | Model name passed to the endpoint |
+
+## How it works
+
+```
+question ──▶ MiniLM embed ──▶ FAISS top-k ──▶ context + question
+                                                    │
+                                                    ▼
+                             edgeimpulse-docs-qwen-0.5b (llama.cpp / Ollama)
+                                                    │
+                                                    ▼
+                                    grounded answer + cited sources
+```
+
+The generator is a small model, so retrieval quality matters: the assistant is
+most accurate when the right chunk is retrieved, and it may be terse or repeat
+itself on out-of-scope questions. Sampling defaults (`temperature 0.3`,
+`repeat_penalty 1.2`) are tuned to keep it from looping.
+
+## Related
+
+- Generator model: [`edgeimpulse/edgeimpulse-docs-qwen-0.5b`](https://huggingface.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b)
+- API-scoped variant: [`edgeimpulse/edgeimpulse-api-docs-rag`](https://huggingface.co/edgeimpulse/edgeimpulse-api-docs-rag)
+
+## License
+
+Apache-2.0. Documentation content belongs to Edge Impulse.

data/index/chunks.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7f90aa5667bc03f9743895628228e5a43d186a17d677dfce039f191c65b68a9
+size 922471

data/index/edge_impulse_docs.faiss

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a00fcd8072d979872290f1c180c0c603bd6d01bf47fa34db28399f84a35daab
+size 837165

data/index/metadata.json

@@ -0,0 +1,8 @@
+{
+  "embedding_model": "sentence-transformers/all-MiniLM-L6-v2",
+  "docs_dir": "data/raw_docs",
+  "chunk_count": 545,
+  "embedding_dim": 384,
+  "chunk_chars": 1800,
+  "overlap_chars": 250
+}

rag.py

@@ -0,0 +1,154 @@
+"""Edge Impulse docs RAG — retrieval + grounded generation.
+
+Retrieval: FAISS (inner-product) over the prebuilt index in ``data/index`` using
+the same ``all-MiniLM-L6-v2`` sentence embedder the index was built with.
+
+Generation: the published quantized model ``edgeimpulse/edgeimpulse-docs-qwen-0.5b``
+served through any OpenAI-compatible endpoint — e.g. llama.cpp's ``llama-server``
+or Ollama. Only the tiny GGUF is needed for generation, so no training stack is
+required to run this assistant.
+
+The raw document corpus and the index-building pipeline are intentionally not
+part of this repository; the prebuilt index is all you need at inference time.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import pickle
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+import faiss
+import requests
+from sentence_transformers import SentenceTransformer
+
+DEFAULT_INDEX_DIR = Path(os.environ.get("RAG_INDEX_DIR", "data/index"))
+
+# OpenAI-compatible generation endpoint (llama.cpp `llama-server` or Ollama).
+#   llama.cpp : llama-server -m qwen-edgeai-q4_k_m.gguf --port 8080 --jinja
+#   ollama    : ollama run hf.co/edgeimpulse/edgeimpulse-docs-qwen-0.5b
+DEFAULT_API_BASE = os.environ.get("RAG_API_BASE", "http://127.0.0.1:8080/v1")
+DEFAULT_MODEL = os.environ.get("RAG_MODEL", "edgeimpulse/edgeimpulse-docs-qwen-0.5b")
+DEFAULT_API_KEY = os.environ.get("RAG_API_KEY", "sk-no-key-required")
+
+SYSTEM_PROMPT = (
+    "You are an Edge Impulse documentation assistant. Answer only from the "
+    "provided context. If the context does not contain the answer, say what is "
+    "missing and suggest the closest relevant docs source. Be concise."
+)
+
+
+@lru_cache(maxsize=1)
+def load_retriever(index_dir: str):
+    root = Path(index_dir)
+    metadata = json.loads((root / "metadata.json").read_text(encoding="utf-8"))
+    index = faiss.read_index(str(root / "edge_impulse_docs.faiss"))
+    with (root / "chunks.pkl").open("rb") as f:
+        chunks = pickle.load(f)
+    embedder = SentenceTransformer(metadata["embedding_model"])
+    return index, chunks, embedder, metadata
+
+
+def retrieve(question: str, index_dir: Path = DEFAULT_INDEX_DIR, k: int = 4) -> list[dict[str, Any]]:
+    index, chunks, embedder, _ = load_retriever(str(index_dir))
+    q_emb = embedder.encode(
+        [question], convert_to_numpy=True, normalize_embeddings=True
+    ).astype("float32")
+    scores, ids = index.search(q_emb, k)
+    results: list[dict[str, Any]] = []
+    for score, idx in zip(scores[0], ids[0]):
+        if idx < 0:
+            continue
+        record = dict(chunks[int(idx)])
+        record["score"] = float(score)
+        results.append(record)
+    return results
+
+
+def build_messages(question: str, contexts: list[dict[str, Any]]) -> list[dict[str, str]]:
+    context_text = "\n\n".join(
+        f"Source: {item['source']}\n{item['text']}" for item in contexts
+    )
+    user = f"Context:\n{context_text}\n\nQuestion: {question}"
+    return [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": user},
+    ]
+
+
+def generate(
+    messages: list[dict[str, str]],
+    api_base: str = DEFAULT_API_BASE,
+    model: str = DEFAULT_MODEL,
+    api_key: str = DEFAULT_API_KEY,
+    max_new_tokens: int = 320,
+) -> str:
+    payload = {
+        "model": model,
+        "messages": messages,
+        "temperature": 0.3,
+        "top_p": 0.9,
+        "max_tokens": max_new_tokens,
+        # Honoured by llama.cpp's server; ignored by backends that don't support it.
+        "repeat_penalty": 1.2,
+    }
+    resp = requests.post(
+        f"{api_base.rstrip('/')}/chat/completions",
+        headers={"Authorization": f"Bearer {api_key}"},
+        json=payload,
+        timeout=120,
+    )
+    resp.raise_for_status()
+    return resp.json()["choices"][0]["message"]["content"].strip()
+
+
+def ask(
+    question: str,
+    index_dir: Path = DEFAULT_INDEX_DIR,
+    k: int = 4,
+    max_new_tokens: int = 320,
+    no_generate: bool = False,
+    api_base: str = DEFAULT_API_BASE,
+    model: str = DEFAULT_MODEL,
+) -> str:
+    contexts = retrieve(question, index_dir, k)
+    sources = "\n".join(f"- {item['source']} ({item['score']:.3f})" for item in contexts)
+    if no_generate:
+        return "Retrieved context:\n" + sources
+    answer = generate(
+        build_messages(question, contexts),
+        api_base=api_base,
+        model=model,
+        max_new_tokens=max_new_tokens,
+    )
+    return f"{answer}\n\nSources:\n{sources}"
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Ask the Edge Impulse docs RAG assistant.")
+    parser.add_argument("question")
+    parser.add_argument("--index-dir", type=Path, default=DEFAULT_INDEX_DIR)
+    parser.add_argument("--k", type=int, default=4)
+    parser.add_argument("--max-new-tokens", type=int, default=320)
+    parser.add_argument("--api-base", default=DEFAULT_API_BASE)
+    parser.add_argument("--model", default=DEFAULT_MODEL)
+    parser.add_argument("--no-generate", action="store_true", help="Only print retrieved chunks.")
+    args = parser.parse_args()
+    print(
+        ask(
+            args.question,
+            index_dir=args.index_dir,
+            k=args.k,
+            max_new_tokens=args.max_new_tokens,
+            no_generate=args.no_generate,
+            api_base=args.api_base,
+            model=args.model,
+        )
+    )
+
+
+if __name__ == "__main__":
+    main()

requirements.txt

@@ -0,0 +1,5 @@
+faiss-cpu>=1.8
+sentence-transformers>=2.2
+numpy>=1.24
+requests>=2.31
+flask>=3.0

serve.py

@@ -0,0 +1,61 @@
+"""Minimal HTTP server for the Edge Impulse docs RAG assistant.
+
+    python serve.py --host 0.0.0.0 --port 8000
+
+POST /ask   {"question": "...", "k": 4}      -> {"answer": "..."}
+GET  /health                                 -> {"ok": true}
+"""
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+from flask import Flask, jsonify, request
+
+from rag import DEFAULT_API_BASE, DEFAULT_INDEX_DIR, DEFAULT_MODEL, ask
+
+
+def create_app(index_dir: Path, api_base: str, model: str, k: int) -> Flask:
+    app = Flask(__name__)
+
+    @app.get("/health")
+    def health():
+        return jsonify({"ok": True})
+
+    @app.post("/ask")
+    def ask_route():
+        payload = request.get_json(silent=True) or {}
+        question = str(payload.get("question", "")).strip()
+        if not question:
+            return jsonify({"error": "question is required"}), 400
+        try:
+            answer = ask(
+                question,
+                index_dir=index_dir,
+                k=int(payload.get("k", k)),
+                max_new_tokens=int(payload.get("max_new_tokens", 320)),
+                api_base=api_base,
+                model=model,
+            )
+            return jsonify({"answer": answer})
+        except Exception as exc:  # noqa: BLE001 - surface the error to the client
+            return jsonify({"error": str(exc)}), 500
+
+    return app
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Serve the Edge Impulse docs RAG assistant.")
+    parser.add_argument("--host", default="127.0.0.1")
+    parser.add_argument("--port", type=int, default=8000)
+    parser.add_argument("--index-dir", type=Path, default=DEFAULT_INDEX_DIR)
+    parser.add_argument("--api-base", default=DEFAULT_API_BASE)
+    parser.add_argument("--model", default=DEFAULT_MODEL)
+    parser.add_argument("--k", type=int, default=4)
+    args = parser.parse_args()
+    app = create_app(args.index_dir, args.api_base, args.model, args.k)
+    app.run(host=args.host, port=args.port)
+
+
+if __name__ == "__main__":
+    main()