documents/sample.txt

CPU RAG Space — sample knowledge document.

This Space is a fully CPU, free-tier Retrieval-Augmented Generation service.

Architecture:
- Embedding model: BAAI/bge-small-en-v1.5, run via fastembed (ONNX). It turns
  text into 384-dimensional vectors and needs no GPU or PyTorch.
- Vector store: FAISS (IndexFlatIP) holds the document vectors in memory and
  returns the most similar chunks for a query using cosine similarity.
- Language model: Qwen3.5-0.8B (a small Mixture-of-Experts model) in GGUF
  Q4_K_M form, served by llama.cpp. It reads the retrieved chunks and writes a
  grounded answer.

How retrieval works:
1. Your question is embedded into a vector.
2. FAISS finds the top-K most similar document chunks (default K = 4).
3. Those chunks are inserted into the model's system prompt as context.
4. The model answers using only that context and cites the source file.

Why a small model is fine here:
RAG moves knowledge out of the model's weights and into the retriever, so the
model only needs to read and summarise the provided context rather than
memorise facts. That makes a fast, small model like Qwen3.5-0.8B a good fit for
CPU serving.

Replace this file with your own .txt or .md documents, or upload files at
runtime through the web UI, and the Space will answer questions about them.