Fix error LLM API call failed: 404 Client Error: Not Found for url...

README.md

@@ -59,7 +59,7 @@ This application demonstrates how to build a production-ready RAG system within
 │               ┌──────────────────┐    ┌─────────────────┐   │
 │               │  HF Inference    │    │  Sentence       │   │
 │               │  API (LLM)       │    │  Transformers   │   │
-│               │  Zephyr-7B       │    │  (Embeddings)   │   │
+│               │  Mistral-7B      │    │  (Embeddings)   │   │
 │               └──────────────────┘    └─────────────────┘   │
 └─────────────────────────────────────────────────────────────┘
 ```
@@ -144,22 +144,20 @@ results = collection.query(
 Combine retrieved context with the user's question and generate an answer.
 
 1. **Build prompt** — Load the template from [`prompts/rag_prompt.txt`](prompts/rag_prompt.txt), inject retrieved context and the user's question
-2. **Call LLM API** — Send the prompt to the Hugging Face Inference API (Zephyr-7B)
+2. **Call LLM API** — Send the prompt to the Hugging Face Inference API (Mistral-7B-Instruct-v0.3)
 3. **Return response** — The LLM generates an answer grounded in the provided context
 
 The prompt template (`prompts/rag_prompt.txt`):
 
 ```
-<|system|>
-You are a helpful assistant. Answer the user's question based ONLY on the provided context.
+<s>[INST] You are a helpful assistant. Answer the user's question based ONLY on the provided context.
 If the context does not contain enough information to answer, say "I don't have enough information to answer this question."
-Always be concise and factual.</s>
-<|user|>
+Always be concise and factual.
+
 Context:
 {context}
 
-Question: {question}</s>
-<|assistant|>
+Question: {question} [/INST]
 ```
 
 The template is loaded once at startup and filled at query time:
@@ -337,7 +335,7 @@ curl -X POST http://localhost:7860/ingest \
 |---|---|---|
 | `HF_TOKEN` | (Space Secret) | Hugging Face API token for Inference API |
 | `EMBEDDING_MODEL_NAME` | `sentence-transformers/all-MiniLM-L6-v2` | Model for text embeddings |
-| `LLM_MODEL_NAME` | `HuggingFaceH4/zephyr-7b-beta` | LLM for answer generation |
+| `LLM_MODEL_NAME` | `mistralai/Mistral-7B-Instruct-v0.3` | LLM for answer generation |
 | `CHROMA_PERSIST_DIR` | `./chroma_db` | ChromaDB storage path |
 | `CHUNK_SIZE` | `512` | Text chunk size in characters |
 | `CHUNK_OVERLAP` | `50` | Overlap between chunks |
@@ -380,6 +378,14 @@ The `/query` endpoint is designed to be called by external RAG evaluation framew
 
 ## Troubleshooting
 
+### ChromaDB Telemetry Error
+
+If you see `capture() takes 1 positional argument but 3 were given` in the logs, it is a version incompatibility between ChromaDB and the `posthog` library. Telemetry is disabled at startup (`anonymized_telemetry=False`) so this error should no longer appear. If it persists after updating dependencies, pin `posthog<3.0` in `requirements.txt`.
+
+### LLM 404 — Model Not Found
+
+If the `/query` endpoint logs a `404 Not Found` for the Inference API URL, the configured model has been removed from the free serverless tier. Update `LLM_MODEL_NAME` in `app.py` to an available model and adjust `prompts/rag_prompt.txt` to match its instruction format. The current default (`mistralai/Mistral-7B-Instruct-v0.3`) uses the `<s>[INST] … [/INST]` format.
+
 ### SSL Certificate Verification Error
 
 If you see an error like the one below when running `python app.py`, your machine is most likely behind a **corporate proxy that performs SSL inspection** and has its own root CA that Python does not trust by default.

app.py

@@ -39,7 +39,7 @@ logger = logging.getLogger(__name__)
 # ---------------------------------------------------------------------------
 
 EMBEDDING_MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"
-LLM_MODEL_NAME = "HuggingFaceH4/zephyr-7b-beta"
+LLM_MODEL_NAME = "mistralai/Mistral-7B-Instruct-v0.3"
 CHROMA_PERSIST_DIR = "./chroma_db"
 COLLECTION_NAME = "rag_documents"
 CHUNK_SIZE = 512
@@ -81,7 +81,10 @@ logger.info("Embedding model loaded successfully.")
 # because it requires no external database service.
 
 logger.info(f"Initializing ChromaDB at: {CHROMA_PERSIST_DIR}")
-chroma_client = chromadb.PersistentClient(path=CHROMA_PERSIST_DIR)
+chroma_client = chromadb.PersistentClient(
+    path=CHROMA_PERSIST_DIR,
+    settings=Settings(anonymized_telemetry=False),
+)
 collection = chroma_client.get_or_create_collection(
     name=COLLECTION_NAME,
     metadata={"hnsw:space": "cosine"},

prompts/rag_prompt.txt

@@ -1,10 +1,9 @@
-<|system|>
-You are a helpful assistant. Answer the user's question based ONLY on the provided context.
+<s>[INST] You are a helpful assistant. Answer the user's question based ONLY on the provided context.
 If the context does not contain enough information to answer, say "I don't have enough information to answer this question."
-Always be concise and factual.</s>
-<|user|>
+Always be concise and factual.
+
 Context:
 {context}
 
-Question: {question}</s>
-<|assistant|>
+Question: {question} [/INST]
+