methodology.md

Methodology

This project follows a standard RAG (Retrieval-Augmented Generation) workflow with conversational memory:

1. Document Ingestion

   • Load a fixed manual from temp_docs/samsung_manual.txt using TextLoader with UTF-8 to avoid encoding issues.
   • If the file is missing, initialization fails early with a clear error.

2. Preprocessing & Chunking

   • Split the document with RecursiveCharacterTextSplitter (chunk_size=1000, chunk_overlap=200) to balance recall (overlap) and retrieval speed (chunk size).

3. Embedding

   • Convert each chunk to a dense vector using sentence-transformers/all-MiniLM-L6-v2 via HuggingFaceEmbeddings.
   • This small, fast model offers a good latency/quality trade-off for semantic search.

4. Vector Store (Persistence)

   • Store embeddings in ChromaDB (persist_directory=chroma_db).
   • On startup:
     • If chroma_db/ is empty → build the index from the document and persist it.
     • If chroma_db/ exists → load the persisted index directly (fast startup).

5. Retriever

   • Expose the vector store as a retriever with k=2 to fetch the two most relevant chunks per query.

6. LLM Generation

   • Use google/flan-t5-base through a Hugging Face pipeline("text2text-generation"):
     • max_length=512, temperature=0.1, top_p=0.95, repetition_penalty=1.2.
   • The LLM receives the user question plus retrieved context and generates a grounded answer.

7. Conversational Orchestration

   • Wrap everything with ConversationalRetrievalChain to:
     • Retrieve relevant chunks for each turn.
     • Generate answers conditioned on both context and chat history.

8. Memory

   • Maintain multi-turn context using ConversationBufferMemory (return_messages=True), enabling follow-ups like “and what about the warranty?” without repeating details.

9. UI Layer (Gradio)

   • gr.Blocks() app with:
     • Status banner showing whether the DB was built or loaded.
     • gr.Chatbot for messages and a Textbox + Button for input.
   • submit event calls a wrapper that:
     • Appends the user message to chat_history.
     • Invokes the chain and appends the assistant’s answer.

10. Operational Notes

    • Force re-indexing: delete chroma_db/ and restart.
    • Swap documents: replace temp_docs/samsung_manual.txt (keep plain text for best results).
    • Model changes: update MODEL_NAME_EMBEDDINGS or MODEL_ID_LLM in app.py.

Quality & Evaluation (Lightweight)

• Grounding check: ask questions whose answers are known to be in the manual and verify the response cites the right details.
• Follow-up coherence: ask a sequence of related questions to ensure memory works.
• Latency tracking: note first-run time (indexing) vs. warm start (loading persisted DB).

Limitations

• Works best with clean, textual manuals; PDFs should be converted to text first.
• flan-t5-base is compact; for higher fidelity, upgrade to a stronger model (with GPU if available).
• Retrieval uses k=2; adjust if answers miss context or include irrelevant details.