tests

DEPLOY.md

@@ -0,0 +1,104 @@
+# Deploy CarsRUS to Hugging Face Spaces
+
+This guide ensures you can push new versions to [CarsRUS on Hugging Face](https://huggingface.co/spaces/galbendavids/CarsRUS/tree/main) with confidence.
+
+---
+
+## 1. Pre-push checklist (run locally)
+
+Before pushing, run the test suite from the **CarsRUS** directory:
+
+```bash
+cd CarsRUS
+
+# One-liner: run all required tests
+chmod +x run_tests.sh   # once, if needed
+./run_tests.sh
+
+# Or run manually:
+# 1) Business-logic tests (QA – expected behavior from request_file.txt)
+python test_business_logic.py
+
+# 2) RAG engine tests (init, search, normalization, embeddings)
+python test_rag.py
+
+# 3) Optional: agent flow (requires gemini_api in env for full run)
+# python test_agent.py
+```
+
+- **All of `test_business_logic.py` and `test_rag.py` must pass** before you push.
+- If `test_business_logic.py` or `test_rag.py` fails, fix the issue before deploying.
+
+---
+
+## 2. Push to Hugging Face
+
+### First-time setup (once per machine)
+
+1. **Install Hugging Face CLI and log in** (if not already):
+   ```bash
+   pip install huggingface_hub
+   huggingface-cli login
+   ```
+   Use a token with **write** access to the Space (create at [hf.co/settings/tokens](https://huggingface.co/settings/tokens)).
+
+2. **Clone or link the Space repo** (if you don’t have it yet):
+   ```bash
+   git clone https://huggingface.co/spaces/galbendavids/CarsRUS CarsRUS-hf
+   cd CarsRUS-hf
+   ```
+   Or, if your code already lives in a repo that you push to HF:
+   ```bash
+   cd /path/to/your/CarsRUS   # e.g. your workspace CarsRUS folder
+   git remote add hf https://huggingface.co/spaces/galbendavids/CarsRUS
+   ```
+
+### Push a new version
+
+From the **root of the repo that HF Space uses** (e.g. `CarsRUS` or `CarsRUS-hf`):
+
+```bash
+# 1) Run tests (see Pre-push checklist above)
+python test_business_logic.py && python test_rag.py
+
+# 2) Commit changes (if needed)
+git add .
+git commit -m "Your release message, e.g. agentic rag update"
+
+# 3) Push to Hugging Face
+git push hf main
+# or, if your default remote is HF:
+# git push origin main
+```
+
+- Space repo usually uses branch **`main`**. If your Space is set to another branch, push to that branch instead.
+- After push, Hugging Face will rebuild and restart the Space; check the Space **Logs** for errors.
+
+---
+
+## 3. What the tests guarantee (QA)
+
+| Test file | What it checks |
+|-----------|----------------|
+| **test_business_logic.py** | Supported car list matches knowledge base; unsupported car (e.g. BMW X5) returns refusal with supported list; single supported car → no refusal; comparison with 2 supported cars → no refusal; comparison with 1 supported → refusal; car name normalization (RS3→audi_rs3, etc.); chat handles missing `gemini_api` without crashing. |
+| **test_rag.py** | RAG engine init, hybrid search, car normalization, lazy embedding load. |
+| **test_agent.py** | Agent graph and `prepare_generation` (optional; full run needs `gemini_api`). |
+
+---
+
+## 4. Space configuration on Hugging Face
+
+- **SDK**: Gradio (see `README.md` → `sdk: gradio`, `app_file: app.py`).
+- **Secrets**: In the Space **Settings → Repository secrets**, set:
+  - **`gemini_api`**: your Gemini API key (required for chat).
+- **Hardware**: Default CPU is enough; GPU is optional for faster embedding if you change the app later.
+
+---
+
+## 5. After deploy
+
+1. Open the Space: `https://huggingface.co/spaces/galbendavids/CarsRUS`
+2. Check **Logs** for startup errors (e.g. missing `scraped_data.json` or dependencies).
+3. Send a test query (e.g. “Tell me about the Audi RS3”) and confirm the answer is grounded and not a generic error.
+
+If something fails in production, re-run `test_business_logic.py` and `test_rag.py` locally to confirm behavior matches expectations.

README.md

@@ -28,6 +28,17 @@ A lightweight RAG chatbot that answers **Hebrew/English** questions about specif
 
 ---
 
+## Deploy to Hugging Face
+
+Before pushing to [CarsRUS on Hugging Face](https://huggingface.co/spaces/galbendavids/CarsRUS/tree/main), run tests and follow the steps in **[DEPLOY.md](DEPLOY.md)**. Quick check:
+
+```bash
+cd CarsRUS
+./run_tests.sh   # or: python test_business_logic.py && python test_rag.py
+```
+
+---
+
 ## Quick start (local)
 
 ### Prerequisites

agent.py

@@ -144,7 +144,7 @@ def run_stream(engine: RAGEngine, graph, query: str, api_key: str):
     """
     initial: AgentState = {"query": query, "api_key": api_key}
     last_state: AgentState = initial
-    for _node_name, state in graph.stream(initial):
+    for state in graph.stream(initial, stream_mode="values"):
         last_state = state
         steps_log = state.get("steps_log") or []
         refusal = state.get("refusal")

deploy.sh

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# Run tests, then push CarsRUS to Hugging Face (run from CarsRUS directory).
+# Usage: cd CarsRUS && ./deploy.sh
+set -e
+cd "$(dirname "$0")"
+echo "=== 1. Running tests ==="
+python test_business_logic.py
+python test_rag.py
+echo ""
+echo "=== 2. Git add / commit / push ==="
+GIT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null || true)
+if [ -z "$GIT_ROOT" ]; then
+  echo "Not in a git repo. Commit and push manually from your repo root."
+  exit 1
+fi
+# Path from repo root to this folder (e.g. Desktop/carsRUS/CarsRUS)
+REL_PATH=$(git rev-parse --show-prefix 2>/dev/null | sed 's|/$||')
+if [ -z "$REL_PATH" ]; then
+  REL_PATH="."
+fi
+cd "$GIT_ROOT"
+git add "${REL_PATH}/DEPLOY.md" "${REL_PATH}/README.md" "${REL_PATH}/run_tests.sh" "${REL_PATH}/test_business_logic.py" "${REL_PATH}/deploy.sh" 2>/dev/null || true
+git add "${REL_PATH}/app.py" "${REL_PATH}/agent.py" "${REL_PATH}/rag_engine.py" "${REL_PATH}/requirements.txt" "${REL_PATH}/test_agent.py" "${REL_PATH}/test_rag.py" 2>/dev/null || true
+git status --short "${REL_PATH}" | head -20
+git commit -m "CarsRUS: deploy with DevOps/QA tests and DEPLOY.md" || true
+git push origin main
+echo "Done. Space: https://huggingface.co/spaces/galbendavids/CarsRUS"

requirements.txt

@@ -6,4 +6,4 @@ sentence-transformers
 numpy<2.0.0
 torch>=2.0.0
 langgraph>=0.2.0
-langchain-core>=0.3.0
+langchain-core>=0.3.0

tests/run_tests.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# Run all tests before pushing to Hugging Face.
+# Usage: from CarsRUS directory: ./run_tests.sh   or   bash run_tests.sh
+set -e
+cd "$(dirname "$0")"
+echo "Running business-logic tests (test_business_logic.py)..."
+python test_business_logic.py
+echo ""
+echo "Running RAG engine tests (test_rag.py)..."
+python test_rag.py
+echo ""
+echo "All tests passed. Safe to push to Hugging Face."

tests/test_agent.py

@@ -0,0 +1,72 @@
+"""
+Test script for the LangGraph agent pipeline.
+Runs several queries with a short wait between them to verify the full flow.
+Requires gemini_api in environment for real LLM calls; otherwise only tests prepare_generation (no API).
+"""
+
+import os
+import time
+from rag_engine import RAGEngine
+from agent import build_agent_graph, run_stream
+
+
+def main():
+    print("Loading RAG Engine and building agent graph...")
+    engine = RAGEngine()
+    graph = build_agent_graph(engine)
+    print("OK.\n")
+
+    api_key = os.environ.get("gemini_api")
+    if not api_key:
+        print("⚠️  gemini_api not set. Testing only prepare_generation (no LLM calls).\n")
+        test_queries = [
+            "Tell me about the Audi RS3",
+            "Compare Audi RS3 vs Hyundai Elantra N",
+            "מה דעתך על BMW X5?",  # should trigger refusal
+        ]
+        for i, query in enumerate(test_queries, 1):
+            print(f"--- Test {i}: prepare_generation ---")
+            print(f"Query: {query!r}")
+            refusal, sys_p, user_p, steps = engine.prepare_generation(query)
+            if refusal:
+                print(f"Refusal (expected for unsupported car): {refusal[:150]}...")
+            else:
+                print(f"Steps: {len(steps)}; system_prompt length: {len(sys_p or '')}; user_prompt length: {len(user_p or '')}")
+            print()
+        print("Done (prepare_generation only). Set gemini_api to run full agent.")
+        return
+
+    test_queries = [
+        "Tell me about the Audi RS3",
+        "Compare Audi RS3 vs Hyundai Elantra N",
+        "מה היתרונות של קיה EV9?",
+        "מה דעתך על BMW X5?",  # should trigger refusal (unsupported model)
+    ]
+    wait_seconds = 8
+
+    for i, query in enumerate(test_queries, 1):
+        print(f"--- Test {i}/{len(test_queries)} ---")
+        print(f"Query: {query!r}")
+        last_output = None
+        step_count = 0
+        try:
+            for out in run_stream(engine, graph, query, api_key):
+                last_output = out
+                step_count += 1
+            if last_output:
+                preview = last_output[:400] + "..." if len(last_output) > 400 else last_output
+                print(f"Steps yielded: {step_count}; final length: {len(last_output)}")
+                print(f"Final preview:\n{preview}\n")
+            else:
+                print("No output received.\n")
+        except Exception as e:
+            print(f"Error: {e}\n")
+        if i < len(test_queries):
+            print(f"Waiting {wait_seconds}s before next query...")
+            time.sleep(wait_seconds)
+
+    print("All tests finished.")
+
+
+if __name__ == "__main__":
+    main()

tests/test_business_logic.py

@@ -0,0 +1,180 @@
+#!/usr/bin/env python
+"""
+Business-logic test suite for CarsRUS (QA / DevOps).
+
+Validates expected behavior from request_file.txt:
+- Ingest automotive review content → searchable knowledge base
+- Respond based on retrieved knowledge (no hallucination for unsupported cars)
+- Supported cars: Citroen C3, Audi RS3, Kia EV9, MG S6, Hyundai Elantra N, Aion HT, Genesis GV80, Link & Co 01
+- Unsupported car questions → refusal with supported list
+- Comparison: 2 supported cars → proceed; 1 or 0 → refusal
+- Car name normalization (e.g. RS3 → audi_rs3, קיה EV9 → kia_ev9)
+
+Run before pushing to Hugging Face: python test_business_logic.py
+"""
+
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(__file__))
+
+
+def test_supported_cars_list():
+    """Supported models must match the knowledge base (scraped articles)."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    display = engine._supported_cars_display()
+    expected = [
+        "Citroen C3",
+        "Audi RS3",
+        "Kia EV9",
+        "MG S6",
+        "Hyundai Elantra N",
+        "Aion HT",
+        "Genesis GV80",
+        "Link & Co 01",
+    ]
+    assert set(display) == set(expected), f"Supported cars mismatch: got {display}"
+    assert len(display) == 8, f"Expected 8 supported models, got {len(display)}"
+    print("✅ test_supported_cars_list passed")
+
+
+def test_unsupported_car_returns_refusal():
+    """Asking about a car not in the knowledge base must return a refusal with supported list."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    # Hebrew: "What do you think about BMW X5?"
+    query = "מה דעתך על BMW X5?"
+    refusal, sys_prompt, user_prompt, steps = engine.prepare_generation(query)
+    assert refusal is not None, "Unsupported car query must return refusal"
+    assert sys_prompt is None and user_prompt is None, "Refusal must not return prompts"
+    assert "Citroen C3" in refusal or "Audi RS3" in refusal, "Refusal must list supported models"
+    assert "לא נמצא" in refusal or "not in my knowledge" in refusal or "not in my knowledge base" in refusal
+    print("✅ test_unsupported_car_returns_refusal passed")
+
+
+def test_supported_car_single_no_refusal():
+    """Single supported car question must NOT refuse; must return prompts for generation."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    query = "Tell me about the Audi RS3"
+    refusal, sys_prompt, user_prompt, steps = engine.prepare_generation(query)
+    assert refusal is None, "Supported car query must not refuse"
+    assert sys_prompt and user_prompt, "Must return system and user prompts for LLM"
+    assert len(steps) >= 1, "Steps log must be populated"
+    print("✅ test_supported_car_single_no_refusal passed")
+
+
+def test_comparison_two_supported_no_refusal():
+    """Comparison of two supported cars must NOT refuse."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    query = "Compare Audi RS3 vs Hyundai Elantra N"
+    refusal, sys_prompt, user_prompt, steps = engine.prepare_generation(query)
+    assert refusal is None, "Two supported cars comparison must not refuse"
+    assert sys_prompt and user_prompt
+    print("✅ test_comparison_two_supported_no_refusal passed")
+
+
+def test_comparison_one_supported_refusal():
+    """Comparison mentioning only one supported car (or one unsupported) must refuse."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    # "Compare RS3 vs BMW X5" — only RS3 is supported
+    query = "Compare RS3 vs BMW X5"
+    refusal, sys_prompt, user_prompt, steps = engine.prepare_generation(query)
+    assert refusal is not None, "Comparison with unsupported car must refuse"
+    assert "supported" in refusal.lower() or "נתמכים" in refusal
+    print("✅ test_comparison_one_supported_refusal passed")
+
+
+def test_car_name_normalization():
+    """Normalize car names: RS3 → audi_rs3, קיה EV9 → kia_ev9, Citroen C3 → citroen_c3."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    cases = [
+        ("Audi RS3", "audi_rs3"),
+        ("RS3", "audi_rs3"),
+        ("קיה EV9", "kia_ev9"),
+        ("Citroen C3", "citroen_c3"),
+        ("Kia EV9", "kia_ev9"),
+    ]
+    for text, expected in cases:
+        got = engine._normalize_car_name(text)
+        assert got == expected, f"Normalize {text!r}: expected {expected}, got {got}"
+    print("✅ test_car_name_normalization passed")
+
+
+def test_rag_engine_initialization_and_chunks():
+    """RAG engine must load chunks from scraped_data.json (knowledge base exists)."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    assert len(engine.chunks) > 0, "Knowledge base must have at least one chunk"
+    assert len(engine.chunk_metadata) == len(engine.chunks)
+    print("✅ test_rag_engine_initialization_and_chunks passed")
+
+
+def test_hybrid_search_returns_relevant_results():
+    """Hybrid search must return results for a supported car query."""
+    from rag_engine import RAGEngine
+
+    engine = RAGEngine()
+    results = engine._hybrid_search("Tell me about the Audi RS3", top_k=3)
+    assert len(results) >= 1, "Search must return at least one result for supported car"
+    assert "metadata" in results[0] and "text" in results[0]
+    assert "title" in results[0]["metadata"]
+    print("✅ test_hybrid_search_returns_relevant_results passed")
+
+
+def test_chat_function_requires_gemini_key():
+    """App chat must handle missing API key with clear error (no crash)."""
+    from app import chat_function
+
+    # Temporarily unset if set
+    old_key = os.environ.pop("gemini_api", None)
+    try:
+        out = list(chat_function("Tell me about Audi RS3", []))
+        assert len(out) >= 1
+        assert "gemini" in out[0].lower() or "API key" in out[0] or "Configuration" in out[0]
+    finally:
+        if old_key is not None:
+            os.environ["gemini_api"] = old_key
+    print("✅ test_chat_function_requires_gemini_key passed")
+
+
+def run_all():
+    """Run all business-logic tests. Exit 0 if all pass, 1 otherwise."""
+    tests = [
+        test_supported_cars_list,
+        test_car_name_normalization,
+        test_rag_engine_initialization_and_chunks,
+        test_unsupported_car_returns_refusal,
+        test_supported_car_single_no_refusal,
+        test_comparison_two_supported_no_refusal,
+        test_comparison_one_supported_refusal,
+        test_hybrid_search_returns_relevant_results,
+        test_chat_function_requires_gemini_key,
+    ]
+    failed = []
+    for t in tests:
+        try:
+            t()
+        except Exception as e:
+            failed.append((t.__name__, e))
+            print(f"❌ {t.__name__} failed: {e}")
+    if failed:
+        print(f"\n❌ {len(failed)} test(s) failed: {[n for n, _ in failed]}")
+        return 1
+    print("\n✅ All business-logic tests passed.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(run_all())

tests/test_rag.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python
+"""
+Simple test file for RAG Engine
+Tests basic initialization and search functionality
+"""
+
+import sys
+import os
+
+# Add project to path
+sys.path.insert(0, os.path.dirname(__file__))
+
+def test_initialization():
+    """Test RAG engine initialization"""
+    print("\n" + "="*60)
+    print("TEST 1: RAG Engine Initialization")
+    print("="*60)
+    
+    from rag_engine import RAGEngine
+    
+    try:
+        engine = RAGEngine()
+        print(f"✅ Engine initialized successfully")
+        print(f"   - Chunks loaded: {len(engine.chunks)}")
+        print(f"   - Metadata entries: {len(engine.chunk_metadata)}")
+        print(f"   - Keyword index entries: {len(engine.keyword_index)}")
+        print(f"   - Embeddings: {engine.embeddings}")
+        return True, engine
+    except Exception as e:
+        print(f"❌ Initialization failed: {e}")
+        import traceback
+        traceback.print_exc()
+        return False, None
+
+
+def test_search(engine):
+    """Test hybrid search functionality"""
+    print("\n" + "="*60)
+    print("TEST 2: Hybrid Search")
+    print("="*60)
+    
+    try:
+        query = "Tell me about the Audi RS3"
+        print(f"Testing search for: '{query}'")
+        
+        results = engine._hybrid_search(query, top_k=3)
+        print(f"✅ Search successful")
+        print(f"   - Results found: {len(results)}")
+        
+        if results:
+            print(f"   - Top result score: {results[0]['score']:.3f}")
+            print(f"   - Top result title: {results[0]['metadata']['title']}")
+        
+        return True
+    except Exception as e:
+        print(f"❌ Search failed: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+
+def test_car_normalization(engine):
+    """Test car name normalization"""
+    print("\n" + "="*60)
+    print("TEST 3: Car Name Normalization")
+    print("="*60)
+    
+    test_cases = [
+        ("Audi RS3", "audi_rs3"),
+        ("RS3", "audi_rs3"),
+        ("קיה EV9", "kia_ev9"),
+        ("Citroen C3", "citroen_c3"),
+    ]
+    
+    passed = 0
+    failed = 0
+    
+    for text, expected in test_cases:
+        result = engine._normalize_car_name(text)
+        if result == expected:
+            print(f"✅ '{text}' → {result}")
+            passed += 1
+        else:
+            print(f"❌ '{text}' → {result} (expected {expected})")
+            failed += 1
+    
+    print(f"   - Passed: {passed}/{len(test_cases)}")
+    return failed == 0
+
+
+def test_embeddings(engine):
+    """Test that embeddings are lazy loaded"""
+    print("\n" + "="*60)
+    print("TEST 4: Lazy Embedding Loading")
+    print("="*60)
+    
+    try:
+        # Check initial state
+        if engine.embeddings is None:
+            print("✅ Embeddings are None at startup (lazy loading working)")
+        else:
+            print("⚠️  Embeddings already loaded (not lazy)")
+        
+        # Trigger embedding generation
+        query = "Test query"
+        engine._hybrid_search(query, top_k=1)
+        
+        if engine.embeddings is not None:
+            print(f"✅ Embeddings generated after first search")
+            print(f"   - Shape: {engine.embeddings.shape}")
+            print(f"   - Expected chunks: {len(engine.chunks)}")
+            return True
+        else:
+            print(f"❌ Embeddings not generated")
+            return False
+            
+    except Exception as e:
+        print(f"❌ Embedding test failed: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+
+def main():
+    """Run all tests"""
+    print("\n" + "="*60)
+    print("CARSRUS RAG ENGINE TEST SUITE")
+    print("="*60)
+    
+    # Test 1: Initialization
+    success, engine = test_initialization()
+    if not success:
+        print("\n❌ TESTS FAILED - Initialization error")
+        return 1
+    
+    # Test 2: Normalization
+    if not test_car_normalization(engine):
+        print("\n⚠️  Some normalization tests failed")
+    
+    # Test 3: Search
+    if not test_search(engine):
+        print("\n❌ TESTS FAILED - Search error")
+        return 1
+    
+    # Test 4: Embeddings
+    if not test_embeddings(engine):
+        print("\n⚠️  Embedding test had issues")
+    
+    # Summary
+    print("\n" + "="*60)
+    print("✅ ALL CRITICAL TESTS PASSED")
+    print("="*60)
+    print("\nRAG Engine is ready for deployment!")
+    print("- Initialization: ✅")
+    print("- Data loading: ✅")
+    print("- Search functionality: ✅")
+    print("- Lazy loading: ✅")
+    
+    return 0
+
+
+if __name__ == "__main__":
+    exit(main())