docs/INTERVIEW_GUIDE.md

Telecom RAG - Interview Guide

A comprehensive Q&A guide to explain this project's technical concepts in interviews.

---

Section 1: Project Overview Questions

Q: "Tell me about your project."

I built a Telecom RAG (Retrieval-Augmented Generation) system that provides AI-powered answers for telecom operations support - covering 5G networks, 3GPP standards, troubleshooting, and network KPIs.

It's not just a simple RAG - it implements 6 key innovations:

1. Hybrid Search - combines semantic (dense) vectors with keyword (BM25) search using Reciprocal Rank Fusion
2. Neural Query Router - classifies query intent to pick the optimal search strategy
3. RAGAS Evaluation - 6-metric quality assessment with automatic abstention for low-confidence answers
4. Semantic Caching - deduplicates similar queries via embedding similarity
5. Glossary Enhancement - expands 81+ telecom acronyms for better retrieval
6. Graceful Degradation - works at every level even when components fail

It's deployed on Google Cloud Run and handles a knowledge base of 12,500+ documents from 7 data sources.

Q: "Why did you build this? What problem does it solve?"

Telecom engineers deal with massive documentation - 3GPP specs alone are thousands of pages. They need quick, accurate answers when troubleshooting network issues or checking standards compliance.

The problem with generic LLMs is they hallucinate telecom-specific details. My system grounds every answer in actual retrieved documents, evaluates faithfulness, and refuses to answer when it can't verify the response - which is critical in a domain where wrong answers can cause network outages.

---

Section 2: RAG Architecture Questions

Q: "What is RAG and why did you use it?"

RAG = Retrieval-Augmented Generation. Instead of relying solely on an LLM's parametric knowledge (which can hallucinate), RAG first retrieves relevant documents from a knowledge base, then feeds them as context to the LLM.

Pipeline: Query → Retrieve relevant docs → Build context → LLM generates grounded answer

Why RAG over fine-tuning?

• No retraining needed when documents update (just re-index)
• Can cite specific sources (traceability)
• Can detect when it doesn't have enough information (abstention)
• Much cheaper than fine-tuning large models

Q: "Walk me through what happens when a user asks a question."

9-step pipeline:

1. Rate Limit Check - Sliding window (50 req/min) prevents abuse
2. Cache Check - Embed query, search for similar cached queries (threshold: 0.95 cosine similarity). If hit, return cached answer in <100ms
3. Query Routing - Neural router classifies intent: factual (dense search), procedural (hybrid), or keyword (BM25). Also predicts document category
4. Query Enhancement - Glossary expands acronyms: "HARQ" → "HARQ (Hybrid Automatic Repeat Request)"
5. Hybrid Retrieval - Dense search (ChromaDB) + BM25 keyword search, merged with Reciprocal Rank Fusion
6. Context Building - Top-6 results assembled into a context string with a 1500-token budget
7. LLM Generation - GPT-4o-mini generates a grounded answer using a structured prompt with question repetition
8. RAGAS Evaluation - 6 metrics computed: faithfulness, relevancy, context precision/recall, confidence, trust score. If below threshold → abstain
9. Cache & Return - Cache the response, display with metrics and source citations

Q: "Why hybrid search instead of just semantic search?"

Semantic (dense) search excels at understanding meaning - "How to fix antenna issues" matches documents about "VSWR troubleshooting." But it struggles with exact terms - error codes, alarm IDs, 3GPP specification numbers.

BM25 (keyword) search is the opposite - great for exact matches but misses semantic similarity.

Hybrid search combines both. I use Reciprocal Rank Fusion (RRF) to merge the results:

score(doc) = 1/(60+rank_in_dense) + 1/(60+rank_in_BM25)

This gives documents that appear in both result sets a higher score, while still surfacing documents that are strong in either signal.

---

Section 3: Algorithm Deep Dives

Q: "Explain Reciprocal Rank Fusion. Why k=60?"

Problem: Dense similarity scores (0 to 1) and BM25 scores (0 to unbounded) are on different scales. You can't simply add them.

RRF Solution: Instead of using raw scores, use rank positions:

RRF(doc) = Σ 1/(k + rank_i)

Why k=60? It's a smoothing constant. A smaller k (like 1) would heavily favor top-ranked documents. k=60 makes the fusion more democratic - the difference between rank 1 and rank 5 is smaller, so both dense and BM25 signals contribute meaningfully. This value was empirically validated in the Telco-RAG paper and is also standard in NIST TREC benchmarks.

Q: "How does your evaluation system work?"

I implemented a RAGAS-style evaluation with 6 metrics:

1. Faithfulness - Extract factual claims from the answer, check each against the context via keyword overlap. Score = supported/total claims
2. Relevancy - Measure keyword overlap between question terms and answer terms (technical terms weighted 2x)
3. Context Precision - What fraction of retrieved chunks were actually relevant (similarity > 0.5)
4. Context Recall - What fraction of answer claims are covered by the context
5. Confidence - Average of top-3 retrieval similarity scores
6. Trust Score - Weighted combination: 40% faithfulness + 30% relevancy + 20% precision + 10% confidence

Abstention Logic: If any metric drops below 0.3 (very low), the system refuses to answer and explains why. This prevents hallucinated responses in a safety-critical domain.

There's also an optional LLM-as-judge mode where GPT-4o-mini itself scores faithfulness and relevancy (more accurate but slower).

Q: "What is HyDE and why did you implement it?"

HyDE = Hypothetical Document Embeddings.

Problem: Short queries like "What is HARQ?" don't have much semantic content for embedding-based search.

Solution: Ask the LLM to generate a hypothetical ideal answer, then search for documents similar to that hypothetical answer.

Why it works: The hypothetical answer is closer in embedding space to actual documents about HARQ than the short query alone. It bridges the "query-document gap."

Trade-off: +2-3.5% accuracy but adds an extra LLM call (~1s latency). That's why I made it optional and disabled by default.

Q: "How does the query router work?"

It's a prototype-based nearest neighbor classifier using embedding similarity:

1. Pre-compute embeddings for 6 prototype questions per strategy (DENSE, HYBRID, KEYWORD)
2. When a query arrives, embed it
3. Compute cosine similarity against all prototypes
4. Pick the strategy whose prototypes have the highest max similarity

Example:

• "What is 5G NR?" → highest sim with DENSE prototypes → pure semantic search
• "How to fix VSWR alarm?" → highest sim with HYBRID prototypes → hybrid search
• "Error 5301" → highest sim with KEYWORD prototypes → BM25 search

This is lightweight (no ML training needed) and effective because telecom queries follow predictable patterns.

---

Section 4: Infrastructure & Deployment

Q: "How did you deploy this?"

Google Cloud Run - serverless container platform.

Key decisions:

• Scale-to-zero: Min instances = 0, so no cost when idle
• Pre-built indexes: ChromaDB and BM25 indexes are compressed into the Docker image so cold starts don't require rebuilding
• Feature flags: Reranker disabled on cloud (saves ~1GB memory), Redis disabled (no managed instance), hybrid search enabled
• Environment variables: API keys passed via --set-env-vars, not baked into the image

Deployment flow: deploy-cloudbuild.sh → sources .env → gcloud run deploy --source=. → Cloud Build creates Docker image → deploys to Cloud Run

Q: "How do you handle cold starts?"

Cloud Run cold starts are a challenge. My mitigations:

1. Pre-built ChromaDB - compressed as chroma_db.tar.gz in the Docker image, extracted at build time
2. Cached BM25 index - serialized as bm25_index.pkl, loaded from cache on startup instead of rebuilding from documents
3. OpenAI embeddings - no local model download needed (vs sentence-transformers which needs ~1GB download)
4. Lazy reranker - cross-encoder model only loaded on first use (not at startup)
5. Streamlit caching - @st.cache_resource ensures components initialize once per instance

Q: "Explain your graceful degradation strategy."

Every component is designed to fail gracefully:

Component Failure	Fallback
Redis down	In-memory cache (non-persistent)
LLM unavailable	Returns raw retrieved context
Reranker fails to load	Skips reranking, returns hybrid results
BM25 index missing	Falls back to dense-only search
HuggingFace rate limit	Falls back to public dataset, then built-in KB
OpenAI API key invalid	Shows helpful error with diagnostic command

The system always tries to provide something useful rather than crashing. In production, this means the service stays up even when external dependencies have issues.

---

Section 5: Design Decisions

Q: "Why ChromaDB instead of Pinecone/Weaviate/FAISS?"

ChromaDB was chosen because:

• Embedded - runs inside the application process, no separate database server needed
• Persistent - data survives container restarts via disk storage
• Lightweight - fits within Cloud Run's 4GB memory limit
• Python-native - first-class Python API, easy to integrate

Why not alternatives:

• Pinecone - managed service, adds latency for network calls, costs money per vector
• FAISS - no built-in persistence, metadata filtering is manual
• Weaviate - requires separate server, overkill for this scale

Q: "Why GPT-4o-mini instead of GPT-4o?"

Cost vs quality trade-off:

• GPT-4o-mini: ~$0.15/1M input tokens, ~$0.60/1M output tokens
• GPT-4o: ~$5/1M input tokens, ~$15/1M output tokens

For RAG, the quality difference is minimal because the answer quality depends more on the retrieved context than the model's parametric knowledge. GPT-4o-mini is 30x cheaper and 2-3x faster while being sufficient for summarizing and citing retrieved documents.

Q: "Why 125-token chunks?"

Based on the Telco-RAG paper and empirical testing:

• Too small (50 tokens): loses context, fragments sentences
• Too large (500 tokens): reduces retrieval precision, wastes token budget
• 125 tokens is optimal for Q&A-style telecom documents

I also use dynamic chunk sizing based on category:

• Standards docs: 125 tokens (concise, factual)
• Network operations: 250 tokens (event-based, needs more context)
• Performance data: 500 tokens (time-series, needs surrounding context)

Q: "Why did you set the cache similarity threshold at 0.95?"

It's intentionally strict because we want cache hits only for nearly identical questions:

• "What is HARQ in 5G?" and "What is HARQ in 5G NR?" → same intent, should cache
• "What is HARQ?" and "What is MIMO?" → different intent, should NOT cache

At 0.95, only near-paraphrases match. At 0.90, you'd get false positives that return answers for different questions. The cost of a cache miss (re-running the pipeline, ~3s) is much lower than the cost of returning a wrong cached answer.

---

Section 6: Challenges & Learnings

Q: "What was the hardest challenge?"

Balancing retrieval precision vs recall in a specialized domain.

Telecom has thousands of similar-sounding concepts (PDSCH/PUSCH, gNB/eNB, FR1/FR2). Pure semantic search would often retrieve the wrong related concept. The hybrid search with BM25 solved this because keyword matching catches exact acronyms that embeddings might confuse.

Another challenge was empty answers from HuggingFace datasets - many TeleQnA entries had answer indices pointing to empty choices. I had to implement validation to filter these out (hence the "data quality report" in the ingestion pipeline).

Q: "What would you do differently if starting over?"

1. Use a vector DB with built-in hybrid search (like Qdrant or Weaviate) instead of managing separate BM25 + ChromaDB
2. Implement streaming responses - currently the UI waits for the full pipeline, but users would benefit from seeing partial results
3. Use Google Cloud Secret Manager instead of environment variables for API keys
4. Add automated evaluation with a test suite of known Q&A pairs to catch regressions

---

Section 7: Metrics & Performance

Q: "What are the performance characteristics?"

Metric	Value
Cold start	~10-15s (Cloud Run)
Warm query (cache miss)	~2-4s
Cache hit	<100ms
Knowledge base	12,500+ docs
Index size	~500MB
Memory usage	~2-3GB
Concurrent users	~10-20 (per instance)

Q: "How would you scale this?"

Horizontal: Cloud Run auto-scales to 10 instances. Each instance handles its own requests independently since ChromaDB is embedded and read-only after ingestion.

If I needed more:

1. Move to a managed vector DB (Pinecone/Qdrant) for shared state across instances
2. Add a Redis cluster for shared caching (currently per-instance)
3. Use Cloud CDN for static Streamlit assets
4. Implement async LLM calls to handle more concurrent requests per instance
5. Consider Google Cloud Memorystore for managed Redis

---

Quick Reference: Key Technical Terms

Term	What It Is	Where It's Used
RAG	Retrieval-Augmented Generation	Core architecture pattern
RRF	Reciprocal Rank Fusion	Merging dense + BM25 results
HyDE	Hypothetical Document Embeddings	Optional retrieval improvement
BM25	Best Matching 25 (TF-IDF variant)	Keyword/sparse search
HNSW	Hierarchical Navigable Small World	ChromaDB's vector index
RAGAS	RAG Assessment framework	Answer quality evaluation
TLM	Trustworthy Language Model	Trust score computation
SON	Self-Organizing Networks	Telecom domain concept
HARQ	Hybrid Automatic Repeat Request	Common telecom example query
Cross-Encoder	Model scoring query-doc pairs	Reranking stage