Sathya77/Telecom_Plan_RAG_based

Telecom Plan Advisor – (RAG-LLM) based Question Answering System

Telecom Plan Advisor is a Retrieval-Augmented Generation (RAG) system that helps users compare and choose wireless plans from Bell, Virgin Plus, and Lucky Mobile. It combines FAISS vector search (MiniLM embeddings) with a lightweight seq2seq LLM (flan-alpaca-base or LaMini-Flan-T5-783M) to answer plan-related questions in natural language.

How it works

• Retrieve: FAISS finds the most relevant plan descriptions.
• Generate: LLM produces a concise, friendly answer grounded in retrieved plans.
• Evaluate: System performance measured with BLEU, ROUGE, and BERTScore.

Datasets:

• Synthetic Wireless Plans Dataset (curated from Bell, Virgin Plus and Lucky Mobile)

Quickstart

from transformers import pipeline, AutoTokenizer, AutoModelForSeq2SeqLM
import faiss, pandas as pd
from sentence_transformers import SentenceTransformer

# Load model
tokenizer = AutoTokenizer.from_pretrained("declare-lab/flan-alpaca-base")
model = AutoModelForSeq2SeqLM.from_pretrained("declare-lab/flan-alpaca-base")
qa = pipeline("text2text-generation", model=model, tokenizer=tokenizer)

# Example
print(qa("Best BYOD plan under $50 from Virgin Plus?", max_new_tokens=100)[0]["generated_text"])

📊 Evaluation Results

BLEU: 0.46

ROUGE-1: 0.57

ROUGE-2: 0.35

ROUGE-L: 0.40

BERTScore-F1: 0.93

• The system was evaluated on a small set of plan-related queries using BLEU, ROUGE, and BERTScore.
• Sample results show strong semantic similarity between generated answers and reference plan descriptions, with BERTScore F1 around 0.9+.

Note: BLEU/ROUGE are conservative for free-form LLM outputs, and exact values may vary depending on the test set and chosen base model.

Limitations

• Works only on the curated dataset (does not fetch live pricing).
• Region support (Ontario, Quebec, Alberta) is inferred from plan names.