Sathya77 commited on
Commit
d0a4921
·
verified ·
1 Parent(s): 43d570f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +73 -3
README.md CHANGED
@@ -1,3 +1,73 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ metrics:
6
+ - bertscore
7
+ - bleu
8
+ - rouge
9
+ base_model:
10
+ - MBZUAI/LaMini-Flan-T5-248M
11
+ pipeline_tag: question-answering
12
+ library_name: transformers
13
+ tags:
14
+ - RAG
15
+ - FAISS
16
+ - Telecom
17
+ - Question-Answering
18
+ - Flan-T5
19
+ - Sentence-Transformers
20
+ ---
21
+
22
+
23
+
24
+ # Telecom Plan Advisor – (RAG-LLM) based Question Answering System
25
+
26
+ Telecom Plan Advisor is a **Retrieval-Augmented Generation (RAG)** system that helps users compare and choose wireless plans from **Bell, Virgin Plus, and Lucky Mobile**.
27
+ It combines **FAISS vector search** (MiniLM embeddings) with a lightweight **seq2seq LLM** (`flan-alpaca-base` or `LaMini-Flan-T5-783M`) to answer plan-related questions in natural language.
28
+
29
+ ## How it works
30
+ - **Retrieve**: FAISS finds the most relevant plan descriptions.
31
+ - **Generate**: LLM produces a concise, friendly answer grounded in retrieved plans.
32
+ - **Evaluate**: System performance measured with BLEU, ROUGE, and BERTScore.
33
+
34
+ # Datasets:
35
+ - Synthetic Wireless Plans Dataset (curated from Bell, Virgin Plus and Lucky Mobile)
36
+
37
+ ## Quickstart
38
+ ```python
39
+ from transformers import pipeline, AutoTokenizer, AutoModelForSeq2SeqLM
40
+ import faiss, pandas as pd
41
+ from sentence_transformers import SentenceTransformer
42
+
43
+ # Load model
44
+ tokenizer = AutoTokenizer.from_pretrained("declare-lab/flan-alpaca-base")
45
+ model = AutoModelForSeq2SeqLM.from_pretrained("declare-lab/flan-alpaca-base")
46
+ qa = pipeline("text2text-generation", model=model, tokenizer=tokenizer)
47
+
48
+ # Example
49
+ print(qa("Best BYOD plan under $50 from Virgin Plus?", max_new_tokens=100)[0]["generated_text"])
50
+ ```
51
+
52
+ 📊 Evaluation Results
53
+
54
+ BLEU: 0.46
55
+
56
+ ROUGE-1: 0.57
57
+
58
+ ROUGE-2: 0.35
59
+
60
+ ROUGE-L: 0.40
61
+
62
+ BERTScore-F1: 0.93
63
+
64
+ - The system was evaluated on a small set of plan-related queries using BLEU, ROUGE, and BERTScore.
65
+ - Sample results show **strong semantic similarity** between generated answers and reference plan descriptions, with **BERTScore F1 around 0.9+**.
66
+
67
+ > Note: BLEU/ROUGE are conservative for free-form LLM outputs, and exact values may vary depending on the test set and chosen base model.
68
+
69
+
70
+ ## Limitations
71
+
72
+ - Works only on the curated dataset (does not fetch live pricing).
73
+ - Region support (Ontario, Quebec, Alberta) is inferred from plan names.