AsherKnight commited on
Commit
ad2bdfb
·
1 Parent(s): 3581003
.gitignore ADDED
@@ -0,0 +1 @@
 
 
1
+ .env
README.md CHANGED
@@ -13,6 +13,11 @@ short_description: Smart chatbot for tenancy and property image issues.
13
 
14
  # Multi-Agent Real Estate Chatbot – Detailed Explanation
15
 
 
 
 
 
 
16
  ## Tools & Technologies Used
17
 
18
  ### 1. **Gradio**
@@ -93,4 +98,36 @@ short_description: Smart chatbot for tenancy and property image issues.
93
 
94
  ---
95
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  This detailed overview describes the tools and logic behind the switching mechanism that allows the chatbot to provide contextual and multimodal support effectively.
 
13
 
14
  # Multi-Agent Real Estate Chatbot – Detailed Explanation
15
 
16
+ ## Overview
17
+
18
+ This project is a multi-agent chatbot designed for the real estate domain. It intelligently handles both text-based tenancy FAQs and image-based property issue troubleshooting. Users can interact with the chatbot by typing questions or uploading images, and the system will automatically determine the best agent to respond — whether it's a legal assistant for tenancy issues or an image-based troubleshooting expert.
19
+
20
+
21
  ## Tools & Technologies Used
22
 
23
  ### 1. **Gradio**
 
98
 
99
  ---
100
 
101
+ ## Storage and GPU Limitations: Compromises and Future Work
102
+
103
+ While designing and developing this system, we encountered several constraints due to limited computational resources—especially GPU memory, CPU power, and local/Colab-based VRAM and storage limits. These resource limitations impacted multiple aspects of the solution architecture, leading to compromises in model choice and design. Below are key instances where compromises were made and the proposed future work to address them:
104
+
105
+ ---
106
+
107
+ ### 1. Model Selection for Text Generation
108
+
109
+ Initially, we aimed to use powerful large language models (LLMs) for text generation tasks. However, due to storage and compute limitations, we opted for a smaller variant of the LLaMA model. LLaMA models are generally known for their strong performance in text generation tasks and are open source—making them ideal for POC-level work.
110
+
111
+ - **Compromise**: Used a lightweight LLaMA variant for local compatibility.
112
+ - **Future Work**: Once resources are scaled, we intend to incorporate larger LLaMA models (e.g., `llama-3.1-8b-instruct`) or explore commercial models like GPT-4o or Claude (Anthropic) for enhanced performance and naturalness in generated outputs.
113
+
114
+ ---
115
+
116
+ ### 2. Zero-shot Text Classification for Agent Routing
117
+
118
+ A critical planned feature was dynamic agent switching based on conversation context. For example, if a user, while discussing an image, begins asking tenancy-related questions, a classification pipeline would detect the intent and automatically switch to a relevant agent, passing along the full context. Initially, we used `facebook/bart-large-mnli` for this zero-shot classification task.
119
+
120
+ - **Compromise**: Due to low GPU/CPU/VRAM availability on local setups and Google Colab, we had to remove this functionality.
121
+ - **Future Work**: With access to more powerful hardware or inference APIs, we can reintegrate this feature, significantly improving conversation flow and user experience.
122
+
123
+ ---
124
+
125
+ ### 3. Multi-model Output Scoring Pipeline
126
+
127
+ To boost output quality, we planned to simultaneously generate responses using both LLaMA and Mistral models, and then run a scoring mechanism to select the most relevant response.
128
+
129
+ - **Compromise**: Resource constraints made it infeasible to load and run multiple LLMs in parallel.
130
+ - **Future Work**: Revisit the multi-model setup once better hardware (or hosted services) are available. This will allow ensemble-style approaches for higher quality text generation and response reliability.
131
+
132
+
133
  This detailed overview describes the tools and logic behind the switching mechanism that allows the chatbot to provide contextual and multimodal support effectively.
agents/agent2_tenancy_faq.py CHANGED
@@ -45,7 +45,12 @@ def handle_tenancy_query(user_query, user_context, history=[], location_method="
45
  if location:
46
  user_context["location"] = location
47
 
48
- system_prompt = "You are a legal assistant specializing in tenancy laws."
 
 
 
 
 
49
  prompt=""
50
  if location:
51
  prompt += f" The user is from {location}."
 
45
  if location:
46
  user_context["location"] = location
47
 
48
+ system_prompt = (
49
+ "You are a legal assistant specializing in tenancy laws. "
50
+ "Your primary objective is to provide prompt advice and answers to the user. "
51
+ "Only ask follow-up questions if absolutely necessary and only when you are unclear about the user's request."
52
+ )
53
+
54
  prompt=""
55
  if location:
56
  prompt += f" The user is from {location}."
requirements.txt CHANGED
@@ -5,4 +5,5 @@ Pillow
5
  gradio
6
  ultralytics
7
  spacy
8
- geotext
 
 
5
  gradio
6
  ultralytics
7
  spacy
8
+ geotext
9
+ python-dotenv
utils/llm_utils.py CHANGED
@@ -1,13 +1,13 @@
1
  from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
2
  import torch
3
  import os
4
-
5
  class LLaMAHelper:
6
  def __init__(self, hf_token=None):
7
  self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
8
  self.model_id = "meta-llama/Llama-3.2-3B-Instruct"
9
-
10
- hf_token = hf_token or os.getenv("HUGGINGFACE_TOKEN")
11
 
12
  self.tokenizer = AutoTokenizer.from_pretrained(self.model_id, token=hf_token)
13
  self.model = AutoModelForCausalLM.from_pretrained(
 
1
  from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
2
  import torch
3
  import os
4
+ from dotenv import load_dotenv
5
  class LLaMAHelper:
6
  def __init__(self, hf_token=None):
7
  self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
8
  self.model_id = "meta-llama/Llama-3.2-3B-Instruct"
9
+ load_dotenv()
10
+ hf_token = hf_token or os.getenv("HF_TOKEN")
11
 
12
  self.tokenizer = AutoTokenizer.from_pretrained(self.model_id, token=hf_token)
13
  self.model = AutoModelForCausalLM.from_pretrained(