riazmo commited on
Commit
2def748
Β·
verified Β·
1 Parent(s): bf00b99

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +807 -138
app.py CHANGED
@@ -1,188 +1,857 @@
1
- # πŸ”„ AUTOMATIC RESET - Add This to Your app.py
2
-
3
- ## πŸ“ WHERE TO ADD THE CODE
4
-
5
- Find this section in your **app.py** (around line 200-220):
6
-
7
- ```python
8
- # After clicking "Start Analysis"
9
- if st.button("πŸš€ Start Analysis", type="primary"):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
- # Set HF API key in environment
12
- os.environ['HUGGINGFACE_API_KEY'] = hf_api_key
 
 
 
 
 
13
 
14
- # Scrape reviews
15
- st.info("Scraping reviews...")
16
- scraper.scrape_all_sources()
 
 
 
17
 
18
- # Initialize database
19
- db = EnhancedDatabase()
20
- db.connect()
21
- db.enhance_schema()
 
22
 
23
- # ... rest of code
24
- ```
 
 
 
 
 
 
 
 
 
 
 
25
 
26
- ---
27
 
28
- ## βœ… ADD THESE 3 LINES
29
 
30
- **BEFORE** calling `db.enhance_schema()`, add this:
31
 
32
- ```python
33
- # After clicking "Start Analysis"
34
- if st.button("πŸš€ Start Analysis", type="primary"):
35
 
36
- # Set HF API key in environment
37
- os.environ['HUGGINGFACE_API_KEY'] = hf_api_key
 
 
38
 
39
- # Scrape reviews
40
- st.info("Scraping reviews...")
41
- scraper.scrape_all_sources()
 
42
 
43
- # Initialize database
44
- db = EnhancedDatabase()
45
- db.connect()
 
46
 
47
- # ⭐ ADD THESE 3 LINES ⭐
48
- # Reset the most recent 20 reviews to pending status
49
- st.info("Preparing reviews for processing...")
50
- db.reset_processing_status(limit=20)
51
 
52
- # Continue with existing code
53
- db.enhance_schema()
 
 
 
 
54
 
55
- # ... rest of code
56
- ```
 
 
 
 
57
 
58
- ---
59
 
60
- ## 🎯 COMPLETE MODIFICATION
 
 
61
 
62
- Here's the complete section with the fix highlighted:
 
63
 
64
- ```python
65
- # After clicking "Start Analysis"
66
- if st.button("πŸš€ Start Analysis", type="primary"):
67
-
68
- # Validate API key
69
- if not hf_api_key or len(hf_api_key) < 10:
70
- st.error("❌ Please enter a valid HuggingFace API key!")
71
- st.stop()
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
- # Set environment variable
74
- os.environ['HUGGINGFACE_API_KEY'] = hf_api_key
 
 
75
 
76
- # Create progress container
77
- progress_container = st.container()
 
78
 
79
- with progress_container:
80
- # Step 1: Scraping
81
- st.info("πŸ” Scraping reviews from App Store and Play Store...")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
 
83
- # Initialize scraper
84
- scraper = ReviewScraper()
85
- scraper.scrape_all_sources()
86
 
87
- # Step 2: Database setup
88
- st.info("πŸ“ Setting up database...")
89
- db = EnhancedDatabase()
90
- db.connect()
 
 
 
91
 
92
- # ⭐⭐⭐ CRITICAL: ADD THESE 3 LINES ⭐⭐⭐
93
- # Reset the most recent reviews to pending status
94
- st.info("πŸ”„ Preparing reviews for processing...")
95
- reset_count = db.reset_processing_status(limit=20)
 
 
 
96
 
97
- # Continue with schema enhancement
98
- db.enhance_schema()
 
 
 
99
 
100
- # Step 3: Get pending reviews
101
- reviews_to_process = db.get_pending_reviews()
102
 
103
- if not reviews_to_process:
104
- st.warning("⚠️ No reviews found to process!")
105
- st.stop()
106
 
107
- st.success(f"βœ… Found {len(reviews_to_process)} reviews to process!")
 
108
 
109
- # ... rest of your processing code
110
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
 
112
- ---
113
 
114
- ## 🎊 WHAT THIS DOES
115
 
116
- 1. **Scrapes reviews** from App Store and Play Store
117
- 2. **Resets the 20 most recent reviews** to `pending` status
118
- 3. **Processes them** through your AI pipeline
119
- 4. **Shows results** in the dashboard
120
 
121
- Even if reviews were already processed before, they'll be reprocessed with the latest AI models!
122
 
123
- ---
124
 
125
- ## πŸ“‹ DEPLOYMENT CHECKLIST
126
 
127
- - [ ] Upload `database_enhanced_UPDATED.py` to HF Spaces
128
- - [ ] Rename to `database_enhanced.py`
129
- - [ ] Add the 3 lines to your `app.py` (as shown above)
130
- - [ ] Upload `langgraph_nodes_FINAL.py` to HF Spaces
131
- - [ ] Rename to `langgraph_nodes.py`
132
- - [ ] Commit all changes
133
- - [ ] Wait for rebuild (2 min)
134
- - [ ] Enter your API key
135
- - [ ] Click "Start Analysis"
136
- - [ ] Watch it work! ✨
 
 
 
 
 
 
 
 
 
 
 
 
 
 
137
 
138
- ---
139
 
140
- ## πŸš€ EXPECTED BEHAVIOR
141
 
142
- **Before Fix:**
143
- ```
144
- βœ… Scraped 20 reviews
145
- βœ… Saved 0 new reviews ← Nothing to process!
146
- ⚠️ No reviews found to process
147
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
148
 
149
- **After Fix:**
150
- ```
151
- βœ… Scraped 20 reviews
152
- βœ… Saved 0 new reviews
153
- πŸ”„ Reset 20 reviews to pending status ← Force reprocessing!
154
- βœ… Found 20 reviews to process!
155
- πŸ“ Review ID: abc123
156
- βœ… Stage 1 complete (4.23s)
157
- βœ… Stage 2 complete (0.83s)
158
- βœ… Stage 3 complete (3.17s)
159
- ```
160
 
161
- ---
162
 
163
- ## πŸ’‘ ALTERNATIVE: Change the Limit
 
 
164
 
165
- If you want to process **all reviews** instead of just the latest 20:
166
 
167
- ```python
168
- # Reset ALL reviews
169
- db.reset_processing_status() # No limit parameter
170
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
171
 
172
- Or process only the latest 5:
173
 
174
- ```python
175
- # Reset only 5 most recent
176
- db.reset_processing_status(limit=5)
177
- ```
178
 
179
- ---
180
 
181
- ## ⚑ Quick Summary
182
 
183
- **3 files to upload:**
184
- 1. `database_enhanced_UPDATED.py` β†’ rename to `database_enhanced.py`
185
- 2. `langgraph_nodes_FINAL.py` β†’ rename to `langgraph_nodes.py`
186
- 3. Modify `app.py` β†’ add 3 lines as shown above
187
 
188
- **Then it will work!** πŸŽ‰
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ HuggingFace Spaces - Review Intelligence System (Streamlit)
3
+ Complete app with URL input, progress tracking, and interactive dashboard
4
+ FIXED VERSION - Better UI contrast + Proper field mapping
5
+ """
6
+
7
+ import streamlit as st
8
+ import pandas as pd
9
+ import plotly.express as px
10
+ import plotly.graph_objects as go
11
+ import os
12
+ from datetime import datetime
13
+ from typing import List, Dict, Optional
14
+ import time
15
+
16
+ from gradio_pipeline import GradioPipeline
17
+
18
+
19
+ # ============================================================================
20
+ # PAGE CONFIGURATION
21
+ # ============================================================================
22
+
23
+ st.set_page_config(
24
+ page_title="Review Intelligence System",
25
+ page_icon="🎯",
26
+ layout="wide",
27
+ initial_sidebar_state="expanded"
28
+ )
29
+
30
+ # FIXED Custom CSS - Better Contrast
31
+ st.markdown("""
32
+ <style>
33
+ .main {
34
+ padding: 0rem 1rem;
35
+ }
36
+
37
+ /* FIXED: Metric cards with better contrast */
38
+ .stMetric {
39
+ background: linear-gradient(135deg, #1e3a8a 0%, #3b82f6 100%);
40
+ padding: 20px;
41
+ border-radius: 10px;
42
+ box-shadow: 0 4px 6px rgba(0, 0, 0, 0.3);
43
+ border: 1px solid #60a5fa;
44
+ }
45
 
46
+ .stMetric label {
47
+ color: #dbeafe !important;
48
+ font-size: 14px !important;
49
+ font-weight: 600 !important;
50
+ text-transform: uppercase;
51
+ letter-spacing: 0.5px;
52
+ }
53
 
54
+ .stMetric [data-testid="stMetricValue"] {
55
+ color: #ffffff !important;
56
+ font-size: 36px !important;
57
+ font-weight: bold !important;
58
+ text-shadow: 0 2px 4px rgba(0,0,0,0.2);
59
+ }
60
 
61
+ .stMetric [data-testid="stMetricDelta"] {
62
+ color: #93c5fd !important;
63
+ font-size: 14px !important;
64
+ font-weight: 600 !important;
65
+ }
66
 
67
+ .big-font {
68
+ font-size: 24px !important;
69
+ font-weight: bold;
70
+ }
71
+
72
+ .success-box {
73
+ padding: 25px;
74
+ border-radius: 12px;
75
+ background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
76
+ color: white;
77
+ margin: 20px 0;
78
+ box-shadow: 0 8px 16px rgba(0, 0, 0, 0.3);
79
+ }
80
 
 
81
 
 
82
 
 
83
 
 
 
 
84
 
85
+ .success-box h1 {
86
+ color: white !important;
87
+ text-shadow: 0 2px 4px rgba(0,0,0,0.2);
88
+ }
89
 
90
+ /* Info boxes */
91
+ .stAlert {
92
+ border-radius: 8px;
93
+ }
94
 
95
+ /* Better table styling */
96
+ .dataframe {
97
+ border: 1px solid #e2e8f0 !important;
98
+ }
99
 
100
+ /* Tab styling */
101
+ .stTabs [data-baseweb="tab-list"] {
102
+ gap: 8px;
103
+ }
104
 
105
+ .stTabs [data-baseweb="tab"] {
106
+ background-color: #1e293b;
107
+ border-radius: 8px 8px 0 0;
108
+ padding: 12px 24px;
109
+ color: #94a3b8;
110
+ }
111
 
112
+ .stTabs [aria-selected="true"] {
113
+ background-color: #3b82f6;
114
+ color: white;
115
+ }
116
+ </style>
117
+ """, unsafe_allow_html=True)
118
 
 
119
 
120
+ # ============================================================================
121
+ # SESSION STATE INITIALIZATION
122
+ # ============================================================================
123
 
124
+ if 'processing_complete' not in st.session_state:
125
+ st.session_state.processing_complete = False
126
 
127
+ if 'results' not in st.session_state:
128
+ st.session_state.results = None
129
+
130
+ if 'insights' not in st.session_state:
131
+ st.session_state.insights = None
132
+
133
+ if 'scraped_count' not in st.session_state:
134
+ st.session_state.scraped_count = 0
135
+
136
+
137
+
138
+ # ============================================================================
139
+ # PROCESSING FUNCTIONS
140
+ # ============================================================================
141
+
142
+ def process_reviews_streamlit(app_store_urls: str, play_store_urls: str,
143
+ hf_api_key: str, review_limit: int):
144
+ """
145
+ Process reviews with Streamlit progress tracking
146
+ """
147
 
148
+ # Validate inputs
149
+ if not hf_api_key or not hf_api_key.strip():
150
+ st.error("❌ Please provide your HuggingFace API key")
151
+ return False
152
 
153
+ if not app_store_urls.strip() and not play_store_urls.strip():
154
+ st.error("❌ Please provide at least one App Store or Play Store URL")
155
+ return False
156
 
157
+ try:
158
+ # Set API key
159
+ os.environ['HUGGINGFACE_API_KEY'] = hf_api_key.strip()
160
+
161
+ # Progress indicators
162
+ progress_bar = st.progress(0)
163
+ status_text = st.empty()
164
+
165
+ # Initialize pipeline
166
+ status_text.text("πŸš€ Initializing pipeline...")
167
+ progress_bar.progress(5)
168
+ pipeline = GradioPipeline(review_limit=review_limit)
169
+
170
+ # Parse URLs
171
+ app_urls = [url.strip() for url in app_store_urls.split('\n') if url.strip()]
172
+ play_urls = [url.strip() for url in play_store_urls.split('\n') if url.strip()]
173
+
174
+ # Stage 0: Scraping
175
+ status_text.text("πŸ•·οΈ Scraping reviews from stores...")
176
+ progress_bar.progress(10)
177
 
178
+ scraped_count = 0
179
+ total_apps = len(app_urls) + len(play_urls)
 
180
 
181
+ for i, app_id in enumerate(app_urls, 1):
182
+ status_text.text(f"🍎 Scraping App Store ({i}/{total_apps}): {app_id}")
183
+ reviews = pipeline.scraper.scrape_app_store_rss(app_id, country="ae", limit=review_limit)
184
+ saved = pipeline.scraper.save_reviews_to_db(reviews)
185
+ scraped_count += saved
186
+ progress_bar.progress(10 + int(20 * i / total_apps))
187
+ time.sleep(1)
188
 
189
+ for i, package in enumerate(play_urls, 1):
190
+ status_text.text(f"πŸ€– Scraping Play Store ({i}/{total_apps}): {package}")
191
+ reviews = pipeline.scraper.scrape_play_store_api(package, country="ae", limit=review_limit)
192
+ saved = pipeline.scraper.save_reviews_to_db(reviews)
193
+ scraped_count += saved
194
+ progress_bar.progress(10 + int(20 * (len(app_urls) + i) / total_apps))
195
+ time.sleep(1)
196
 
197
+ if scraped_count == 0:
198
+ st.warning("⚠️ No reviews scraped. Please check your URLs and try again.")
199
+ progress_bar.empty()
200
+ status_text.empty()
201
+ return False
202
 
203
+ st.session_state.scraped_count = scraped_count
 
204
 
205
+ # Stage 1-3: Processing
206
+ status_text.text("πŸ€– Processing reviews with AI models...")
207
+ progress_bar.progress(30)
208
 
209
+ reviews = pipeline.db.get_pending_reviews(limit=review_limit)
210
+ total_reviews = len(reviews)
211
 
212
+ print(f"πŸ“Š DEBUG: Found {total_reviews} reviews to process")
213
+
214
+ processed_states = []
215
+
216
+ for i, review in enumerate(reviews, 1):
217
+ review_id = review.get('review_id', 'unknown')[:20]
218
+ status_text.text(f"πŸ€– Processing review {i}/{total_reviews}: {review_id}...")
219
+ progress_bar.progress(30 + int(60 * i / total_reviews))
220
+
221
+ try:
222
+ from langgraph_state import create_initial_state
223
+ state = create_initial_state(review)
224
+ config = {"configurable": {"thread_id": f"review_{review.get('review_id')}"}}
225
+ final_state = pipeline.review_graph.invoke(state, config=config)
226
+
227
+ # Convert to dict
228
+ state_dict = dict(final_state)
229
+ processed_states.append(state_dict)
230
+
231
+ # DEBUG: Print what we got
232
+ print(f"βœ… Processed {review_id}:")
233
+ print(f" Type: {state_dict.get('classification_type', 'MISSING')}")
234
+ print(f" Dept: {state_dict.get('department', 'MISSING')}")
235
+ print(f" Sentiment: {state_dict.get('final_sentiment', 'MISSING')}")
236
+
237
+ except Exception as e:
238
+ st.warning(f"⚠️ Error processing review: {str(e)}")
239
+ print(f"❌ ERROR: {e}")
240
+ import traceback
241
+ print(traceback.format_exc())
242
+ continue
243
+
244
+ if len(processed_states) == 0:
245
+ st.error("❌ No reviews were processed successfully.")
246
+ progress_bar.empty()
247
+ status_text.empty()
248
+ return False
249
+
250
+ # Stage 4: Batch Analysis
251
+ status_text.text("πŸ“Š Generating batch insights...")
252
+ progress_bar.progress(90)
253
+
254
+
255
+ insights = pipeline.analyze_batch(processed_states)
256
+
257
+
258
+ # Store in session state
259
+ st.session_state.results = processed_states
260
+ st.session_state.insights = insights
261
+ st.session_state.processing_complete = True
262
+
263
+ # Complete
264
+ progress_bar.progress(100)
265
+ status_text.text("βœ… Analysis complete!")
266
+ time.sleep(1)
267
+ progress_bar.empty()
268
+ status_text.empty()
269
+
270
+ return True
271
+
272
+ except Exception as e:
273
+ st.error(f"❌ Error during processing: {str(e)}")
274
+ import traceback
275
+ st.code(traceback.format_exc())
276
+ return False
277
+
278
+
279
+
280
+ # ============================================================================
281
+ # VISUALIZATION FUNCTIONS
282
+ # ============================================================================
283
+
284
+ def create_summary_section(scraped_count: int, results: List[Dict], insights: Dict):
285
+ """Create summary metrics section"""
286
+
287
+ total = len(results)
288
+ positive = insights.get('sentiment_distribution', {}).get('POSITIVE', 0)
289
+ neutral = insights.get('sentiment_distribution', {}).get('NEUTRAL', 0)
290
+ negative = insights.get('sentiment_distribution', {}).get('NEGATIVE', 0)
291
+ critical = insights.get('priority_distribution', {}).get('critical', 0)
292
+ churn_risk = insights.get('churn_risk', 0)
293
+
294
+ # Success header
295
+ st.markdown(
296
+ f"""
297
+ <div class="success-box">
298
+ <h1 style="margin: 0;">βœ… Analysis Complete!</h1>
299
+ <p style="margin: 10px 0 0 0; font-size: 1.2em; opacity: 0.9;">
300
+ Review Intelligence System Results
301
+ </p>
302
+ </div>
303
+ """,
304
+ unsafe_allow_html=True
305
+ )
306
+
307
+ # Metrics with better styling
308
+ col1, col2, col3, col4, col5 = st.columns(5)
309
+
310
+ with col1:
311
+ st.metric("πŸ“Š Total Reviews", total, f"Scraped: {scraped_count}")
312
+
313
+ with col2:
314
+ pos_pct = (positive / total * 100) if total > 0 else 0
315
+ st.metric("😊 Positive", positive, f"{pos_pct:.1f}%")
316
+
317
+ with col3:
318
+ neg_pct = (negative / total * 100) if total > 0 else 0
319
+ st.metric("😞 Negative", negative, f"{neg_pct:.1f}%")
320
+
321
+ with col4:
322
+ st.metric("🚨 Critical", critical, "⚠️" if critical > 0 else "βœ…")
323
+
324
+ with col5:
325
+ st.metric("πŸ“‰ Churn Risk", f"{churn_risk:.1f}%",
326
+ "πŸ”΄ High" if churn_risk > 30 else "🟒 Low")
327
+
328
+ # Recommendations
329
+ if insights.get('recommendations'):
330
+ st.markdown("### πŸ’‘ Key Recommendations")
331
+ for rec in insights.get('recommendations', []):
332
+ st.info(rec)
333
+
334
+
335
+
336
+ def create_sentiment_chart(insights: Dict):
337
+ """Create sentiment distribution donut chart"""
338
+ sentiment_dist = insights.get('sentiment_distribution', {})
339
+
340
+ labels = list(sentiment_dist.keys())
341
+ values = list(sentiment_dist.values())
342
+ colors = ['#10b981', '#f59e0b', '#ef4444']
343
+
344
+ fig = go.Figure(data=[go.Pie(
345
+ labels=labels,
346
+ values=values,
347
+ hole=0.5,
348
+ marker_colors=colors,
349
+ textinfo='label+percent',
350
+ textposition='outside',
351
+ textfont_size=14
352
+ )])
353
+
354
+ fig.update_layout(
355
+ title="😊 Sentiment Distribution",
356
+ showlegend=True,
357
+ height=400
358
+ )
359
+
360
+ return fig
361
+
362
+
363
+
364
+ def create_priority_chart(insights: Dict):
365
+ """Create priority distribution bar chart"""
366
+ priority_dist = insights.get('priority_distribution', {})
367
+
368
+ priority_order = ['critical', 'high', 'medium', 'low']
369
+ labels = [p for p in priority_order if p in priority_dist]
370
+ values = [priority_dist.get(p, 0) for p in labels]
371
+ colors = ['#dc2626', '#f59e0b', '#3b82f6', '#10b981']
372
+
373
+ fig = go.Figure(data=[go.Bar(
374
+ x=labels,
375
+ y=values,
376
+ marker_color=colors[:len(labels)],
377
+ text=values,
378
+ textposition='auto'
379
+ )])
380
+
381
+ fig.update_layout(
382
+ title="🎯 Priority Levels",
383
+ xaxis_title="Priority",
384
+ yaxis_title="Count",
385
+ height=400
386
+ )
387
+
388
+ return fig
389
+
390
+
391
+
392
+ def create_department_chart(insights: Dict):
393
+ """Create department routing horizontal bar chart"""
394
+ dept_dist = insights.get('department_distribution', {})
395
+
396
+ labels = list(dept_dist.keys())
397
+ values = list(dept_dist.values())
398
+
399
+ fig = go.Figure(data=[go.Bar(
400
+ x=values,
401
+ y=labels,
402
+ orientation='h',
403
+ marker_color='#667eea',
404
+ text=values,
405
+ textposition='auto'
406
+ )])
407
+
408
+ fig.update_layout(
409
+ title="🏒 Department Routing",
410
+ xaxis_title="Number of Issues",
411
+ yaxis_title="Department",
412
+ height=400
413
+ )
414
+
415
+ return fig
416
+
417
 
 
418
 
 
419
 
 
 
 
 
420
 
 
421
 
 
422
 
 
423
 
424
+ def create_emotion_chart(insights: Dict):
425
+ """Create emotion distribution chart"""
426
+ emotion_dist = insights.get('emotion_distribution', {})
427
+
428
+ labels = list(emotion_dist.keys())
429
+ values = list(emotion_dist.values())
430
+
431
+ fig = px.bar(
432
+ x=labels,
433
+ y=values,
434
+ labels={'x': 'Emotion', 'y': 'Count'},
435
+ color=values,
436
+ color_continuous_scale='Viridis'
437
+ )
438
+
439
+ fig.update_layout(
440
+ title="😊 Emotional Analysis",
441
+ xaxis_title="Emotion Type",
442
+ yaxis_title="Number of Reviews",
443
+ height=300,
444
+ showlegend=False
445
+ )
446
+
447
+ return fig
448
 
 
449
 
 
450
 
451
+ def create_reviews_dataframe(results: List[Dict]) -> pd.DataFrame:
452
+ """
453
+ FIXED: Create DataFrame with proper field mapping
454
+ Checks both state field names AND database field names
455
+ """
456
+
457
+ df_data = []
458
+ for review in results:
459
+ # FIXED: Check state fields FIRST, fall back to database fields
460
+ df_data.append({
461
+ 'Review ID': review.get('review_id', 'N/A')[:20],
462
+ 'Rating': review.get('rating', 0),
463
+ 'Review': (review.get('review_text', 'N/A') or '')[:100] + '...',
464
+ 'Sentiment': review.get('final_sentiment', review.get('stage3_final_sentiment', 'N/A')),
465
+ 'Type': review.get('classification_type', review.get('stage1_llm1_type', 'N/A')),
466
+ 'Department': review.get('department', review.get('stage1_llm1_department', 'N/A')),
467
+ 'Priority': review.get('priority', review.get('stage1_llm1_priority', 'N/A')),
468
+ 'Emotion': review.get('emotion', review.get('stage1_llm2_emotion', 'N/A')),
469
+ 'Needs Review': '🚨 Yes' if review.get('needs_human_review', review.get('stage3_needs_human_review')) else 'βœ… No'
470
+ })
471
+
472
+ return pd.DataFrame(df_data)
473
 
 
 
 
 
 
 
 
 
 
 
 
474
 
 
475
 
476
+ # ============================================================================
477
+ # MAIN APP
478
+ # ============================================================================
479
 
 
480
 
481
+ def main():
482
+ """Main Streamlit app"""
483
+
484
+ # Title
485
+ st.title("🎯 Review Intelligence System")
486
+ st.markdown("### Multi-Stage AI Analysis Dashboard")
487
+ st.markdown("Powered by **LangGraph** + **HuggingFace** β€’ 4-Stage Processing Pipeline")
488
+ st.markdown("---")
489
+
490
+ # Sidebar - Input or View Mode
491
+ with st.sidebar:
492
+ st.header("πŸŽ›οΈ Control Panel")
493
+
494
+ if st.session_state.processing_complete:
495
+ st.success("βœ… Analysis Complete!")
496
+ if st.button("πŸ”„ Start New Analysis", use_container_width=True):
497
+ st.session_state.processing_complete = False
498
+ st.session_state.results = None
499
+ st.session_state.insights = None
500
+ st.rerun()
501
+ else:
502
+ st.info("πŸ‘ˆ Enter URLs below to start")
503
+
504
+ # Main content - Input or Results
505
+ if not st.session_state.processing_complete:
506
+ show_input_form()
507
+ else:
508
+ show_results_dashboard()
509
 
 
510
 
 
 
 
 
511
 
 
512
 
 
513
 
 
 
 
 
514
 
515
+ def show_input_form():
516
+ """Show input form for URLs and API key"""
517
+
518
+ st.markdown("### πŸ“ Step 1: Enter Store URLs")
519
+
520
+ col1, col2 = st.columns(2)
521
+
522
+ with col1:
523
+ st.markdown("#### 🍎 App Store IDs")
524
+ st.markdown(
525
+ """
526
+ **Format:** Just paste the app ID
527
+ - Example: `1158907446` (UAE)
528
+ - Example: `1234567890` (US)
529
+ """
530
+ )
531
+ app_store_urls = st.text_area(
532
+ "App Store IDs (one per line)",
533
+ placeholder="1158907446\n1234567890",
534
+ height=150,
535
+ key="app_urls"
536
+ )
537
+
538
+ with col2:
539
+ st.markdown("#### πŸ€– Play Store Packages")
540
+ st.markdown(
541
+ """
542
+ **Format:** Package name
543
+ - Example: `com.yas.app`
544
+ - Example: `com.company.app`
545
+ """
546
+ )
547
+ play_store_urls = st.text_area(
548
+ "Play Store Package Names (one per line)",
549
+ placeholder="com.yas.app\ncom.company.app",
550
+ height=150,
551
+ key="play_urls"
552
+ )
553
+
554
+ st.markdown("---")
555
+ st.markdown("### πŸ”‘ Step 2: Configure Settings")
556
+
557
+ col1, col2 = st.columns([2, 1])
558
+
559
+ with col1:
560
+ hf_api_key = st.text_input(
561
+ "πŸ”‘ HuggingFace API Key",
562
+ type="password",
563
+ placeholder="hf_...",
564
+ help="Get your key from: https://huggingface.co/settings/tokens",
565
+ key="hf_key"
566
+ )
567
+
568
+ with col2:
569
+ review_limit = st.slider(
570
+ "πŸ“Š Reviews per App",
571
+ min_value=5,
572
+ max_value=100,
573
+ value=20,
574
+ step=5,
575
+ help="More reviews = longer processing time",
576
+ key="review_limit"
577
+ )
578
+
579
+ st.markdown("---")
580
+
581
+ # Submit button
582
+ col1, col2, col3 = st.columns([1, 1, 1])
583
+
584
+ with col2:
585
+ if st.button("πŸš€ Start Analysis", use_container_width=True, type="primary"):
586
+ with st.spinner("Processing..."):
587
+ success = process_reviews_streamlit(
588
+ app_store_urls,
589
+ play_store_urls,
590
+ hf_api_key,
591
+ review_limit
592
+ )
593
+
594
+ if success:
595
+ st.balloons()
596
+ st.rerun()
597
+
598
+ # Documentation
599
+ with st.expander("πŸ“š How to Use"):
600
+ st.markdown("""
601
+ ### πŸ“– Quick Guide
602
+
603
+ **1. Get HuggingFace API Key:**
604
+ - Visit: https://huggingface.co/settings/tokens
605
+ - Create new token (Read access)
606
+ - Copy token (starts with `hf_`)
607
+
608
+ **2. Enter URLs:**
609
+ - **App Store**: Just the ID number (e.g., `1234567890`)
610
+ - **Play Store**: Package name (e.g., `com.company.app`)
611
+ - One per line
612
+
613
+ **3. Click Start:**
614
+ - Watch progress bar
615
+ - Wait for completion (~7 sec per review)
616
+ - View results automatically
617
+
618
+ ### πŸ—οΈ What Happens:
619
+ - πŸ•·οΈ **Stage 0**: Scrapes reviews from stores
620
+ - πŸ€– **Stage 1**: Classifies with 3 AI models (Type, Department, Priority)
621
+ - 😊 **Stage 2**: Analyzes sentiment with dual BERT models
622
+ - πŸ“Š **Stage 3**: Synthesizes insights and recommendations
623
+ - πŸ’‘ **Stage 4**: Generates batch analytics
624
+
625
+ ### ⚑ Performance:
626
+ - ~7 seconds per review
627
+ - 7 AI models working together
628
+ - Parallel execution for speed
629
+ """)
630
+
631
+
632
+
633
+ def show_results_dashboard():
634
+ """Show results dashboard with charts and tables"""
635
+
636
+ results = st.session_state.results
637
+ insights = st.session_state.insights
638
+ scraped_count = st.session_state.scraped_count
639
+
640
+ # Summary section
641
+ create_summary_section(scraped_count, results, insights)
642
+
643
+ st.markdown("---")
644
+
645
+ # Tabs for different views
646
+ tab1, tab2, tab3, tab4 = st.tabs([
647
+ "πŸ“Š Sentiment Analysis",
648
+ "🚨 Critical Issues",
649
+ "πŸ“‹ All Reviews",
650
+ "πŸ“₯ Export"
651
+ ])
652
+
653
+ # TAB 1: Sentiment Analysis
654
+ with tab1:
655
+ st.header("πŸ“Š Sentiment Analysis Overview")
656
+
657
+ col1, col2 = st.columns(2)
658
+
659
+ with col1:
660
+ fig_sentiment = create_sentiment_chart(insights)
661
+ st.plotly_chart(fig_sentiment, use_container_width=True)
662
+
663
+ with col2:
664
+ fig_priority = create_priority_chart(insights)
665
+ st.plotly_chart(fig_priority, use_container_width=True)
666
+
667
+ st.markdown("### 🏒 Department Routing")
668
+ fig_dept = create_department_chart(insights)
669
+ st.plotly_chart(fig_dept, use_container_width=True)
670
+
671
+ st.markdown("### 😊 Emotional Analysis")
672
+ fig_emotion = create_emotion_chart(insights)
673
+ st.plotly_chart(fig_emotion, use_container_width=True)
674
+
675
+ # TAB 2: Critical Issues
676
+ with tab2:
677
+ st.header("🚨 Critical Issues Requiring Attention")
678
+
679
+ # Filter critical reviews
680
+ critical_reviews = [
681
+ r for r in results
682
+ if (r.get('priority') == 'critical' or
683
+ r.get('stage1_llm1_priority') == 'critical' or
684
+ r.get('needs_human_review', r.get('stage3_needs_human_review')) or
685
+ (r.get('final_sentiment', r.get('stage3_final_sentiment')) == 'NEGATIVE' and r.get('rating', 5) <= 2))
686
+ ]
687
+
688
+ if len(critical_reviews) == 0:
689
+ st.success("βœ… No critical issues found! All reviews are in good shape.")
690
+ else:
691
+ st.warning(f"Found {len(critical_reviews)} critical issues")
692
+
693
+ for review in critical_reviews:
694
+ with st.expander(
695
+ f"⚠️ {review.get('review_id', 'Unknown')[:30]} - "
696
+ f"Rating: {review.get('rating', 'N/A')}/5"
697
+ ):
698
+ col1, col2 = st.columns([2, 1])
699
+
700
+ with col1:
701
+ st.markdown("**Review Text:**")
702
+ st.write(review.get('review_text', 'No text available'))
703
+
704
+ st.markdown("**Reasoning:**")
705
+ reasoning = review.get('reasoning', review.get('stage3_reasoning', 'No reasoning available'))
706
+ st.info(reasoning)
707
+
708
+ with col2:
709
+ st.markdown("**Classification:**")
710
+ st.write(f"πŸ“Œ Type: {review.get('classification_type', review.get('stage1_llm1_type', 'N/A'))}")
711
+ st.write(f"🏒 Department: {review.get('department', review.get('stage1_llm1_department', 'N/A'))}")
712
+ st.write(f"🎯 Priority: {review.get('priority', review.get('stage1_llm1_priority', 'N/A'))}")
713
+ st.write(f"πŸ˜” Emotion: {review.get('emotion', review.get('stage1_llm2_emotion', 'N/A'))}")
714
+ st.write(f"πŸ’­ Sentiment: {review.get('final_sentiment', review.get('stage3_final_sentiment', 'N/A'))}")
715
+
716
+ st.markdown("**Action:**")
717
+ action = review.get('action_recommendation', review.get('stage3_action_recommendation', 'No action specified'))
718
+ st.error(action)
719
+
720
+ # TAB 3: All Reviews
721
+ with tab3:
722
+ st.header("πŸ“‹ Detailed Review Analysis")
723
+
724
+ # Create DataFrame
725
+ df = create_reviews_dataframe(results)
726
+
727
+ # Filters
728
+ col1, col2, col3 = st.columns(3)
729
+
730
+ with col1:
731
+ sentiment_filter = st.multiselect(
732
+ "Filter by Sentiment",
733
+ options=df['Sentiment'].unique(),
734
+ default=df['Sentiment'].unique()
735
+ )
736
+
737
+ with col2:
738
+ dept_filter = st.multiselect(
739
+ "Filter by Department",
740
+ options=df['Department'].unique(),
741
+ default=df['Department'].unique()
742
+ )
743
+
744
+ with col3:
745
+ priority_filter = st.multiselect(
746
+ "Filter by Priority",
747
+ options=df['Priority'].unique(),
748
+ default=df['Priority'].unique()
749
+ )
750
+
751
+ # Apply filters
752
+ filtered_df = df[
753
+ (df['Sentiment'].isin(sentiment_filter)) &
754
+ (df['Department'].isin(dept_filter)) &
755
+ (df['Priority'].isin(priority_filter))
756
+ ]
757
+
758
+ st.info(f"Showing {len(filtered_df)} of {len(df)} reviews")
759
+
760
+ # Display table
761
+ st.dataframe(
762
+ filtered_df,
763
+ use_container_width=True,
764
+ height=600
765
+ )
766
+
767
+ # TAB 4: Export
768
+ with tab4:
769
+ st.header("πŸ“₯ Export Results")
770
+
771
+ st.markdown("### Download Options")
772
+
773
+ col1, col2 = st.columns(2)
774
+
775
+ with col1:
776
+ st.markdown("#### πŸ“Š CSV Export")
777
+ st.write("Download complete analysis with all classifications")
778
+
779
+ df = create_reviews_dataframe(results)
780
+ csv = df.to_csv(index=False)
781
+
782
+ st.download_button(
783
+ label="πŸ“₯ Download CSV Report",
784
+ data=csv,
785
+ file_name=f"review_analysis_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv",
786
+ mime="text/csv",
787
+ use_container_width=True
788
+ )
789
+
790
+ with col2:
791
+ st.markdown("#### πŸ“‹ JSON Export")
792
+ st.write("Download raw data with all details")
793
+
794
+ import json
795
+ json_data = json.dumps({
796
+ 'results': results,
797
+ 'insights': insights,
798
+ 'scraped_count': scraped_count,
799
+ 'export_date': datetime.now().isoformat()
800
+ }, indent=2)
801
+
802
+ st.download_button(
803
+ label="πŸ“₯ Download JSON Data",
804
+ data=json_data,
805
+ file_name=f"review_data_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json",
806
+ mime="application/json",
807
+ use_container_width=True
808
+ )
809
+
810
+ st.markdown("---")
811
+ st.markdown("### πŸ“Š Summary Statistics")
812
+
813
+ col1, col2, col3 = st.columns(3)
814
+
815
+ with col1:
816
+ st.metric("Total Reviews Analyzed", len(results))
817
+
818
+ with col2:
819
+ positive = insights.get('sentiment_distribution', {}).get('POSITIVE', 0)
820
+ total = len(results)
821
+ pct = (positive / total * 100) if total > 0 else 0
822
+ st.metric("Positive Rate", f"{pct:.1f}%")
823
+
824
+ with col3:
825
+ critical = insights.get('priority_distribution', {}).get('critical', 0)
826
+ st.metric("Critical Issues", critical)
827
+
828
+
829
+ # ============================================================================
830
+ # FOOTER
831
+ # ============================================================================
832
+
833
+ def show_footer():
834
+ """Show footer with credits"""
835
+ st.markdown("---")
836
+ st.markdown(
837
+ """
838
+ <div style='text-align: center'>
839
+ <p>πŸ€– Powered by Multi-Stage AI Pipeline |
840
+ Stage 1: Classification (Qwen, Mistral, Llama) |
841
+ Stage 2: Sentiment (Twitter-BERT) |
842
+ Stage 3: Finalization (Llama 70B) |
843
+ Stage 4: Batch Analysis</p>
844
+ <p>Built with ❀️ using LangGraph + HuggingFace + Streamlit</p>
845
+ </div>
846
+ """,
847
+ unsafe_allow_html=True
848
+ )
849
+
850
+
851
+ # ============================================================================
852
+ # RUN APP
853
+ # ============================================================================
854
+
855
+ if __name__ == "__main__":
856
+ main()
857
+ show_footer()