visualization/README.md

Musora Sentiment Analysis Dashboard

A comprehensive, interactive Streamlit dashboard for visualizing sentiment analysis results from multiple data sources: social media comments (Facebook, Instagram, YouTube, Twitter) and Musora internal app comments across Musora brands (Drumeo, Pianote, Guitareo, Singeo).

Features

Main Dashboard

• Overall sentiment distribution with interactive pie charts and gauge indicators
• Sentiment analysis by brand (Drumeo, Pianote, Musora) with stacked bar charts
• Sentiment analysis by platform (Facebook, Instagram, etc.) with percentage distributions
• Intent analysis showing multi-label intent distributions (praise, question, request, etc.)
• Cross-dimensional heatmaps showing negative sentiment by brand and platform
• Reply requirements analysis with urgency breakdown
• Language distribution analysis
• Temporal trends with customizable time granularity (daily, weekly, monthly)
• Hierarchical sunburst visualization for brand > platform > sentiment

Sentiment Analysis Page

• Multi-sentiment filtering - Filter by any combination of sentiments (positive, negative, neutral, etc.) to analyze both good and bad performance
• Intent filtering - Filter contents by specific user intents (question, praise, feedback_negative, etc.)
• Dynamic severity scoring - Ranks contents based on selected sentiments, adapts calculations to your filter choices
• Advanced ranking controls - Customize with minimum comment thresholds and multiple dynamic sort options
• Sort options - Severity Score (balanced), Sentiment %, Sentiment Count (absolute), or Total Comments (volume)
• Engagement scatter plot showing relationship between comment volume and sentiment
• Thumbnail display for Musora internal app content (visual content previews)
• Detailed content analysis with sentiment and intent distributions for each content
• AI-Powered Analysis - Optional AI-generated insights and recommendations for each content
• View filtered comments for each content with expandable sections
• Actionable insights and recommendations based on sentiment patterns
• Export functionality to download results as CSV with dynamic columns

Reply Required Page

• Prioritized comment queue with urgency indicators (Urgent, High, Medium, Low)
• Smart filtering by priority, platform, brand, and intent
• Pagination for easy navigation through large comment lists
• Comment cards showing full context (author, timestamp, sentiment, intent)
• Original and translated text with expandable view for non-English comments
• Reply requirements by content showing which contents need most attention
• Export functionality for team collaboration or CRM import

Architecture

visualization/
├── app.py                          # Main Streamlit application
├── config/
│   └── viz_config.json            # Configuration for colors, settings, queries
├── data/
│   └── data_loader.py             # Snowflake data loading with caching
├── utils/
│   ├── data_processor.py          # Data aggregation and processing
│   └── metrics.py                 # Metrics calculation (KPIs, scores)
├── components/
│   ├── dashboard.py               # Main dashboard page
│   ├── sentiment_analysis.py      # Comprehensive sentiment analysis page
│   └── reply_required.py          # Reply management page
├── visualizations/
│   ├── sentiment_charts.py        # Sentiment visualization functions
│   ├── distribution_charts.py     # Distribution visualization functions
│   └── content_cards.py           # Display components and cards
├── requirements.txt               # Python dependencies
└── README.md                      # This file

Installation

Prerequisites

• Python 3.8+
• Snowflake account with access to sentiment analysis data
• Required environment variables in parent .env file:
  • SNOWFLAKE_USER
  • SNOWFLAKE_PASSWORD
  • SNOWFLAKE_ACCOUNT
  • SNOWFLAKE_ROLE
  • SNOWFLAKE_DATABASE
  • SNOWFLAKE_WAREHOUSE
  • SNOWFLAKE_SCHEMA

Setup

1. Navigate to the visualization directory:

cd visualization

2. Install dependencies:

pip install -r requirements.txt

3. Ensure parent .env file is properly configured with Snowflake credentials

Usage

Running the Dashboard

From the visualization directory:

streamlit run app.py

The dashboard will open in your default browser at http://localhost:8501

Navigation

Use the sidebar to:

• Select pages (Dashboard, Sentiment Analysis, Reply Required)
• Apply global filters by platform, brand, sentiment, and date range
• Reload data to fetch latest updates from Snowflake
• View data information (record count, last update time)

Filtering Data

1. Select desired filters in the sidebar:

   • Platforms: Filter by data source (Facebook, Instagram, YouTube, Twitter, musora_app)
   • Brands: Filter by Musora brand (Drumeo, Pianote, Guitareo, Singeo, Musora)
   • Sentiments: Filter by sentiment polarity
   • Date Range: Filter by comment timestamp

2. Click "Apply Filters" to update visualizations

3. Click "Reset Filters" to clear all filters

Exporting Data

Each page provides export functionality:

• Sentiment Analysis: Download top N contents as CSV with dynamic columns based on active filters
• Reply Required: Download filtered comments as CSV

Configuration

Color Schemes

Edit config/viz_config.json to customize:

• Sentiment colors: Colors for each sentiment polarity
• Intent colors: Colors for each intent category
• Platform colors: Brand colors for each platform
• Brand colors: Colors for each Musora brand

Dashboard Settings

Configure in viz_config.json:

• default_date_range_days: Default date range for filtering
• max_comments_display: Maximum comments to display per page
• chart_height: Default height for charts
• top_n_contents: Number of contents to show in poor sentiment page

Data Query

The Snowflake query is configured in viz_config.json:

"snowflake": {
    "query": "SELECT s.*, c.CHANNEL_NAME as BRAND, c.MESSAGE as CONTENT_DESCRIPTION, c.PERMALINK_URL FROM SOCIAL_MEDIA_DB.ML_FEATURES.COMMENT_SENTIMENT_FEATURES s JOIN SOCIAL_MEDIA_DB.CORE.DIM_CONTENT c ON s.CONTENT_SK = c.CONTENT_SK"
}

Extending the Dashboard

Adding New Pages

1. Create a new component file in components/:

# components/new_page.py
def render_new_page(df):
    st.title("New Page")
    # Your page logic here

2. Import and add to navigation in app.py:

from components.new_page import render_new_page

# Add to page selection
page = st.radio("Select Page", [..., "New Page"])

# Add to page rendering
elif page == "New Page":
    render_new_page(df)

Adding New Visualizations

1. Add visualization function to appropriate module:

   • visualizations/sentiment_charts.py for sentiment-related charts
   • visualizations/distribution_charts.py for distribution charts

2. Use the function in page components

Example:

def create_new_chart(df, title="New Chart"):
    fig = go.Figure(...)
    return fig

Adding New Metrics

Add calculation methods to utils/metrics.py:

@staticmethod
def calculate_new_metric(df):
    # Your metric calculation
    return metric_value

Customizing Card Displays

Modify display methods in visualizations/content_cards.py:

@staticmethod
def display_custom_card(data):
    # Your custom card layout
    pass

Data Schema

The dashboard expects the following columns from Snowflake:

Required Columns

• comment_sk: Unique comment identifier
• comment_id: Comment ID
• original_text: Original comment text
• platform: Social media platform
• brand: Musora brand name
• sentiment_polarity: Sentiment classification (very_positive, positive, neutral, negative, very_negative)
• intent: Comma-separated intent labels
• requires_reply: Boolean indicating if reply is needed
• content_sk: Content identifier
• content_description: Description of the content
• permalink_url: URL to the original content

Optional Columns

• comment_timestamp: When comment was posted
• processed_at: When sentiment analysis was performed
• translated_text: English translation for non-English comments
• detected_language: Detected language of comment
• is_english: Boolean indicating if comment is in English
• sentiment_confidence: Confidence level of sentiment analysis
• author_name: Comment author name
• channel_name: Channel name
• thumbnail_url: Content thumbnail URL (for Musora internal app content)
• parent_comment_id: ID of parent comment (for replies)
• parent_comment_text: Text of parent comment (for reply context)

Performance Optimization

Caching

• Data loading is cached for 5 minutes using @st.cache_data
• Clear cache using "Reload Data" button in sidebar

Pagination

• Comments requiring reply are paginated (10 per page)
• Reduces memory usage and improves rendering speed

Filtering

• Apply filters to reduce dataset size before visualization
• Filters are applied efficiently using pandas operations

Troubleshooting

Connection Issues

• Verify Snowflake credentials in parent .env file
• Check network connectivity to Snowflake
• Ensure correct database, schema, and table names

No Data Displayed

• Check if Snowflake query returns data
• Verify column names match expected schema
• Check applied filters - try resetting them

Slow Performance

• Reduce date range in filters
• Use "Apply Filters" to work with smaller datasets
• Consider adding database indexes on frequently filtered columns

Visualization Errors

• Check for missing or null values in data
• Verify data types match expected types (dates, booleans, etc.)
• Review browser console for JavaScript errors

Best Practices

1. Regular Data Updates: Reload data periodically to see latest comments
2. Use Filters: Apply filters to focus on specific segments
3. Export Insights: Download CSV reports for offline analysis
4. Monitor Reply Queue: Check "Reply Required" page daily
5. Track Trends: Use temporal visualizations to identify patterns
6. Prioritize Urgent: Address urgent replies (negative sentiment) first

Support

For issues or feature requests:

1. Check the troubleshooting section
2. Review configuration files for correct settings
3. Consult the main project README for sentiment analysis pipeline details

Version History

v1.3 (Current)

• Comprehensive Sentiment Analysis Page Redesign

  • Renamed "Poor Sentiment Contents" to "Sentiment Analysis" page
  • NEW: Multi-Sentiment Filtering - Filter by any combination of sentiments (positive, negative, neutral, very_positive, very_negative)
  • NEW: Intent Filtering - Filter contents by specific user intents (question, praise, feedback_negative, request, etc.)
  • Filter Status Indicator - Visual feedback showing when filters are active

• Dynamic Ranking & Calculations

  • Dynamic severity scoring - Automatically calculates based on selected sentiments (not just negative)
  • Dynamic metrics - Sentiment percentages and counts adapt to your filter selection
  • Sort options now work with any sentiment combination

• Enhanced User Experience

  • Summary statistics that dynamically adapt to filters
  • Contextual explanations that change based on selected sentiments
  • Export with dynamic columns based on active filters
  • Backward compatible - works like original when no filters selected

• New Use Cases Enabled

  • Analyze high-performing content (filter by positive sentiments)
  • Identify successful patterns (combine sentiment + intent filters)
  • Compare sentiment types side-by-side
  • Focus on specific user behaviors

v1.2

• Multi-source data support - Integrated Musora internal app comments alongside social media
• Smart severity scoring - Content ranking now balances sentiment % with comment volume
• Advanced ranking controls - Min comments filter and multiple sort options (severity, %, count, volume)
• Thumbnail display - Visual content previews for Musora internal app content
• Platform disambiguation - Renamed internal platform to "musora_app" to differentiate from "musora" brand
• Improved chart stability - Fixed duplicate chart ID errors with unique keys
• Enhanced data schema - Added support for thumbnail_url and parent comment fields

v1.1

• AI-Powered Agents - ContentSummaryAgent for intelligent comment analysis
• AI Analysis button on Sentiment Analysis page
• LLM Helper with OpenAI API integration
• Modular agent architecture ready for expansion

v1.0

• Initial release
• Main dashboard with comprehensive visualizations
• Sentiment contents analysis page
• Reply required management page
• Global filtering and export functionality
• Plotly-based interactive visualizations
• Modular, extensible architecture