# Musora Sentiment Analysis Dashboard

A comprehensive, interactive Streamlit dashboard for visualizing sentiment analysis results from **multiple data sources**: social media comments (Facebook, Instagram, YouTube, Twitter) and Musora internal app comments across Musora brands (Drumeo, Pianote, Guitareo, Singeo).

## Features

### Main Dashboard
- **Overall sentiment distribution** with interactive pie charts and gauge indicators
- **Sentiment analysis by brand** (Drumeo, Pianote, Musora) with stacked bar charts
- **Sentiment analysis by platform** (Facebook, Instagram, etc.) with percentage distributions
- **Intent analysis** showing multi-label intent distributions (praise, question, request, etc.)
- **Cross-dimensional heatmaps** showing negative sentiment by brand and platform
- **Reply requirements analysis** with urgency breakdown
- **Language distribution** analysis
- **Temporal trends** with customizable time granularity (daily, weekly, monthly)
- **Hierarchical sunburst** visualization for brand > platform > sentiment

### Sentiment Analysis Page
- **Multi-sentiment filtering** - Filter by any combination of sentiments (positive, negative, neutral, etc.) to analyze both good and bad performance
- **Intent filtering** - Filter contents by specific user intents (question, praise, feedback_negative, etc.)
- **Dynamic severity scoring** - Ranks contents based on selected sentiments, adapts calculations to your filter choices
- **Advanced ranking controls** - Customize with minimum comment thresholds and multiple dynamic sort options
- **Sort options** - Severity Score (balanced), Sentiment %, Sentiment Count (absolute), or Total Comments (volume)
- **Engagement scatter plot** showing relationship between comment volume and sentiment
- **Thumbnail display** for Musora internal app content (visual content previews)
- **Detailed content analysis** with sentiment and intent distributions for each content
- **AI-Powered Analysis** - Optional AI-generated insights and recommendations for each content
- **View filtered comments** for each content with expandable sections
- **Actionable insights** and recommendations based on sentiment patterns
- **Export functionality** to download results as CSV with dynamic columns

### Reply Required Page
- **Prioritized comment queue** with urgency indicators (Urgent, High, Medium, Low)
- **Smart filtering** by priority, platform, brand, and intent
- **Pagination** for easy navigation through large comment lists
- **Comment cards** showing full context (author, timestamp, sentiment, intent)
- **Original and translated text** with expandable view for non-English comments
- **Reply requirements by content** showing which contents need most attention
- **Export functionality** for team collaboration or CRM import

## Architecture

```
visualization/
├── app.py                          # Main Streamlit application
├── config/
│   └── viz_config.json            # Configuration for colors, settings, queries
├── data/
│   └── data_loader.py             # Snowflake data loading with caching
├── utils/
│   ├── data_processor.py          # Data aggregation and processing
│   └── metrics.py                 # Metrics calculation (KPIs, scores)
├── components/
│   ├── dashboard.py               # Main dashboard page
│   ├── sentiment_analysis.py      # Comprehensive sentiment analysis page
│   └── reply_required.py          # Reply management page
├── visualizations/
│   ├── sentiment_charts.py        # Sentiment visualization functions
│   ├── distribution_charts.py     # Distribution visualization functions
│   └── content_cards.py           # Display components and cards
├── requirements.txt               # Python dependencies
└── README.md                      # This file
```

## Installation

### Prerequisites
- Python 3.8+
- Snowflake account with access to sentiment analysis data
- Required environment variables in parent `.env` file:
  - `SNOWFLAKE_USER`
  - `SNOWFLAKE_PASSWORD`
  - `SNOWFLAKE_ACCOUNT`
  - `SNOWFLAKE_ROLE`
  - `SNOWFLAKE_DATABASE`
  - `SNOWFLAKE_WAREHOUSE`
  - `SNOWFLAKE_SCHEMA`

### Setup

1. Navigate to the visualization directory:
```bash
cd visualization
```

2. Install dependencies:
```bash
pip install -r requirements.txt
```

3. Ensure parent `.env` file is properly configured with Snowflake credentials

## Usage

### Running the Dashboard

From the `visualization` directory:

```bash
streamlit run app.py
```

The dashboard will open in your default browser at `http://localhost:8501`

### Navigation

Use the sidebar to:
- **Select pages** (Dashboard, Sentiment Analysis, Reply Required)
- **Apply global filters** by platform, brand, sentiment, and date range
- **Reload data** to fetch latest updates from Snowflake
- **View data information** (record count, last update time)

### Filtering Data

1. Select desired filters in the sidebar:
   - **Platforms**: Filter by data source (Facebook, Instagram, YouTube, Twitter, musora_app)
   - **Brands**: Filter by Musora brand (Drumeo, Pianote, Guitareo, Singeo, Musora)
   - **Sentiments**: Filter by sentiment polarity
   - **Date Range**: Filter by comment timestamp

2. Click "Apply Filters" to update visualizations

3. Click "Reset Filters" to clear all filters

### Exporting Data

Each page provides export functionality:
- **Sentiment Analysis**: Download top N contents as CSV with dynamic columns based on active filters
- **Reply Required**: Download filtered comments as CSV

## Configuration

### Color Schemes

Edit `config/viz_config.json` to customize:
- **Sentiment colors**: Colors for each sentiment polarity
- **Intent colors**: Colors for each intent category
- **Platform colors**: Brand colors for each platform
- **Brand colors**: Colors for each Musora brand

### Dashboard Settings

Configure in `viz_config.json`:
- `default_date_range_days`: Default date range for filtering
- `max_comments_display`: Maximum comments to display per page
- `chart_height`: Default height for charts
- `top_n_contents`: Number of contents to show in poor sentiment page

### Data Query

The Snowflake query is configured in `viz_config.json`:
```json
"snowflake": {
    "query": "SELECT s.*, c.CHANNEL_NAME as BRAND, c.MESSAGE as CONTENT_DESCRIPTION, c.PERMALINK_URL FROM SOCIAL_MEDIA_DB.ML_FEATURES.COMMENT_SENTIMENT_FEATURES s JOIN SOCIAL_MEDIA_DB.CORE.DIM_CONTENT c ON s.CONTENT_SK = c.CONTENT_SK"
}
```

## Extending the Dashboard

### Adding New Pages

1. Create a new component file in `components/`:
```python
# components/new_page.py
def render_new_page(df):
    st.title("New Page")
    # Your page logic here
```

2. Import and add to navigation in `app.py`:
```python
from components.new_page import render_new_page

# Add to page selection
page = st.radio("Select Page", [..., "New Page"])

# Add to page rendering
elif page == "New Page":
    render_new_page(df)
```

### Adding New Visualizations

1. Add visualization function to appropriate module:
   - `visualizations/sentiment_charts.py` for sentiment-related charts
   - `visualizations/distribution_charts.py` for distribution charts

2. Use the function in page components

Example:
```python
def create_new_chart(df, title="New Chart"):
    fig = go.Figure(...)
    return fig
```

### Adding New Metrics

Add calculation methods to `utils/metrics.py`:
```python
@staticmethod
def calculate_new_metric(df):
    # Your metric calculation
    return metric_value
```

### Customizing Card Displays

Modify display methods in `visualizations/content_cards.py`:
```python
@staticmethod
def display_custom_card(data):
    # Your custom card layout
    pass
```

## Data Schema

The dashboard expects the following columns from Snowflake:

### Required Columns
- `comment_sk`: Unique comment identifier
- `comment_id`: Comment ID
- `original_text`: Original comment text
- `platform`: Social media platform
- `brand`: Musora brand name
- `sentiment_polarity`: Sentiment classification (very_positive, positive, neutral, negative, very_negative)
- `intent`: Comma-separated intent labels
- `requires_reply`: Boolean indicating if reply is needed
- `content_sk`: Content identifier
- `content_description`: Description of the content
- `permalink_url`: URL to the original content

### Optional Columns
- `comment_timestamp`: When comment was posted
- `processed_at`: When sentiment analysis was performed
- `translated_text`: English translation for non-English comments
- `detected_language`: Detected language of comment
- `is_english`: Boolean indicating if comment is in English
- `sentiment_confidence`: Confidence level of sentiment analysis
- `author_name`: Comment author name
- `channel_name`: Channel name
- `thumbnail_url`: Content thumbnail URL (for Musora internal app content)
- `parent_comment_id`: ID of parent comment (for replies)
- `parent_comment_text`: Text of parent comment (for reply context)

## Performance Optimization

### Caching
- Data loading is cached for 5 minutes using `@st.cache_data`
- Clear cache using "Reload Data" button in sidebar

### Pagination
- Comments requiring reply are paginated (10 per page)
- Reduces memory usage and improves rendering speed

### Filtering
- Apply filters to reduce dataset size before visualization
- Filters are applied efficiently using pandas operations

## Troubleshooting

### Connection Issues
- Verify Snowflake credentials in parent `.env` file
- Check network connectivity to Snowflake
- Ensure correct database, schema, and table names

### No Data Displayed
- Check if Snowflake query returns data
- Verify column names match expected schema
- Check applied filters - try resetting them

### Slow Performance
- Reduce date range in filters
- Use "Apply Filters" to work with smaller datasets
- Consider adding database indexes on frequently filtered columns

### Visualization Errors
- Check for missing or null values in data
- Verify data types match expected types (dates, booleans, etc.)
- Review browser console for JavaScript errors

## Best Practices

1. **Regular Data Updates**: Reload data periodically to see latest comments
2. **Use Filters**: Apply filters to focus on specific segments
3. **Export Insights**: Download CSV reports for offline analysis
4. **Monitor Reply Queue**: Check "Reply Required" page daily
5. **Track Trends**: Use temporal visualizations to identify patterns
6. **Prioritize Urgent**: Address urgent replies (negative sentiment) first

## Support

For issues or feature requests:
1. Check the troubleshooting section
2. Review configuration files for correct settings
3. Consult the main project README for sentiment analysis pipeline details

## Version History

### v1.3 (Current)
- **Comprehensive Sentiment Analysis Page Redesign**
  - Renamed "Poor Sentiment Contents" to "Sentiment Analysis" page
  - **NEW: Multi-Sentiment Filtering** - Filter by any combination of sentiments (positive, negative, neutral, very_positive, very_negative)
  - **NEW: Intent Filtering** - Filter contents by specific user intents (question, praise, feedback_negative, request, etc.)
  - **Filter Status Indicator** - Visual feedback showing when filters are active

- **Dynamic Ranking & Calculations**
  - **Dynamic severity scoring** - Automatically calculates based on selected sentiments (not just negative)
  - **Dynamic metrics** - Sentiment percentages and counts adapt to your filter selection
  - Sort options now work with any sentiment combination

- **Enhanced User Experience**
  - Summary statistics that dynamically adapt to filters
  - Contextual explanations that change based on selected sentiments
  - Export with dynamic columns based on active filters
  - Backward compatible - works like original when no filters selected

- **New Use Cases Enabled**
  - Analyze high-performing content (filter by positive sentiments)
  - Identify successful patterns (combine sentiment + intent filters)
  - Compare sentiment types side-by-side
  - Focus on specific user behaviors

### v1.2
- **Multi-source data support** - Integrated Musora internal app comments alongside social media
- **Smart severity scoring** - Content ranking now balances sentiment % with comment volume
- **Advanced ranking controls** - Min comments filter and multiple sort options (severity, %, count, volume)
- **Thumbnail display** - Visual content previews for Musora internal app content
- **Platform disambiguation** - Renamed internal platform to "musora_app" to differentiate from "musora" brand
- **Improved chart stability** - Fixed duplicate chart ID errors with unique keys
- **Enhanced data schema** - Added support for thumbnail_url and parent comment fields

### v1.1
- **AI-Powered Agents** - ContentSummaryAgent for intelligent comment analysis
- AI Analysis button on Sentiment Analysis page
- LLM Helper with OpenAI API integration
- Modular agent architecture ready for expansion

### v1.0
- Initial release
- Main dashboard with comprehensive visualizations
- Sentiment contents analysis page
- Reply required management page
- Global filtering and export functionality
- Plotly-based interactive visualizations
- Modular, extensible architecture