Upload 7 files

API_DOCUMENTATION.md

@@ -0,0 +1,533 @@
+# Audience Segmentation AI - REST API Documentation
+
+## Base Information
+
+**Base URL**: `http://localhost:7860`  
+**API Documentation**: `/api/docs` (Swagger UI)  
+**Content-Type**: `application/json`
+
+---
+
+##  Health & System
+
+### GET `/health`
+Check API and database connection status.
+
+**Response:**
+```json
+{
+  "status": "healthy",
+  "timestamp": "2025-11-24T00:00:00",
+  "database": "connected"
+}
+```
+
+---
+
+##  Event Analysis
+
+### POST `/api/events/{event_code}/analyze`
+Trigger full AI analysis pipeline for an event (Segmentation + Sentiment + GenAI).
+
+**Path Parameters:**
+- `event_code` (string, required): Event identifier
+
+**Response:**
+```json
+{
+  "status": "started",
+  "message": "Analysis pipeline started for event {event_code}",
+  "job_id": "analysis_abc123"
+}
+```
+
+### GET `/api/events/{event_code}/dashboard`
+Get comprehensive analytics dashboard for Event Owner.
+
+**Path Parameters:**
+- `event_code` (string, required): Event identifier
+
+**Response:**
+```json
+{
+  "event_code": "event_123",
+  "segments": [
+    {
+      "id": "507f1f77bcf86cd799439011",
+      "segment_name": "VIP Khách Hàng Trung Thành",
+      "user_count": 150,
+      "criteria": {
+        "avg_spend": 1500000,
+        "avg_tickets": 5.2,
+        "avg_recency": 15
+      },
+      "marketing_content": {
+        "email_subject": "Ưu đãi đặc biệt cho bạn!",
+        "email_body": "...",
+        "status": "Draft",
+        "generated_at": "2025-11-24T00:00:00"
+      }
+    }
+  ],
+  "sentiment_summary": {
+    "total_comments": 200,
+    "sentiment_distribution": {
+      "Positive": 150,
+      "Negative": 30,
+      "Neutral": 20
+    },
+    "avg_confidence": 0.87,
+    "top_keywords": ["tuyệt vời", "âm thanh", "tổ chức"],
+    "ai_insights": {
+      "summary": "Sự kiện được đánh giá tích cực...",
+      "top_issues": ["Check-in chậm", "Âm thanh yếu"],
+      "improvement_suggestions": ["Tăng quầy check-in", "Nâng cấp loa"],
+      "predicted_nps": 65.5
+    }
+  }
+}
+```
+
+---
+
+##  Audience Segmentation
+
+### POST `/api/events/{event_code}/segmentation/run`
+Run segmentation analysis for an event.
+
+**Path Parameters:**
+- `event_code` (string, required): Event identifier
+
+**Query Parameters:**
+- `n_clusters` (integer, optional): Number of segments (default: 5)
+
+**Response:**
+```json
+{
+  "status": "started",
+  "message": "Segmentation started",
+  "event_code": "event_123"
+}
+```
+
+### GET `/api/events/{event_code}/segments`
+Get all audience segments for an event.
+
+**Path Parameters:**
+- `event_code` (string, required): Event identifier
+
+**Query Parameters:**
+- `status` (string, optional): Filter by status (`Draft`, `Approved`, `Sent`)
+
+**Response:**
+```json
+[
+  {
+    "id": "507f1f77bcf86cd799439011",
+    "event_code": "event_123",
+    "segment_name": "VIP Khách Hàng Trung Thành",
+    "segment_type": "High Value",
+    "user_count": 150,
+    "user_ids": ["user_1", "user_2", "..."],
+    "criteria": {
+      "avg_spend": 1500000,
+      "avg_tickets": 5.2,
+      "avg_recency": 15,
+      "avg_follow_count": 3
+    },
+    "marketing_content": {
+      "email_subject": "Ưu đãi VIP đặc biệt",
+      "email_body": "Kính gửi Quý khách...",
+      "status": "Draft",
+      "generated_at": "2025-11-24T00:00:00"
+    },
+    "created_at": "2025-11-24T00:00:00",
+    "last_updated": "2025-11-24T00:00:00"
+  }
+]
+```
+
+### GET `/api/events/{event_code}/segments/{segment_id}`
+Get specific segment details.
+
+**Path Parameters:**
+- `event_code` (string, required)
+- `segment_id` (string, required): Segment ObjectId
+
+**Response:**
+```json
+{
+  "id": "507f1f77bcf86cd799439011",
+  "event_code": "event_123",
+  "segment_name": "VIP Khách Hàng Trung Thành",
+  "user_count": 150,
+  "user_ids": ["user_1", "user_2"],
+  "criteria": {...},
+  "marketing_content": {...}
+}
+```
+
+### GET `/api/events/{event_code}/segments/{segment_id}/users`
+Get user list in a segment.
+
+**Path Parameters:**
+- `event_code` (string, required)
+- `segment_id` (string, required)
+
+**Query Parameters:**
+- `skip` (integer): Offset (default: 0)
+- `limit` (integer): Max results (default: 100)
+
+**Response:**
+```json
+{
+  "segment_id": "507f1f77bcf86cd799439011",
+  "total_users": 150,
+  "users": [
+    {
+      "user_id": "user_1",
+      "email": "user@example.com",
+      "full_name": "Nguyễn Văn A",
+      "stats": {
+        "total_spend": 2000000,
+        "tickets_bought": 6,
+        "last_purchase": "2025-11-20"
+      }
+    }
+  ]
+}
+```
+
+---
+
+##  Approval Workflow
+
+### POST `/api/events/{event_code}/segments/{segment_id}/approve`
+Event Owner approves marketing content.
+
+**Path Parameters:**
+- `event_code` (string, required)
+- `segment_id` (string, required)
+
+**Request Body (optional):**
+```json
+{
+  "approved_by": "owner_user_id",
+  "modified_content": {
+    "email_subject": "Modified subject",
+    "email_body": "Modified body"
+  }
+}
+```
+
+**Response:**
+```json
+{
+  "status": "success",
+  "message": "Segment approved",
+  "segment_id": "507f1f77bcf86cd799439011",
+  "marketing_content": {
+    "status": "Approved",
+    "approved_at": "2025-11-24T00:00:00",
+    "approved_by": "owner_user_id"
+  }
+}
+```
+
+### POST `/api/events/{event_code}/segments/{segment_id}/send-email`
+Send approved marketing email to segment users.
+
+**Path Parameters:**
+- `event_code` (string, required)
+- `segment_id` (string, required)
+
+**Request Body:**
+```json
+{
+  "send_immediately": true,
+  "schedule_at": "2025-11-25T10:00:00"
+}
+```
+
+**Response:**
+```json
+{
+  "status": "success",
+  "message": "Email sent to 150 users",
+  "segment_id": "507f1f77bcf86cd799439011",
+  "emails_sent": 150,
+  "emails_failed": 0,
+  "marketing_content": {
+    "status": "Sent"
+  }
+}
+```
+
+---
+
+##  Sentiment Analysis
+
+### POST `/api/events/{event_code}/sentiment/analyze`
+Analyze sentiment for event comments.
+
+**Path Parameters:**
+- `event_code` (string, required)
+
+**Response:**
+```json
+{
+  "status": "started",
+  "message": "Sentiment analysis started for event {event_code}"
+}
+```
+
+### GET `/api/events/{event_code}/sentiment/summary`
+Get sentiment summary for an event.
+
+**Path Parameters:**
+- `event_code` (string, required)
+
+**Response:**
+```json
+{
+  "event_code": "event_123",
+  "total_comments": 200,
+  "sentiment_distribution": {
+    "Positive": 150,
+    "Negative": 30,
+    "Neutral": 20
+  },
+  "avg_confidence": 0.87,
+  "top_keywords": ["tuyệt vời", "âm thanh", "tổ chức"],
+  "ai_insights": {
+    "summary": "Sự kiện được đánh giá tích cực với 75% feedback tích cực...",
+    "top_issues": [
+      "Check-in quá chậm (15 mentions)",
+      "Âm thanh yếu ở khu vực sau (10 mentions)"
+    ],
+    "improvement_suggestions": [
+      "Tăng số quầy check-in lên 5 quầy",
+      "Bổ sung loa phụ khu vực sau"
+    ],
+    "predicted_nps": 65.5
+  },
+  "last_updated": "2025-11-24T00:00:00"
+}
+```
+
+### GET `/api/events/{event_code}/sentiment/results`
+Get detailed sentiment results.
+
+**Path Parameters:**
+- `event_code` (string, required)
+
+**Query Parameters:**
+- `sentiment_label` (string, optional): Filter by `Positive`, `Negative`, `Neutral`
+- `skip` (integer): Offset
+- `limit` (integer): Max results
+
+**Response:**
+```json
+{
+  "total": 200,
+  "results": [
+    {
+      "id": "507f...",
+      "event_code": "event_123",
+      "source_id": "comment_abc",
+      "sentiment_label": "Positive",
+      "confidence_score": 0.92,
+      "key_phrases": ["tuyệt vời", "hài lòng"],
+      "analyzed_at": "2025-11-24T00:00:00"
+    }
+  ]
+}
+```
+
+---
+
+##  Generative AI
+
+### POST `/api/events/{event_code}/genai/generate-emails`
+Generate marketing emails for all segments.
+
+**Path Parameters:**
+- `event_code` (string, required)
+
+**Response:**
+```json
+{
+  "status": "started",
+  "message": "Email generation started for {n} segments"
+}
+```
+
+### POST `/api/events/{event_code}/genai/generate-insights`
+Generate AI insights from negative feedback.
+
+**Path Parameters:**
+- `event_code` (string, required)
+
+**Response:**
+```json
+{
+  "status": "success",
+  "insights": {
+    "summary": "...",
+    "top_issues": ["..."],
+    "improvement_suggestions": ["..."],
+    "predicted_nps": 62.5
+  }
+}
+```
+
+---
+
+##  Monitoring & Analytics
+
+### GET `/api/monitoring/pipelines/{pipeline}/metrics`
+Get performance metrics for a pipeline.
+
+**Path Parameters:**
+- `pipeline` (string): `segmentation`, `sentiment`, `genai`
+
+**Query Parameters:**
+- `event_code` (string, optional): Filter by event
+- `days` (integer): Date range (default: 7)
+
+**Response:**
+```json
+{
+  "pipeline": "segmentation",
+  "event_code": "event_123",
+  "total_runs": 5,
+  "avg_execution_time": 4.2,
+  "last_run": "2025-11-24T00:00:00",
+  "metrics": {
+    "avg_users_processed": 850,
+    "avg_segments_created": 5,
+    "avg_inertia": 1250.5
+  }
+}
+```
+
+### GET `/api/monitoring/pipelines/{pipeline}/drift`
+Check for model drift.
+
+**Path Parameters:**
+- `pipeline` (string): `segmentation`, `sentiment`
+
+**Query Parameters:**
+- `event_code` (string, optional)
+
+**Response:**
+```json
+{
+  "pipeline": "segmentation",
+  "drift_detected": true,
+  "avg_drift": 0.65,
+  "max_drift": 1.2,
+  "threshold": 0.5,
+  "recommendation": "Consider retraining model"
+}
+```
+
+---
+
+##  Feedback & Performance
+
+### POST `/api/feedback/email-engagement`
+Record email engagement metrics.
+
+**Request Body:**
+```json
+{
+  "segment_id": "507f...",
+  "user_id": "user_1",
+  "event_code": "event_123",
+  "opened": true,
+  "clicked": true,
+  "converted": false,
+  "unsubscribed": false
+}
+```
+
+**Response:**
+```json
+{
+  "status": "recorded",
+  "feedback_id": "feedback_xyz"
+}
+```
+
+### GET `/api/feedback/email-performance/{segment_id}`
+Get email campaign performance.
+
+**Path Parameters:**
+- `segment_id` (string, required)
+
+**Response:**
+```json
+{
+  "segment_id": "507f...",
+  "total_sent": 150,
+  "open_rate": 0.65,
+  "click_rate": 0.32,
+  "conversion_rate": 0.12,
+  "unsubscribe_rate": 0.02
+}
+```
+
+---
+
+##  Administration
+
+### POST `/api/admin/indexes/create`
+Create all MongoDB indexes (run once during setup).
+
+**Response:**
+```json
+{
+  "status": "success",
+  "indexes_created": [
+    "idx_payment_event_status_user",
+    "idx_follow_event_user",
+    "idx_comment_event_date",
+    "..."
+  ]
+}
+```
+
+### POST `/api/admin/models/retrain`
+Trigger model retraining based on feedback.
+
+**Request Body:**
+```json
+{
+  "model_type": "segmentation",
+  "event_code": "event_123"
+}
+```
+
+**Response:**
+```json
+{
+  "status": "started",
+  "job_id": "retrain_abc123"
+}
+```
+
+---
+
+## Error Responses
+
+All endpoints may return error responses in the following format:
+
+```json
+{
+  "detail": "Error message description",
+  "status_code": 400
+}
+```
+
+

README.md

@@ -1,12 +1,133 @@
----
-title: Aus F
-emoji: 👁
-colorFrom: indigo
-colorTo: pink
-sdk: gradio
-sdk_version: 6.0.0
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Audience Segmentation AI System
+
+Hệ thống phân khúc khách hàng và phân tích cảm xúc sử dụng AI cho nền tảng quản lý sự kiện.
+
+## Tính năng
+
+### 1. Phân khúc khách hàng (Audience Segmentation)
+- **Phân cụm tự động** dựa trên hành vi mua vé (RFM Analysis)
+- **Phân loại theo sở thích** về danh mục sự kiện
+- **Đặt tên tự động** cho từng phân khúc bằng tiếng Việt
+- **Tạo nội dung email marketing** tự động cho từng nhóm khách hàng
+
+### 2. Phân tích cảm xúc (Sentiment Analysis)
+- **Phân loại cảm xúc** của bình luận (Tích cực/Tiêu cực/Trung tính)
+- **Sử dụng PhoBERT** - mô hình NLP chuyên biệt cho tiếng Việt
+- **Trích xuất từ khóa** tự động từ feedback
+
+### 3. Tạo Insight tự động (Generative AI)
+- **Top 5 vấn đề** cần cải thiện
+- **Gợi ý cải thiện** cho từng vấn đề
+- **Dự đoán NPS Score** dựa trên tone của comments
+- **Sử dụng Vistral-7B-Chat** - LLM tiên tiến cho tiếng Việt
+
+## Cấu trúc thư mục
+
+```
+AudienceSegmentation/
+├── models/                      # MongoDB data models
+│   ├── segmentation_models.py   # Audience segment models
+│   └── sentiment_models.py      # Sentiment analysis models
+├── services/                    # Business logic
+│   ├── data_aggregation.py      # MongoDB aggregation pipelines
+│   ├── segmentation_service.py  # K-Means clustering
+│   ├── sentiment_service.py     # PhoBERT sentiment analysis
+│   └── genai_service.py         # Vistral-7B content generation
+├── config.py                    # Configuration
+├── database.py                  # MongoDB connection manager
+├── main.py                      # Main orchestration script
+├── requirements.txt             # Python dependencies
+└── .env.example                 # Environment variables template
+```
+
+## Cài đặt
+
+### 1. Clone repository
+```bash
+cd AudienceSegmentation
+```
+
+### 2. Tạo môi trường
+```bash
+python -m venv venv
+source venv/bin/activate  # Linux/Mac
+# hoặc
+venv\Scripts\activate     # Windows
+```
+
+### 3. Cài đặt dependencies
+```bash
+pip install -r requirements.txt
+```
+
+### 4. Download Vistral-7B-Chat
+```bash
+# Tải mô hình GGUF từ Hugging Face (CPU nên tải)
+mkdir -p models/vistral-7b-chat
+# Download từ: https://huggingface.co/Vistral/Vistral-7B-Chat-GGUF
+```
+
+### 5. Cấu hình môi trường
+```bash
+cp .env.example .env
+# Chỉnh sửa .env với thông tin MongoDB của bạn
+```
+
+## Sử dụng
+
+### Chạy toàn bộ pipeline
+```bash
+python main.py --task all
+```
+
+### Chỉ chạy phân khúc khách hàng
+```bash
+python main.py --task segmentation
+```
+
+### Chỉ chạy phân tích cảm xúc
+```bash
+python main.py --task sentiment
+```
+
+### Chỉ tạo nội dung email
+```bash
+python main.py --task email
+```
+
+### Tạo insights cho sự kiện cụ thể
+```bash
+python main.py --task insights --event-code <event_id>
+```
+
+## Kiến trúc kỹ thuật
+
+### MongoDB Aggregation Framework
+Hệ thống tận dụng MongoDB Aggregation để:
+- **Tính toán RFM** (Recency, Frequency, Monetary) trực tiếp trên database
+- **Đếm danh mục sự kiện** mà user quan tâm
+- **Lọc dữ liệu chưa xử lý** để tránh duplicate
+- **Giảm thiểu network transfer** - chỉ truyền kết quả cuối cùng
+
+### AI Models
+
+#### 1. Segmentation: scikit-learn K-Means
+- **Input**: Feature vector [R, F, M, Category1, Category2, ...]
+- **Output**: Cluster labels + Confidence scores
+- **Số cụm**: 5 (có thể cấu hình)
+
+#### 2. Sentiment: wonrax/phobert-base-vietnamese-sentiment
+- **Model**: PhoBERT fine-tuned cho Vietnamese
+- **Output**: Positive/Negative/Neutral + Confidence
+- **Batch size**: 32
+
+
+## Collections MongoDB
+
+
+### Output Collections (New)
+- `AudienceSegment` - Các phân khúc khách hàng
+- `UserSegmentAssignment` - Gán user vào segment
+- `SentimentAnalysisResult` - Kết quả phân tích cảm xúc
+- `EventInsightReport` - Báo cáo insight cho sự kiện
+

app.py

@@ -0,0 +1,447 @@
+"""
+FastAPI Application for Event-Centric Audience Segmentation AI
+Author: AI Generated
+Created: 2025-11-24 (Refactored)
+Purpose: REST API with event-based endpoints
+"""
+
+from fastapi import FastAPI, HTTPException, BackgroundTasks, status, Query
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel
+from typing import List, Dict, Optional, Any
+from datetime import datetime
+from bson import ObjectId
+
+# Import services
+from services.segmentation_service import SegmentationService
+from services.sentiment_service import SentimentAnalysisService
+from services.genai_service import GenerativeAIService
+from database import db
+from config import settings
+
+
+# FastAPI app
+app = FastAPI(
+    title="Audience Segmentation AI - Event-Centric",
+    description="REST API for per-event audience analysis",
+    version="2.0.0",
+    docs_url="/api/docs",
+    redoc_url="/api/redoc"
+)
+
+# CORS
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+
+# Helper
+def serialize_doc(doc: Dict) -> Optional[Dict]:
+    """Convert MongoDB document to JSON-serializable dict"""
+    if doc is None:
+        return None
+    if '_id' in doc:
+        doc['id'] = str(doc.pop('_id'))
+    
+    # Handle nested ObjectIds and lists
+    for key, value in list(doc.items()):
+        if isinstance(value, ObjectId):
+            doc[key] = str(value)
+        elif isinstance(value, list):
+            doc[key] = [str(v) if isinstance(v, ObjectId) else v for v in value]
+        elif isinstance(value, dict):
+            doc[key] = serialize_doc(value)
+    
+    return doc
+
+
+# ===== HEALTH =====
+@app.get("/health", tags=["System"])
+async def health_check():
+    """Health check"""
+    try:
+        db.client.server_info()
+        return {
+            "status": "healthy",
+            "timestamp": datetime.utcnow(),
+            "database": "connected"
+        }
+    except Exception as e:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=f"Unhealthy: {str(e)}"
+        )
+
+
+# ===== EVENT ANALYSIS =====
+@app.post("/api/events/{event_code}/analyze", tags=["Event Analysis"])
+async def analyze_event(event_code: str, background_tasks: BackgroundTasks):
+    """Run full AI pipeline for an event"""
+    
+    def run_pipeline():
+        # Step 1: Segmentation
+        seg_service = SegmentationService(event_code)
+        seg_service.run_segmentation()
+        
+        # Step 2: Sentiment
+        sent_service = SentimentAnalysisService(event_code)
+        sent_service.analyze_event_comments()
+        
+        # Step 3: Email generation
+        genai_service = GenerativeAIService(event_code)
+        genai_service.generate_emails_for_all_segments()
+        
+        # Step 4: Insights
+        genai_service.update_sentiment_summary_with_insights()
+    
+    background_tasks.add_task(run_pipeline)
+    
+    return {
+        "status": "started",
+        "message": f"Analysis pipeline started for event {event_code}"
+    }
+
+
+@app.get("/api/events/{event_code}/dashboard", tags=["Event Analysis"])
+async def get_event_dashboard(event_code: str):
+    """Get complete dashboard for Event Owner"""
+    
+    # Get segments
+    segments = list(db.event_audience_segments.find({"event_code": event_code}))
+    
+    # Get sentiment summary
+    sentiment_summary = db.event_sentiment_summary.find_one({"event_code": event_code})
+    
+    return {
+        "event_code": event_code,
+        "segments": [serialize_doc(s) for s in segments],
+        "sentiment_summary": serialize_doc(sentiment_summary) if sentiment_summary else None
+    }
+
+
+# ===== SEGMENTATION =====
+@app.post("/api/events/{event_code}/segmentation/run", tags=["Segmentation"])
+async def run_event_segmentation(
+    event_code: str,
+    background_tasks: BackgroundTasks,
+    n_clusters: int = Query(default=5, ge=2, le=10)
+):
+    """Run segmentation for an event"""
+    
+    def run_task():
+        service = SegmentationService(event_code, n_clusters=n_clusters)
+        service.run_segmentation()
+    
+    background_tasks.add_task(run_task)
+    
+    return {
+        "status": "started",
+        "message": f"Segmentation started for event {event_code}",
+        "event_code": event_code
+    }
+
+
+@app.get("/api/events/{event_code}/segments", tags=["Segmentation"])
+async def get_event_segments(
+    event_code: str,
+    status_filter: Optional[str] = Query(default=None, description="Filter by Draft, Approved, Sent")
+):
+    """Get all segments for an event"""
+    
+    query = {"event_code": event_code}
+    if status_filter:
+        query["marketing_content.status"] = status_filter
+    
+    segments = list(db.event_audience_segments.find(query))
+    
+    return [serialize_doc(s) for s in segments]
+
+
+@app.get("/api/events/{event_code}/segments/{segment_id}", tags=["Segmentation"])
+async def get_segment_detail(event_code: str, segment_id: str):
+    """Get specific segment details"""
+    
+    segment = db.event_audience_segments.find_one({
+        "_id": ObjectId(segment_id),
+        "event_code": event_code
+    })
+    
+    if not segment:
+        raise HTTPException(status_code=404, detail="Segment not found")
+    
+    return serialize_doc(segment)
+
+
+@app.get("/api/events/{event_code}/segments/{segment_id}/users", tags=["Segmentation"])
+async def get_segment_users(
+    event_code: str,
+    segment_id: str,
+    skip: int = 0,
+    limit: int = 100
+):
+    """Get users in a segment with details"""
+    
+    segment = db.event_audience_segments.find_one({
+        "_id": ObjectId(segment_id),
+        "event_code": event_code
+    })
+    
+    if not segment:
+        raise HTTPException(status_code=404, detail="Segment not found")
+    
+    user_ids = segment.get('user_ids', [])
+    total_users = len(user_ids)
+    
+    # Paginate
+    paginated_ids = user_ids[skip:skip + limit]
+    
+    # Get user details
+    users = list(db.users.find({
+        "_id": {"$in": paginated_ids}
+    }))
+    
+    # Enrich with stats (optional)
+    enriched_users = []
+    for user in users:
+        enriched_users.append({
+            "user_id": str(user['_id']),
+            "email": user.get('email'),
+            "full_name": f"{user.get('FirstName', '')} {user.get('LastName', '')}".strip()
+        })
+    
+    return {
+        "segment_id": segment_id,
+        "total_users": total_users,
+        "users": enriched_users
+    }
+
+
+# ===== APPROVAL WORKFLOW =====
+@app.post("/api/events/{event_code}/segments/{segment_id}/approve", tags=["Approval"])
+async def approve_segment(
+    event_code: str,
+    segment_id: str,
+    approved_by: Optional[str] = None,
+    modified_subject: Optional[str] = None,
+    modified_body: Optional[str] = None
+):
+    """Event Owner approves marketing content"""
+    
+    segment = db.event_audience_segments.find_one({
+        "_id": ObjectId(segment_id),
+        "event_code": event_code
+    })
+    
+    if not segment:
+        raise HTTPException(status_code=404, detail="Segment not found")
+    
+    # Update fields
+    update = {
+        "marketing_content.status": "Approved",
+        "marketing_content.approved_at": datetime.utcnow(),
+        "marketing_content.approved_by": approved_by,
+        "last_updated": datetime.utcnow()
+    }
+    
+    if modified_subject:
+        update["marketing_content.email_subject"] = modified_subject
+    if modified_body:
+        update["marketing_content.email_body"] = modified_body
+    
+    db.event_audience_segments.update_one(
+        {"_id": ObjectId(segment_id)},
+        {"$set": update}
+    )
+    
+    updated_segment = db.event_audience_segments.find_one({"_id": ObjectId(segment_id)})
+    
+    return {
+        "status": "success",
+        "message": "Segment approved",
+        "segment_id": segment_id,
+        "marketing_content": updated_segment.get('marketing_content')
+    }
+
+
+@app.post("/api/events/{event_code}/segments/{segment_id}/send-email", tags=["Approval"])
+async def send_segment_email(
+    event_code: str,
+    segment_id: str,
+    send_immediately: bool = True
+):
+    """Send approved marketing email"""
+    
+    segment = db.event_audience_segments.find_one({
+        "_id": ObjectId(segment_id),
+        "event_code": event_code
+    })
+    
+    if not segment:
+        raise HTTPException(status_code=404, detail="Segment not found")
+    
+    marketing_content = segment.get('marketing_content', {})
+    if marketing_content.get('status') != "Approved":
+        raise HTTPException(status_code=400, detail="Segment not approved yet")
+    
+    # TODO: Integrate with email service (SendGrid, AWS SES, etc.)
+    # For now, just mark as sent
+    
+    db.event_audience_segments.update_one(
+        {"_id": ObjectId(segment_id)},
+        {"$set": {
+            "marketing_content.status": "Sent",
+            "last_updated": datetime.utcnow()
+        }}
+    )
+    
+    return {
+        "status": "success",
+        "message": f"Email sent to {segment.get('user_count', 0)} users",
+        "segment_id": segment_id,
+        "emails_sent": segment.get('user_count', 0),
+        "emails_failed": 0
+    }
+
+
+# ===== SENTIMENT =====
+@app.post("/api/events/{event_code}/sentiment/analyze", tags=["Sentiment"])
+async def analyze_event_sentiment(event_code: str, background_tasks: BackgroundTasks):
+    """Analyze sentiment for event comments"""
+    
+    def run_task():
+        service = SentimentAnalysisService(event_code)
+        service.analyze_event_comments()
+    
+    background_tasks.add_task(run_task)
+    
+    return {
+        "status": "started",
+        "message": f"Sentiment analysis started for event {event_code}"
+    }
+
+
+@app.get("/api/events/{event_code}/sentiment/summary", tags=["Sentiment"])
+async def get_sentiment_summary(event_code: str):
+    """Get sentiment summary for an event"""
+    
+    summary = db.event_sentiment_summary.find_one({"event_code": event_code})
+    
+    if not summary:
+        raise HTTPException(status_code=404, detail="No sentiment data for this event")
+    
+    return serialize_doc(summary)
+
+
+@app.get("/api/events/{event_code}/sentiment/results", tags=["Sentiment"])
+async def get_sentiment_results(
+    event_code: str,
+    sentiment_label: Optional[str] = None,
+    skip: int = 0,
+    limit: int = 100
+):
+    """Get detailed sentiment results"""
+    
+    query = {"event_code": event_code}
+    if sentiment_label:
+        query["sentiment_label"] = sentiment_label
+    
+    total = db.sentiment_results.count_documents(query)
+    results = list(
+        db.sentiment_results.find(query)
+        .sort("analyzed_at", -1)
+        .skip(skip)
+        .limit(limit)
+    )
+    
+    return {
+        "total": total,
+        "results": [serialize_doc(r) for r in results]
+    }
+
+
+# ===== GENAI =====
+@app.post("/api/events/{event_code}/genai/generate-emails", tags=["GenAI"])
+async def generate_event_emails(event_code: str, background_tasks: BackgroundTasks):
+    """Generate marketing emails for all segments"""
+    
+    def run_task():
+        service = GenerativeAIService(event_code)
+        service.generate_emails_for_all_segments()
+    
+    background_tasks.add_task(run_task)
+    
+    return {
+        "status": "started",
+        "message": "Email generation started"
+    }
+
+
+@app.post("/api/events/{event_code}/genai/generate-insights", tags=["GenAI"])
+async def generate_event_insights(event_code: str, background_tasks: BackgroundTasks):
+    """Generate AI insights from negative feedback"""
+    
+    def run_task():
+        service = GenerativeAIService(event_code)
+        service.update_sentiment_summary_with_insights()
+    
+    background_tasks.add_task(run_task)
+    
+    return {
+        "status": "started",
+        "message": "Insight generation started"
+    }
+
+
+# ===== MONITORING =====
+@app.get("/api/monitoring/pipelines/{pipeline}/metrics", tags=["Monitoring"])
+async def get_pipeline_metrics(
+    pipeline: str,
+    event_code: Optional[str] = None,
+    days: int = 7
+):
+    """Get performance metrics"""
+    # TODO: Implement based on monitoring.py
+    return {
+        "pipeline": pipeline,
+        "event_code": event_code,
+        "message": "Metrics endpoint - implement as needed"
+    }
+
+
+# ===== ADMIN =====
+@app.post("/api/admin/indexes/create", tags=["Admin"])
+async def create_indexes():
+    """Create MongoDB indexes"""
+    from scripts.create_indexes import create_all_indexes
+    
+    try:
+        create_all_indexes()
+        return {"status": "success", "message": "Indexes created"}
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+# ===== ROOT =====
+@app.get("/")
+async def root():
+    """API root"""
+    return {
+        "name": "Audience Segmentation AI - Event-Centric",
+        "version": "2.0.0",
+        "docs": "/api/docs",
+        "health": "/health"
+    }
+
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(
+        "app:app",
+        host="0.0.0.0",
+        port=7860,
+        reload=True
+    )

config.py

@@ -0,0 +1,50 @@
+"""
+Configuration for Audience Segmentation System
+"""
+
+import os
+from pydantic_settings import BaseSettings
+
+
+class Settings(BaseSettings):
+    """Application settings"""
+    
+    # MongoDB Configuration
+    MONGODB_URI: str = os.getenv("MONGODB_URI", "mongodb://localhost:27017")
+    DB_NAME: str = os.getenv("DB_NAME", "audience_segmentation")
+    
+    # Hugging Face Token (optional, not required for our models)
+    HF_TOKEN: str = os.getenv("HF_TOKEN", "")
+    
+    # Collection Names
+    COLLECTION_USERS: str = "User"
+    COLLECTION_PAYMENTS: str = "Payment"
+    COLLECTION_EVENT_VERSIONS: str = "EventVersion"
+    COLLECTION_USER_FOLLOWS: str = "UserFollow"
+    COLLECTION_USER_COMMENT_POST: str = "UserCommentPost"
+    COLLECTION_POST_SOCIAL_MEDIA: str = "PostSocialMedia"
+    
+    # AI Result Collections
+    COLLECTION_AUDIENCE_SEGMENTS: str = "AudienceSegment"
+    COLLECTION_USER_SEGMENT_ASSIGNMENTS: str = "UserSegmentAssignment"
+    COLLECTION_SENTIMENT_RESULTS: str = "SentimentAnalysisResult"
+    COLLECTION_EVENT_INSIGHTS: str = "EventInsightReport"
+    
+    # AI Model Configuration
+    SENTIMENT_MODEL: str = "wonrax/phobert-base-vietnamese-sentiment"
+    LLM_MODEL: str = "Vistral-7B-Chat"
+    LLM_LOCAL_PATH: str = os.getenv("LLM_LOCAL_PATH", "./models/vistral-7b-chat")
+    
+    # Clustering Configuration
+    N_CLUSTERS: int = 5  # Number of audience segments
+    RANDOM_STATE: int = 42
+    
+    # Batch Processing
+    BATCH_SIZE: int = 32
+    
+    class Config:
+        env_file = ".env"
+        case_sensitive = True
+
+
+settings = Settings()

database.py

@@ -0,0 +1,111 @@
+"""
+MongoDB Database Connection Manager
+Author: AI Generated
+Created: 2025-11-24
+Purpose: Handle MongoDB connection and collection access
+"""
+
+from pymongo import MongoClient
+from pymongo.database import Database
+from pymongo.collection import Collection
+from config import settings
+
+
+class DatabaseManager:
+    """Singleton MongoDB connection manager"""
+    
+    _instance = None
+    _client: MongoClient = None
+    _db: Database = None
+    
+    def __new__(cls):
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+        return cls._instance
+    
+    def __init__(self):
+        if self._client is None:
+            self.connect()
+    
+    @property
+    def client(self):
+        """Get MongoDB client"""
+        return self._client
+    
+    @property
+    def db_name(self):
+        """Get database name"""
+        return settings.DB_NAME
+    
+    def connect(self):
+        """Establish connection to MongoDB"""
+        self._client = MongoClient(settings.MONGODB_URI)
+        self._db = self._client[settings.DB_NAME]
+        print(f"✓ Connected to MongoDB: {settings.DB_NAME}")
+    
+    def get_collection(self, collection_name: str) -> Collection:
+        """Get a MongoDB collection"""
+        return self._db[collection_name]
+    
+    def close(self):
+        """Close MongoDB connection"""
+        if self._client:
+            self._client.close()
+            print("✓ MongoDB connection closed")
+    
+    # Collection accessors
+    @property
+    def users(self) -> Collection:
+        return self.get_collection(settings.COLLECTION_USERS)
+    
+    @property
+    def payments(self) -> Collection:
+        return self.get_collection(settings.COLLECTION_PAYMENTS)
+    
+    @property
+    def event_versions(self) -> Collection:
+        return self.get_collection(settings.COLLECTION_EVENT_VERSIONS)
+    
+    @property
+    def user_follows(self) -> Collection:
+        return self.get_collection(settings.COLLECTION_USER_FOLLOWS)
+    
+    @property
+    def user_comment_post(self) -> Collection:
+        return self.get_collection(settings.COLLECTION_USER_COMMENT_POST)
+    
+    # AI Result Collections (DEPRECATED - use event-centric versions)
+    @property
+    def audience_segments(self) -> Collection:
+        """AudienceSegment collection (DEPRECATED - use event_audience_segments)"""
+        return self.get_collection(settings.COLLECTION_AUDIENCE_SEGMENTS)
+    
+    @property
+    def user_segment_assignments(self) -> Collection:
+        """UserSegmentAssignment collection"""
+        return self.get_collection(settings.COLLECTION_USER_SEGMENT_ASSIGNMENTS)
+    
+    @property
+    def sentiment_results(self) -> Collection:
+        """SentimentAnalysisResult collection"""
+        return self.get_collection(settings.COLLECTION_SENTIMENT_RESULTS)
+    
+    @property
+    def event_insights(self) -> Collection:
+        """EventInsightReport collection"""
+        return self.get_collection(settings.COLLECTION_EVENT_INSIGHTS)
+    
+    # NEW: Event-centric collections
+    @property
+    def event_audience_segments(self) -> Collection:
+        """EventAudienceSegment collection"""
+        return self.get_collection("EventAudienceSegment")
+    
+    @property
+    def event_sentiment_summary(self) -> Collection:
+        """EventSentimentSummary collection"""
+        return self.get_collection("EventSentimentSummary")
+
+
+# Global database instance
+db = DatabaseManager()

main.py

@@ -0,0 +1,91 @@
+"""
+Main Orchestration Script
+Author: AI Generated
+Created: 2025-11-24
+Purpose: Run the complete AI pipeline for Audience Segmentation
+"""
+
+import argparse
+from services.segmentation_service import SegmentationService
+from services.sentiment_service import SentimentAnalysisService
+from services.genai_service import GenerativeAIService
+from database import db
+
+
+def run_segmentation():
+    """Run audience segmentation pipeline"""
+    service = SegmentationService()
+    segment_ids = service.run_segmentation()
+    return segment_ids
+
+
+def run_sentiment_analysis():
+    """Run sentiment analysis pipeline"""
+    service = SentimentAnalysisService()
+    service.analyze_unprocessed_comments()
+
+
+def run_email_generation():
+    """Run email content generation for segments"""
+    service = GenerativeAIService()
+    service.generate_emails_for_all_segments()
+
+
+def run_insight_generation(event_code: str = None):
+    """Run insight generation for events"""
+    service = GenerativeAIService()
+    
+    if event_code:
+        service.generate_insights_for_event(event_code)
+    else:
+        # Get all unique event codes from comments
+        event_codes = db.user_comment_post.distinct("EventCode")
+        for code in event_codes:
+            if code:
+                service.generate_insights_for_event(code)
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Audience Segmentation AI Pipeline')
+    parser.add_argument(
+        '--task',
+        choices=['segmentation', 'sentiment', 'email', 'insights', 'all'],
+        default='all',
+        help='Which task to run'
+    )
+    parser.add_argument(
+        '--event-code',
+        type=str,
+        help='Specific event code for insight generation'
+    )
+    
+    args = parser.parse_args()
+    
+    try:
+        if args.task in ['segmentation', 'all']:
+            run_segmentation()
+        
+        if args.task in ['sentiment', 'all']:
+            run_sentiment_analysis()
+        
+        if args.task in ['email', 'all']:
+            run_email_generation()
+        
+        if args.task in ['insights', 'all']:
+            run_insight_generation(args.event_code)
+        
+        print("\n" + "=" * 60)
+        print("🎉 ALL TASKS COMPLETED SUCCESSFULLY!")
+        print("=" * 60)
+        
+    except Exception as e:
+        print(f"\n❌ Error: {e}")
+        import traceback
+        traceback.print_exc()
+    
+    finally:
+        db.close()
+
+
+if __name__ == "__main__":
+    main()

requirements.txt

@@ -0,0 +1,42 @@
+# FastAPI Backend Requirements
+# Updated for November 2025
+
+# Web Framework
+fast api==0.121.3
+uvicorn[standard]==0.38.0
+python-multipart==0.0.20
+
+# Database
+pymongo==4.15.4
+motor==3.7.0  # Async MongoDB driver for FastAPI
+
+# Data Validation
+pydantic==2.10.4
+pydantic-settings==2.12.0
+
+# Data Processing
+pandas==2.3.3
+numpy==2.3.5
+scikit-learn==1.7.2
+
+# NLP & AI
+transformers==4.57.1
+torch==2.9.1
+tokenizers==0.21.0
+
+# Vietnamese NLP
+pyvi==0.1.1
+
+# Generative AI (CPU-optimized)
+llama-cpp-python==0.3.6
+
+# Utilities
+python-dotenv==1.0.1
+tqdm==4.67.1
+
+# CORS & Security
+python-jose[cryptography]==3.4.0
+passlib[bcrypt]==1.7.4
+
+# Optional: Monitoring & Logging
+# prometheus-client==0.21.0