auto-analyst-backend-2

Running

App Files Files Community

auto-analyst-backend-2 / docs /api /routes /analytics.md

Arslan1997

Initial backend commit

141cc25 11 months ago

preview code

Raw

History Blame Contribute Delete

12.1 kB

Analytics Routes Documentation

These routes provide comprehensive analytics functionality for the Auto-Analyst backend, including dashboard summaries, user analytics, model performance metrics, cost analysis, and system monitoring.

Authentication

All analytics endpoints require admin authentication via API key:

ADMIN_API_KEY = os.getenv("ADMIN_API_KEY", "default-admin-key-change-me")

The API key can be provided via:

Header: X-Admin-API-Key
Query parameter: admin_api_key

Dashboard Endpoints

GET /analytics/dashboard

Returns comprehensive dashboard data combining usage statistics, model performance, and user activity.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "total_tokens": 123456,
  "total_cost": 25.50,
  "total_requests": 1000,
  "total_users": 50,
  "daily_usage": [
    {
      "date": "2023-05-01",
      "tokens": 5000,
      "cost": 1.25,
      "requests": 100
    }
  ],
  "model_usage": [
    {
      "model_name": "claude-3-sonnet-20241022",
      "tokens": 10000,
      "cost": 10.00,
      "requests": 200
    }
  ],
  "top_users": [
    {
      "user_id": "123",
      "tokens": 5000,
      "cost": 5.00,
      "requests": 50
    }
  ],
  "start_date": "2023-04-01",
  "end_date": "2023-05-01"
}

WebSocket /analytics/dashboard/realtime

WebSocket endpoint for real-time dashboard updates. Accepts connections and maintains them for broadcasting live data updates.

User Analytics Endpoints

GET /analytics/users

Returns user list with usage statistics from the past 7 days.

Query Parameters:

limit (optional): Maximum users to return (default: 100)
offset (optional): Pagination offset (default: 0)

Response:

{
  "users": [
    {
      "user_id": "123",
      "tokens": 5000,
      "cost": 5.00,
      "requests": 50,
      "first_seen": "2023-04-01T12:00:00Z",
      "last_seen": "2023-05-01T12:00:00Z"
    }
  ],
  "total": 200,
  "limit": 100,
  "offset": 0
}

GET /analytics/users/activity

Returns daily user activity metrics with new user tracking.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "user_activity": [
    {
      "date": "2023-05-01",
      "activeUsers": 20,
      "newUsers": 5,
      "sessions": 30
    }
  ]
}

GET /analytics/users/sessions/stats

Returns session statistics including total users, active users today, average queries per session, and average session time.

Response:

{
  "totalUsers": 500,
  "activeToday": 25,
  "avgQueriesPerSession": 3.2,
  "avgSessionTime": 300
}

WebSocket /analytics/realtime

WebSocket endpoint for real-time user analytics updates.

Model Analytics Endpoints

GET /analytics/usage/models

Returns model usage breakdown with performance metrics.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "model_usage": [
    {
      "model_name": "claude-3-sonnet-20241022",
      "tokens": 10000,
      "cost": 10.00,
      "requests": 200,
      "avg_response_time": 1.5
    }
  ]
}

GET /analytics/models/history

Returns daily model usage history with trend data.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "model_history": [
    {
      "date": "2023-05-01",
      "models": [
        {
          "name": "claude-3-sonnet-20241022",
          "tokens": 5000,
          "requests": 100
        }
      ]
    }
  ]
}

GET /analytics/models/metrics

Returns model performance metrics including success rates and response times.

Response:

{
  "model_metrics": [
    {
      "name": "claude-3-sonnet-20241022",
      "avg_tokens": 250.5,
      "avg_response_time": 1.2,
      "success_rate": 0.95
    }
  ]
}

Cost Analytics Endpoints

GET /analytics/costs/summary

Returns cost summary with averages and totals.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "totalCost": 25.50,
  "totalTokens": 100000,
  "totalRequests": 1000,
  "avgDailyCost": 0.85,
  "costPerThousandTokens": 0.255,
  "daysInPeriod": 30,
  "startDate": "2023-04-01",
  "endDate": "2023-05-01"
}

GET /analytics/costs/daily

Returns daily cost breakdown with filled gaps for missing dates.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "daily_costs": [
    {
      "date": "2023-05-01",
      "cost": 1.25,
      "tokens": 5000
    }
  ]
}

GET /analytics/costs/models

Returns cost breakdown by model.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "model_costs": [
    {
      "model_name": "claude-3-sonnet-20241022",
      "cost": 15.50,
      "tokens": 50000,
      "requests": 500
    }
  ]
}

GET /analytics/costs/projections

Returns cost projections based on last 30 days usage.

Response:

{
  "nextMonth": 75.00,
  "next3Months": 225.00,
  "nextYear": 900.00,
  "tokensNextMonth": 300000,
  "dailyCost": 2.50,
  "dailyTokens": 10000,
  "baselineDays": 30
}

GET /analytics/costs/today

Returns today's cost data.

Response:

{
  "date": "2023-05-01",
  "cost": 2.50,
  "tokens": 10000,
  "requests": 100
}

Tier Analytics Endpoints

GET /analytics/tiers/usage

Returns usage data categorized by model tiers with aggregated statistics.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "tier_data": {
    "tier_1": {
      "name": "Basic",
      "credits": 1,
      "total_tokens": 50000,
      "total_requests": 500,
      "total_cost": 5.00,
      "avg_tokens_per_query": 100,
      "cost_per_1k_tokens": 0.10,
      "total_credit_cost": 500,
      "cost_per_credit": 0.01,
      "models": [...]
    }
  },
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01"
}

GET /analytics/tiers/projections

Returns tier-based cost and usage projections.

Response:

{
  "daily_usage": {...},
  "projections": {
    "monthly": {...},
    "quarterly": {...},
    "yearly": {...}
  },
  "tier_definitions": {...}
}

GET /analytics/tiers/efficiency

Returns efficiency metrics by tier including cost per credit and tokens per credit.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "efficiency_data": {...},
  "most_efficient_tier": "tier_2",
  "best_value_tier": "tier_1",
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01"
}

Code Execution Analytics Endpoints

GET /analytics/code-executions/summary

Returns code execution statistics including success rates and model performance.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01",
  "overall_stats": {
    "total_executions": 1000,
    "successful_executions": 950,
    "failed_executions": 50,
    "success_rate": 0.95,
    "total_users": 100,
    "total_chats": 200
  },
  "model_performance": [...],
  "failed_agents": [...]
}

GET /analytics/code-executions/detailed

Returns detailed code execution records with filtering options.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)
success_filter (optional): Filter by success status (boolean)
user_id (optional): Filter by user ID
model_name (optional): Filter by model name
limit (optional): Maximum results (default: 100)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01", 
  "end_date": "2023-05-01",
  "count": 50,
  "executions": [...]
}

GET /analytics/code-executions/users

Returns code execution statistics grouped by user.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)
limit (optional): Maximum users (default: 50)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01", 
  "users": [...]
}

GET /analytics/code-executions/error-analysis

Returns error analysis with categorized error types and agent failure patterns.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01",
  "total_failed_executions": 50,
  "error_types": [...],
  "error_by_agent": [...]
}

Feedback Analytics Endpoints

GET /analytics/feedback/summary

Returns feedback summary statistics including rating distributions and trends.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01",
  "total_feedback": 500,
  "avg_rating": 4.2,
  "chats_with_feedback": 200,
  "ratings_distribution": [
    {"rating": 1, "count": 10},
    {"rating": 2, "count": 20},
    {"rating": 3, "count": 50},
    {"rating": 4, "count": 200},
    {"rating": 5, "count": 220}
  ],
  "models_data": [...],
  "feedback_trend": [...]
}

GET /analytics/feedback/detailed

Returns detailed feedback records with filtering and pagination.

Query Parameters:

period (optional): Time period (7d, 30d, 90d, default: 30d)
min_rating (optional): Minimum rating filter
max_rating (optional): Maximum rating filter
model_name (optional): Filter by model name
limit (optional): Maximum results (default: 100)
offset (optional): Pagination offset (default: 0)

Response:

{
  "period": "30d",
  "start_date": "2023-04-01",
  "end_date": "2023-05-01", 
  "total": 500,
  "count": 100,
  "offset": 0,
  "limit": 100,
  "feedback": [...]
}

Public Endpoints

GET /analytics/public/ticker

Returns public ticker data for landing page statistics. No authentication required.

Response:

{
  "total_signups": 1000,
  "total_tokens": 5000000,
  "total_requests": 50000,
  "last_updated": "2023-05-01T12:00:00Z"
}

Utility Endpoints

GET /analytics/usage/summary

Returns overall usage summary (legacy endpoint, calls dashboard with 30d period).

GET /analytics/debug/model_usage

Debug endpoint for testing admin API key validation.

Response:

{
  "status": "success",
  "message": "Admin API key validated successfully"
}

Error Categorization

The system automatically categorizes code execution errors into the following types:

NameError: Variable or function name not found
SyntaxError: Invalid Python syntax
TypeError: Type-related errors
AttributeError: Attribute access errors
IndexError/KeyError: Index or key access errors
ImportError: Module import errors
ValueError: Invalid values passed to functions
OperationError: Unsupported operations
IndentationError: Python indentation errors
PermissionError: File/system permission errors
FileNotFoundError: File access errors
MemoryError: Memory allocation errors
TimeoutError: Operation timeout errors
OtherError: Uncategorized errors

Real-time Updates

The analytics system supports real-time updates through WebSocket connections:

Dashboard updates: Broadcasted when new model usage is recorded
User activity updates: Broadcasted for user activity changes
Model performance updates: Broadcasted for model-specific metrics

All real-time updates are sent as JSON messages with type field indicating the update category and metrics containing the delta or new values.