Hugging Face's logo

Spaces:
shimen
/
kir
Sleeping

App Files Community
kir / docs /IMPLEMENTATION_TASKS.md
Shinway Deploy
feat: Add enhanced dashboard with HF Spaces support
866dfb3
preview code
|
Raw
History Blame Contribute Delete
10.6 kB

Implementation Tasks

Kiro Gateway: Metrics, Dashboard & Load Balancing

Master Task List

Phase 1: Metrics Foundation

1.1 Backend Metrics Module

  • TASK-1.1.1 Create /home/kiro/metrics/ directory structure
  • TASK-1.1.2 Create __init__.py with module exports
  • TASK-1.1.3 Implement prometheus.py:
    • Define Counter for kiro_requests_total
    • Define Histogram for kiro_request_duration_seconds
    • Define Counter for kiro_tokens_input_total
    • Define Counter for kiro_tokens_output_total
    • Define Gauge for kiro_active_requests
    • Define Counter for kiro_errors_total
  • TASK-1.1.4 Implement opentelemetry.py:
    • Configure TracerProvider
    • Configure MeterProvider
    • Set up OTLP exporter
    • Auto-instrument FastAPI
  • TASK-1.1.5 Implement middleware.py:
    • Create MetricsMiddleware class
    • Track request start/end times
    • Record status codes
    • Handle streaming responses
  • TASK-1.1.6 Implement collectors.py:
    • Custom collector for token usage
    • Per-model metrics aggregation

1.2 Integration

  • TASK-1.2.1 Add metrics middleware to main.py
  • TASK-1.2.2 Add /metrics endpoint for Prometheus scraping
  • TASK-1.2.3 Integrate token counting in streaming_openai.py
  • TASK-1.2.4 Integrate token counting in streaming_anthropic.py
  • TASK-1.2.5 Add metrics config to config.py:
    • METRICS_ENABLED
    • METRICS_PREFIX
    • OTEL_EXPORTER_OTLP_ENDPOINT
  • TASK-1.2.6 Update requirements.txt with metrics dependencies

1.3 Docker & Testing

  • TASK-1.3.1 Create docker/prometheus/prometheus.yml
  • TASK-1.3.2 Update docker-compose.yml with Prometheus service
  • TASK-1.3.3 Write unit tests for metrics collection
  • TASK-1.3.4 Test Prometheus scraping

Phase 2: Dashboard MVP

2.1 Project Setup

  • TASK-2.1.1 Initialize Next.js 16 project:
    npx create-next-app@latest dashboard --typescript --tailwind --app --src-dir
  • TASK-2.1.2 Configure tailwind.config.js
  • TASK-2.1.3 Install and configure shadcn/ui:
    npx shadcn-ui@latest init
  • TASK-2.1.4 Install Recharts:
    npm install recharts
  • TASK-2.1.5 Install dependencies:
    npm install @tanstack/react-query zustand socket.io-client lucide-react
  • TASK-2.1.6 Set up project structure (folders)

2.2 Core Components

  • TASK-2.2.1 Create components/ui/ with shadcn components:
    • Button, Card, Input, Select, Table, Tabs, Badge, Avatar
  • TASK-2.2.2 Create components/dashboard/Sidebar.tsx
  • TASK-2.2.3 Create components/dashboard/Header.tsx
  • TASK-2.2.4 Create components/dashboard/StatsCards.tsx
  • TASK-2.2.5 Create components/common/ThemeToggle.tsx
  • TASK-2.2.6 Create app/(dashboard)/layout.tsx

2.3 Charts (Recharts)

  • TASK-2.3.1 Create components/charts/RequestsChart.tsx (AreaChart)
  • TASK-2.3.2 Create components/charts/LatencyChart.tsx (BarChart)
  • TASK-2.3.3 Create components/charts/TokenUsageChart.tsx (ComposedChart)
  • TASK-2.3.4 Create components/charts/ErrorRateChart.tsx (PieChart)
  • TASK-2.3.5 Create components/charts/AccountHealthChart.tsx (BarChart)

2.4 Pages

  • TASK-2.4.1 Implement app/(dashboard)/overview/page.tsx:
    • Stats cards (requests, tokens, latency, uptime)
    • Mini charts
    • Recent activity feed
  • TASK-2.4.2 Implement app/(dashboard)/metrics/page.tsx:
    • Time range selector
    • Model filter
    • Full-size interactive charts
  • TASK-2.4.3 Implement app/(dashboard)/accounts/page.tsx:
    • Account list table
    • Health status indicators
    • Add/Edit modal
    • Delete confirmation
  • TASK-2.4.4 Implement app/(dashboard)/config/page.tsx:
    • Configuration viewer
    • Edit form for editable settings
    • Model aliases manager
  • TASK-2.4.5 Implement app/(dashboard)/logs/page.tsx:
    • Log level filter
    • Search input
    • Virtual scrolling log list
    • WebSocket connection

2.5 API & State

  • TASK-2.5.1 Create lib/api.ts:
    • Axios/fetch client setup
    • API base URL configuration
    • Auth header injection
  • TASK-2.5.2 Create lib/websocket.ts:
    • Socket.io connection
    • Event handlers
    • Reconnection logic
  • TASK-2.5.3 Create hooks/useMetrics.ts with TanStack Query
  • TASK-2.5.4 Create hooks/useAccounts.ts with mutations
  • TASK-2.5.5 Create Zustand store for global state
  • TASK-2.5.6 Create types/ TypeScript definitions

Phase 3: Load Balancer Core

3.1 Module Structure

  • TASK-3.1.1 Create /home/kiro/loadbalancer/ directory
  • TASK-3.1.2 Create __init__.py with exports
  • TASK-3.1.3 Create models.py:
    • KiroAccount Pydantic model
    • LoadBalancerConfig model
    • AccountHealth enum

3.2 Core Implementation

  • TASK-3.2.1 Implement pool.py:
    • AccountPool class
    • Account registration/removal
    • Get next account method
    • Account state tracking
  • TASK-3.2.2 Implement strategies.py:
    • LoadBalancingStrategy abstract base
    • RoundRobinStrategy
    • WeightedRoundRobinStrategy
    • LeastConnectionsStrategy
    • RandomStrategy
    • FailoverStrategy
  • TASK-3.2.3 Implement health_checker.py:
    • Async health check loop
    • Token refresh verification
    • API ping test
    • Error rate tracking

3.3 Integration

  • TASK-3.3.1 Modify KiroAuthManager to support multiple instances
  • TASK-3.3.2 Create account factory function
  • TASK-3.3.3 Update main.py lifespan to initialize pool
  • TASK-3.3.4 Update routes_openai.py to use load balancer:
    • Get account from pool before request
    • Return account to pool after request
    • Handle account failures
  • TASK-3.3.5 Update routes_anthropic.py similarly
  • TASK-3.3.6 Add load balancer config to config.py:
    • LOAD_BALANCER_ENABLED
    • LOAD_BALANCER_STRATEGY
    • LOAD_BALANCER_ACCOUNTS_FILE
    • LOAD_BALANCER_HEALTH_CHECK_INTERVAL

3.4 Account Management

  • TASK-3.4.1 Create accounts.json schema
  • TASK-3.4.2 Implement account loading from JSON file
  • TASK-3.4.3 Implement account loading from environment variables
  • TASK-3.4.4 Implement runtime account CRUD API

Phase 4: Dashboard Backend API

4.1 API Implementation

  • TASK-4.1.1 Create /home/kiro/dashboard_api/ directory
  • TASK-4.1.2 Create models.py:
    • DashboardStats response model
    • MetricsQuery request model
    • AccountCreate/Update models
  • TASK-4.1.3 Implement routes.py:
    • GET /api/dashboard/stats
    • GET /api/dashboard/metrics
    • GET /api/dashboard/accounts
    • POST /api/dashboard/accounts
    • PUT /api/dashboard/accounts/{id}
    • DELETE /api/dashboard/accounts/{id}
    • GET /api/dashboard/config
    • PUT /api/dashboard/config
  • TASK-4.1.4 Implement websocket.py:
    • WebSocket endpoint for logs
    • Real-time metrics streaming
  • TASK-4.1.5 Register routes in main.py

4.2 CORS & Security

  • TASK-4.2.1 Configure CORS for dashboard origin
  • TASK-4.2.2 Add dashboard API key verification
  • TASK-4.2.3 Mask sensitive data in responses

Phase 5: Polish & Deployment

5.1 Dashboard Polish

  • TASK-5.1.1 Implement dark/light theme persistence
  • TASK-5.1.2 Add loading states to all components
  • TASK-5.1.3 Add error boundaries
  • TASK-5.1.4 Implement responsive design for mobile
  • TASK-5.1.5 Add keyboard shortcuts
  • TASK-5.1.6 Create empty states for no data

5.2 Testing

  • TASK-5.2.1 Write unit tests for load balancer strategies
  • TASK-5.2.2 Write unit tests for health checker
  • TASK-5.2.3 Write integration tests for dashboard API
  • TASK-5.2.4 Write Playwright E2E tests for dashboard
  • TASK-5.2.5 Performance testing for metrics overhead

5.3 Docker & Deployment

  • TASK-5.3.1 Create docker/Dockerfile.dashboard
  • TASK-5.3.2 Create docker/docker-compose.full.yml:
    • Kiro Gateway service
    • Dashboard service
    • Prometheus service
    • Grafana service (optional)
  • TASK-5.3.3 Create Grafana dashboard templates
  • TASK-5.3.4 Update main README with new features
  • TASK-5.3.5 Create deployment guide

Quick Reference: File Locations

Backend (Python) 

/home/kiro/metrics/__init__.py
/home/kiro/metrics/prometheus.py
/home/kiro/metrics/opentelemetry.py
/home/kiro/metrics/middleware.py
/home/kiro/metrics/collectors.py
/home/kiro/loadbalancer/__init__.py
/home/kiro/loadbalancer/pool.py
/home/kiro/loadbalancer/strategies.py
/home/kiro/loadbalancer/health_checker.py
/home/kiro/loadbalancer/models.py
/home/kiro/dashboard_api/__init__.py
/home/kiro/dashboard_api/routes.py
/home/kiro/dashboard_api/models.py
/home/kiro/dashboard_api/websocket.py

Dashboard (Next.js) 

/home/dashboard/app/layout.tsx
/home/dashboard/app/(dashboard)/overview/page.tsx
/home/dashboard/app/(dashboard)/metrics/page.tsx
/home/dashboard/app/(dashboard)/accounts/page.tsx
/home/dashboard/app/(dashboard)/config/page.tsx
/home/dashboard/app/(dashboard)/logs/page.tsx
/home/dashboard/components/ui/*
/home/dashboard/components/charts/*
/home/dashboard/components/dashboard/*
/home/dashboard/lib/api.ts
/home/dashboard/lib/websocket.ts
/home/dashboard/hooks/*
/home/dashboard/types/*

Docker 

/home/docker/Dockerfile.dashboard
/home/docker/docker-compose.full.yml
/home/docker/prometheus/prometheus.yml

Estimated Timeline

Phase Duration Dependencies
Phase 1: Metrics 1 week None
Phase 2: Dashboard MVP 2 weeks Phase 1 partial
Phase 3: Load Balancer 1.5 weeks None
Phase 4: Dashboard API 1 week Phase 1, 3
Phase 5: Polish 1 week All phases

Total: ~6-7 weeks

Priority Order (MCP - Most Critical Path)

  1. HIGH - TASK-1.1.3 (Prometheus metrics) - Foundation for everything
  2. HIGH - TASK-3.2.1 (Account pool) - Core load balancer
  3. HIGH - TASK-2.1.1 (Next.js setup) - Dashboard foundation
  4. MEDIUM - TASK-1.2.2 (/metrics endpoint) - Enables monitoring
  5. MEDIUM - TASK-3.2.2 (LB strategies) - Load balancer features
  6. MEDIUM - TASK-2.4.1 (Overview page) - Dashboard MVP
  7. LOW - TASK-5.3.3 (Grafana templates) - Nice to have