| # Implementation Tasks |
| # Kiro Gateway: Metrics, Dashboard & Load Balancing |
|
|
| ## Master Task List |
|
|
| --- |
|
|
| ## Phase 1: Metrics Foundation |
|
|
| ### 1.1 Backend Metrics Module |
|
|
| - [ ] **TASK-1.1.1** Create `/home/kiro/metrics/` directory structure |
| - [ ] **TASK-1.1.2** Create `__init__.py` with module exports |
| - [ ] **TASK-1.1.3** Implement `prometheus.py`: |
| - Define Counter for `kiro_requests_total` |
| - Define Histogram for `kiro_request_duration_seconds` |
| - Define Counter for `kiro_tokens_input_total` |
| - Define Counter for `kiro_tokens_output_total` |
| - Define Gauge for `kiro_active_requests` |
| - Define Counter for `kiro_errors_total` |
| - [ ] **TASK-1.1.4** Implement `opentelemetry.py`: |
| - Configure TracerProvider |
| - Configure MeterProvider |
| - Set up OTLP exporter |
| - Auto-instrument FastAPI |
| - [ ] **TASK-1.1.5** Implement `middleware.py`: |
| - Create `MetricsMiddleware` class |
| - Track request start/end times |
| - Record status codes |
| - Handle streaming responses |
| - [ ] **TASK-1.1.6** Implement `collectors.py`: |
| - Custom collector for token usage |
| - Per-model metrics aggregation |
|
|
| ### 1.2 Integration |
|
|
| - [ ] **TASK-1.2.1** Add metrics middleware to `main.py` |
| - [ ] **TASK-1.2.2** Add `/metrics` endpoint for Prometheus scraping |
| - [ ] **TASK-1.2.3** Integrate token counting in `streaming_openai.py` |
| - [ ] **TASK-1.2.4** Integrate token counting in `streaming_anthropic.py` |
| - [ ] **TASK-1.2.5** Add metrics config to `config.py`: |
| - `METRICS_ENABLED` |
| - `METRICS_PREFIX` |
| - `OTEL_EXPORTER_OTLP_ENDPOINT` |
| - [ ] **TASK-1.2.6** Update `requirements.txt` with metrics dependencies |
|
|
| ### 1.3 Docker & Testing |
|
|
| - [ ] **TASK-1.3.1** Create `docker/prometheus/prometheus.yml` |
| - [ ] **TASK-1.3.2** Update `docker-compose.yml` with Prometheus service |
| - [ ] **TASK-1.3.3** Write unit tests for metrics collection |
| - [ ] **TASK-1.3.4** Test Prometheus scraping |
|
|
| --- |
|
|
| ## Phase 2: Dashboard MVP |
|
|
| ### 2.1 Project Setup |
|
|
| - [ ] **TASK-2.1.1** Initialize Next.js 16 project: |
| ```bash |
| npx create-next-app@latest dashboard --typescript --tailwind --app --src-dir |
| ``` |
| - [ ] **TASK-2.1.2** Configure `tailwind.config.js` |
| - [ ] **TASK-2.1.3** Install and configure shadcn/ui: |
| ```bash |
| npx shadcn-ui@latest init |
| ``` |
| - [ ] **TASK-2.1.4** Install Recharts: |
| ```bash |
| npm install recharts |
| ``` |
| - [ ] **TASK-2.1.5** Install dependencies: |
| ```bash |
| npm install @tanstack/react-query zustand socket.io-client lucide-react |
| ``` |
| - [ ] **TASK-2.1.6** Set up project structure (folders) |
|
|
| ### 2.2 Core Components |
|
|
| - [ ] **TASK-2.2.1** Create `components/ui/` with shadcn components: |
| - Button, Card, Input, Select, Table, Tabs, Badge, Avatar |
| - [ ] **TASK-2.2.2** Create `components/dashboard/Sidebar.tsx` |
| - [ ] **TASK-2.2.3** Create `components/dashboard/Header.tsx` |
| - [ ] **TASK-2.2.4** Create `components/dashboard/StatsCards.tsx` |
| - [ ] **TASK-2.2.5** Create `components/common/ThemeToggle.tsx` |
| - [ ] **TASK-2.2.6** Create `app/(dashboard)/layout.tsx` |
|
|
| ### 2.3 Charts (Recharts) |
|
|
| - [ ] **TASK-2.3.1** Create `components/charts/RequestsChart.tsx` (AreaChart) |
| - [ ] **TASK-2.3.2** Create `components/charts/LatencyChart.tsx` (BarChart) |
| - [ ] **TASK-2.3.3** Create `components/charts/TokenUsageChart.tsx` (ComposedChart) |
| - [ ] **TASK-2.3.4** Create `components/charts/ErrorRateChart.tsx` (PieChart) |
| - [ ] **TASK-2.3.5** Create `components/charts/AccountHealthChart.tsx` (BarChart) |
|
|
| ### 2.4 Pages |
|
|
| - [ ] **TASK-2.4.1** Implement `app/(dashboard)/overview/page.tsx`: |
| - Stats cards (requests, tokens, latency, uptime) |
| - Mini charts |
| - Recent activity feed |
| - [ ] **TASK-2.4.2** Implement `app/(dashboard)/metrics/page.tsx`: |
| - Time range selector |
| - Model filter |
| - Full-size interactive charts |
| - [ ] **TASK-2.4.3** Implement `app/(dashboard)/accounts/page.tsx`: |
| - Account list table |
| - Health status indicators |
| - Add/Edit modal |
| - Delete confirmation |
| - [ ] **TASK-2.4.4** Implement `app/(dashboard)/config/page.tsx`: |
| - Configuration viewer |
| - Edit form for editable settings |
| - Model aliases manager |
| - [ ] **TASK-2.4.5** Implement `app/(dashboard)/logs/page.tsx`: |
| - Log level filter |
| - Search input |
| - Virtual scrolling log list |
| - WebSocket connection |
|
|
| ### 2.5 API & State |
|
|
| - [ ] **TASK-2.5.1** Create `lib/api.ts`: |
| - Axios/fetch client setup |
| - API base URL configuration |
| - Auth header injection |
| - [ ] **TASK-2.5.2** Create `lib/websocket.ts`: |
| - Socket.io connection |
| - Event handlers |
| - Reconnection logic |
| - [ ] **TASK-2.5.3** Create `hooks/useMetrics.ts` with TanStack Query |
| - [ ] **TASK-2.5.4** Create `hooks/useAccounts.ts` with mutations |
| - [ ] **TASK-2.5.5** Create Zustand store for global state |
| - [ ] **TASK-2.5.6** Create `types/` TypeScript definitions |
|
|
| --- |
|
|
| ## Phase 3: Load Balancer Core |
|
|
| ### 3.1 Module Structure |
|
|
| - [ ] **TASK-3.1.1** Create `/home/kiro/loadbalancer/` directory |
| - [ ] **TASK-3.1.2** Create `__init__.py` with exports |
| - [ ] **TASK-3.1.3** Create `models.py`: |
| - `KiroAccount` Pydantic model |
| - `LoadBalancerConfig` model |
| - `AccountHealth` enum |
|
|
| ### 3.2 Core Implementation |
|
|
| - [ ] **TASK-3.2.1** Implement `pool.py`: |
| - `AccountPool` class |
| - Account registration/removal |
| - Get next account method |
| - Account state tracking |
| - [ ] **TASK-3.2.2** Implement `strategies.py`: |
| - `LoadBalancingStrategy` abstract base |
| - `RoundRobinStrategy` |
| - `WeightedRoundRobinStrategy` |
| - `LeastConnectionsStrategy` |
| - `RandomStrategy` |
| - `FailoverStrategy` |
| - [ ] **TASK-3.2.3** Implement `health_checker.py`: |
| - Async health check loop |
| - Token refresh verification |
| - API ping test |
| - Error rate tracking |
|
|
| ### 3.3 Integration |
|
|
| - [ ] **TASK-3.3.1** Modify `KiroAuthManager` to support multiple instances |
| - [ ] **TASK-3.3.2** Create account factory function |
| - [ ] **TASK-3.3.3** Update `main.py` lifespan to initialize pool |
| - [ ] **TASK-3.3.4** Update `routes_openai.py` to use load balancer: |
| - Get account from pool before request |
| - Return account to pool after request |
| - Handle account failures |
| - [ ] **TASK-3.3.5** Update `routes_anthropic.py` similarly |
| - [ ] **TASK-3.3.6** Add load balancer config to `config.py`: |
| - `LOAD_BALANCER_ENABLED` |
| - `LOAD_BALANCER_STRATEGY` |
| - `LOAD_BALANCER_ACCOUNTS_FILE` |
| - `LOAD_BALANCER_HEALTH_CHECK_INTERVAL` |
|
|
| ### 3.4 Account Management |
|
|
| - [ ] **TASK-3.4.1** Create `accounts.json` schema |
| - [ ] **TASK-3.4.2** Implement account loading from JSON file |
| - [ ] **TASK-3.4.3** Implement account loading from environment variables |
| - [ ] **TASK-3.4.4** Implement runtime account CRUD API |
|
|
| --- |
|
|
| ## Phase 4: Dashboard Backend API |
|
|
| ### 4.1 API Implementation |
|
|
| - [ ] **TASK-4.1.1** Create `/home/kiro/dashboard_api/` directory |
| - [ ] **TASK-4.1.2** Create `models.py`: |
| - `DashboardStats` response model |
| - `MetricsQuery` request model |
| - `AccountCreate/Update` models |
| - [ ] **TASK-4.1.3** Implement `routes.py`: |
| - `GET /api/dashboard/stats` |
| - `GET /api/dashboard/metrics` |
| - `GET /api/dashboard/accounts` |
| - `POST /api/dashboard/accounts` |
| - `PUT /api/dashboard/accounts/{id}` |
| - `DELETE /api/dashboard/accounts/{id}` |
| - `GET /api/dashboard/config` |
| - `PUT /api/dashboard/config` |
| - [ ] **TASK-4.1.4** Implement `websocket.py`: |
| - WebSocket endpoint for logs |
| - Real-time metrics streaming |
| - [ ] **TASK-4.1.5** Register routes in `main.py` |
|
|
| ### 4.2 CORS & Security |
|
|
| - [ ] **TASK-4.2.1** Configure CORS for dashboard origin |
| - [ ] **TASK-4.2.2** Add dashboard API key verification |
| - [ ] **TASK-4.2.3** Mask sensitive data in responses |
|
|
| --- |
|
|
| ## Phase 5: Polish & Deployment |
|
|
| ### 5.1 Dashboard Polish |
|
|
| - [ ] **TASK-5.1.1** Implement dark/light theme persistence |
| - [ ] **TASK-5.1.2** Add loading states to all components |
| - [ ] **TASK-5.1.3** Add error boundaries |
| - [ ] **TASK-5.1.4** Implement responsive design for mobile |
| - [ ] **TASK-5.1.5** Add keyboard shortcuts |
| - [ ] **TASK-5.1.6** Create empty states for no data |
|
|
| ### 5.2 Testing |
|
|
| - [ ] **TASK-5.2.1** Write unit tests for load balancer strategies |
| - [ ] **TASK-5.2.2** Write unit tests for health checker |
| - [ ] **TASK-5.2.3** Write integration tests for dashboard API |
| - [ ] **TASK-5.2.4** Write Playwright E2E tests for dashboard |
| - [ ] **TASK-5.2.5** Performance testing for metrics overhead |
|
|
| ### 5.3 Docker & Deployment |
|
|
| - [ ] **TASK-5.3.1** Create `docker/Dockerfile.dashboard` |
| - [ ] **TASK-5.3.2** Create `docker/docker-compose.full.yml`: |
| - Kiro Gateway service |
| - Dashboard service |
| - Prometheus service |
| - Grafana service (optional) |
| - [ ] **TASK-5.3.3** Create Grafana dashboard templates |
| - [ ] **TASK-5.3.4** Update main README with new features |
| - [ ] **TASK-5.3.5** Create deployment guide |
|
|
| --- |
|
|
| ## Quick Reference: File Locations |
|
|
| ### Backend (Python) |
| ``` |
| /home/kiro/metrics/__init__.py |
| /home/kiro/metrics/prometheus.py |
| /home/kiro/metrics/opentelemetry.py |
| /home/kiro/metrics/middleware.py |
| /home/kiro/metrics/collectors.py |
| /home/kiro/loadbalancer/__init__.py |
| /home/kiro/loadbalancer/pool.py |
| /home/kiro/loadbalancer/strategies.py |
| /home/kiro/loadbalancer/health_checker.py |
| /home/kiro/loadbalancer/models.py |
| /home/kiro/dashboard_api/__init__.py |
| /home/kiro/dashboard_api/routes.py |
| /home/kiro/dashboard_api/models.py |
| /home/kiro/dashboard_api/websocket.py |
| ``` |
|
|
| ### Dashboard (Next.js) |
| ``` |
| /home/dashboard/app/layout.tsx |
| /home/dashboard/app/(dashboard)/overview/page.tsx |
| /home/dashboard/app/(dashboard)/metrics/page.tsx |
| /home/dashboard/app/(dashboard)/accounts/page.tsx |
| /home/dashboard/app/(dashboard)/config/page.tsx |
| /home/dashboard/app/(dashboard)/logs/page.tsx |
| /home/dashboard/components/ui/* |
| /home/dashboard/components/charts/* |
| /home/dashboard/components/dashboard/* |
| /home/dashboard/lib/api.ts |
| /home/dashboard/lib/websocket.ts |
| /home/dashboard/hooks/* |
| /home/dashboard/types/* |
| ``` |
|
|
| ### Docker |
| ``` |
| /home/docker/Dockerfile.dashboard |
| /home/docker/docker-compose.full.yml |
| /home/docker/prometheus/prometheus.yml |
| ``` |
|
|
| --- |
|
|
| ## Estimated Timeline |
|
|
| | Phase | Duration | Dependencies | |
| |-------|----------|--------------| |
| | Phase 1: Metrics | 1 week | None | |
| | Phase 2: Dashboard MVP | 2 weeks | Phase 1 partial | |
| | Phase 3: Load Balancer | 1.5 weeks | None | |
| | Phase 4: Dashboard API | 1 week | Phase 1, 3 | |
| | Phase 5: Polish | 1 week | All phases | |
|
|
| **Total: ~6-7 weeks** |
|
|
| --- |
|
|
| ## Priority Order (MCP - Most Critical Path) |
|
|
| 1. **HIGH** - TASK-1.1.3 (Prometheus metrics) - Foundation for everything |
| 2. **HIGH** - TASK-3.2.1 (Account pool) - Core load balancer |
| 3. **HIGH** - TASK-2.1.1 (Next.js setup) - Dashboard foundation |
| 4. **MEDIUM** - TASK-1.2.2 (/metrics endpoint) - Enables monitoring |
| 5. **MEDIUM** - TASK-3.2.2 (LB strategies) - Load balancer features |
| 6. **MEDIUM** - TASK-2.4.1 (Overview page) - Dashboard MVP |
| 7. **LOW** - TASK-5.3.3 (Grafana templates) - Nice to have |
|
|
