Add files via upload

PROJECT_STRUCTURE_REPORT.md

@@ -0,0 +1,128 @@
+# Crypto Data Source - Project Structure Report
+
+## Overview
+
+A comprehensive cryptocurrency data aggregation and analysis platform designed for deployment on Hugging Face Spaces. The system provides real-time market data, AI-powered sentiment analysis, trading signals, and multi-source data aggregation.
+
+## Architecture Layers
+
+### 1. **Entry Points**
+
+- **`main.py`**: FastAPI entry point for HuggingFace Spaces (port 7860)
+- **`app.py`**: Flask-based fallback server with basic endpoints
+- **`hf_unified_server.py`**: Main FastAPI application with unified routing
+
+### 2. **API Layer** (`/api/`)
+
+- **FastAPI Routers** (`backend/routers/`): 28 router modules for different API domains
+- **Legacy Endpoints** (`api/`): 15+ endpoint modules for various services
+- **WebSocket Support**: Real-time data streaming via WebSocket endpoints
+- **Key Features**:
+  - Multi-source data aggregation
+  - AI trading signals and sentiment analysis
+  - OHLCV data endpoints
+  - News aggregation
+  - Resource management APIs
+
+### 3. **Backend Services** (`backend/services/`)
+
+- **70 service modules** organized by functionality:
+  - **Data Collection**: `unified_data_collector.py`, `market_data_aggregator.py`, `news_aggregator.py`
+  - **AI/ML**: `real_ai_models.py`, `ai_service_unified.py`, `hf_inference_api_client.py`
+  - **Trading**: `futures_trading_service.py`, `backtesting_service.py`
+  - **Providers**: Integration with CoinGecko, Binance, CryptoPanic, etc.
+  - **Fallback Management**: `multi_source_fallback_engine.py`, `provider_fallback_manager.py`
+  - **Resource Management**: `master_resource_orchestrator.py`, `resources_registry_service.py`
+
+### 4. **Data Collection** (`collectors/`)
+
+- **15 collector modules** for:
+  - Market data collection
+  - News aggregation
+  - Sentiment analysis
+  - On-chain data
+  - Whale tracking
+  - Scheduled data collection
+
+### 5. **Database Layer** (`database/`)
+
+- **SQLAlchemy models** (`models.py`)
+- **Database manager** (`db_manager.py`)
+- **Data access layer** (`data_access.py`)
+- **Migration support** (`migrations.py`)
+- **Schema definition** (`schema_complete.sql`)
+
+### 6. **Monitoring & Health** (`monitoring/`)
+
+- Health checking system
+- Rate limiting
+- Source pool management
+- Scheduler for background tasks
+
+### 7. **Core Infrastructure** (`core/`)
+
+- Smart proxy manager
+- Smart fallback manager
+- Resource management utilities
+
+### 8. **Configuration**
+
+- **`config.py`**: Main configuration with HuggingFace integration
+- **`providers_config_extended.json`**: Provider configurations
+- **`api-resources/`**: Unified API resource registry
+- **Strategy/Scoring configs**: Trading and scoring configurations
+
+### 9. **Frontend** (`static/`, `templates/`)
+
+- **263 static files**: HTML, CSS, JavaScript
+- Dashboard UI
+- System monitoring interface
+- Multi-page architecture
+
+### 10. **Workers** (`workers/`)
+
+- Background worker processes
+- Data processing tasks
+
+## Key Technologies
+
+- **Backend**: FastAPI, Flask
+- **AI/ML**: HuggingFace Inference API, custom sentiment models
+- **Data Sources**: CoinGecko, Binance, CryptoPanic, AlphaVantage, etc.
+- **Database**: SQLAlchemy (SQLite/PostgreSQL)
+- **Real-time**: WebSocket support
+- **Deployment**: Docker, HuggingFace Spaces
+
+## Key Features
+
+1. **Multi-Source Data Aggregation**: Aggregates data from 70+ API providers
+2. **AI-Powered Analysis**: Sentiment analysis, trading signals, decision support
+3. **Fallback System**: Automatic failover between data sources
+4. **Real-time Updates**: WebSocket support for live data streaming
+5. **Resource Management**: Dynamic API key rotation and smart access management
+6. **Health Monitoring**: Self-healing system with health checks
+7. **Trading Support**: Backtesting, futures trading, signal generation
+
+## Project Statistics
+
+- **Total Python Files**: ~200+
+- **API Endpoints**: 100+ endpoints across multiple routers
+- **Service Modules**: 70 backend services
+- **Data Collectors**: 15 collector modules
+- **API Providers**: 70+ integrated providers
+- **Frontend Assets**: 263 static files
+
+## Deployment
+
+- **Primary**: HuggingFace Spaces (Docker)
+- **Port**: 7860 (HF standard)
+- **Entry Point**: `hf_unified_server:app`
+- **Health Check**: `/api/health`
+
+## Notable Design Patterns
+
+- **Multi-source fallback**: Automatic provider switching on failure
+- **Lazy loading**: Resources loaded on-demand to optimize memory
+- **Service-oriented**: Modular service architecture
+- **Router-based**: FastAPI router pattern for API organization
+- **Provider abstraction**: Unified interface for multiple data sources

UI_STRUCTURE_GUIDE.md

@@ -0,0 +1,339 @@
+# Crypto Intelligence Hub - UI Structure Guide
+
+## Overview
+
+The application uses a **multi-page architecture** with shared components and a dynamic layout injection system. Each page is standalone but shares common layouts, utilities, and components.
+
+## Architecture
+
+### 1. **Page Structure** (`/static/pages/`)
+
+Each page is a self-contained module in its own directory:
+
+```
+static/pages/
+├── dashboard/          # Main dashboard with stats
+├── market/            # Market data and prices
+├── models/            # AI models status
+├── sentiment/         # Sentiment analysis
+├── ai-analyst/        # AI trading advisor
+├── trading-assistant/  # Trading signals
+├── news/              # News feed
+├── providers/         # API provider management
+├── diagnostics/       # System diagnostics
+└── api-explorer/     # API testing tool
+```
+
+**Page Template Structure:**
+
+```html
+<!DOCTYPE html>
+<html>
+<head>
+  <!-- Meta, CSS imports -->
+  <!-- Shared CSS: design-system, layout, components -->
+  <!-- Page-specific CSS -->
+</head>
+<body>
+  <div class="app-container">
+    <!-- Layout containers (injected by LayoutManager) -->
+    <aside id="sidebar-container"></aside>
+    <header id="header-container"></header>
+    
+    <!-- Page-specific content -->
+    <main class="main-content">
+      <div class="page-content">
+        <!-- Your page content here -->
+      </div>
+    </main>
+  </div>
+  
+  <!-- Scripts -->
+  <script type="module">
+    // Initialize LayoutManager
+    import LayoutManager from '/static/shared/js/core/layout-manager.js';
+    await LayoutManager.init('page-name');
+    
+    // Your page logic here
+  </script>
+</body>
+</html>
+```
+
+### 2. **Shared Components** (`/static/shared/`)
+
+#### **Layouts** (`/shared/layouts/`)
+
+- `header.html` - App header with status badge
+- `sidebar.html` - Navigation sidebar
+- `footer.html` - Footer content
+
+#### **Core JavaScript** (`/shared/js/core/`)
+
+- `layout-manager.js` - Injects layouts, manages navigation
+- `api-client.js` - HTTP client with caching
+- `polling-manager.js` - Auto-refresh system
+- `config.js` - Central configuration
+
+#### **Components** (`/shared/js/components/`)
+
+- `toast.js` - Notification system
+- `modal.js` - Modal dialogs
+- `table.js` - Data tables
+- `chart.js` - Chart.js wrapper
+- `loading.js` - Loading states
+
+#### **CSS** (`/shared/css/`)
+
+- `design-system.css` - CSS variables, tokens
+- `global.css` - Base styles
+- `layout.css` - Layout styles
+- `components.css` - Component styles
+
+### 3. **NewResourceApi** (`/NewResourceApi/`)
+
+Protobuf-based API structure for resource management:
+
+- `api.py` - API definitions
+- `api_pb2.py` - Generated protobuf code
+- `test_api.py` - API testing utilities
+
+## Key Systems
+
+### Layout Manager
+
+**Purpose:** Dynamically injects shared layouts (header, sidebar, footer) into pages.
+
+**Usage:**
+
+```javascript
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+
+// Initialize layouts (injects header, sidebar, footer)
+await LayoutManager.init('dashboard'); // 'dashboard' = active page name
+
+// Set active page in navigation
+LayoutManager.setActivePage('market');
+
+// Update API status badge
+LayoutManager.updateApiStatus('online', '✓ System Active');
+```
+
+**Features:**
+
+- Automatic layout injection
+- API health monitoring
+- Theme management (dark/light)
+- Mobile-responsive sidebar
+- Fallback layouts if files fail to load
+
+### API Client
+
+**Purpose:** Centralized HTTP client with caching and error handling.
+
+**Usage:**
+
+```javascript
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+
+const client = new ApiClient();
+
+// GET request with caching
+const data = await client.get('/api/market/top', {
+  cache: true,
+  ttl: 30000 // 30 seconds
+});
+
+// POST request
+const result = await client.post('/api/sentiment/analyze', {
+  text: 'Bitcoin is bullish!'
+});
+```
+
+### Polling Manager
+
+**Purpose:** Auto-refresh data with smart pause/resume.
+
+**Usage:**
+
+```javascript
+import { PollingManager } from '/static/shared/js/core/polling-manager.js';
+
+const poller = new PollingManager({
+  interval: 5000, // 5 seconds
+  pauseOnHidden: true // Pause when tab is hidden
+});
+
+// Start polling
+poller.start(async () => {
+  const data = await fetch('/api/market/top').then(r => r.json());
+  updateUI(data);
+});
+
+// Stop polling
+poller.stop();
+```
+
+### Component System
+
+**Toast Notifications:**
+
+```javascript
+import { Toast } from '/static/shared/js/components/toast.js';
+
+Toast.success('Data loaded successfully');
+Toast.error('Failed to fetch data');
+Toast.info('Processing...');
+```
+
+**Modal Dialogs:**
+
+```javascript
+import { Modal } from '/static/shared/js/components/modal.js';
+
+const modal = new Modal({
+  title: 'Confirm Action',
+  content: '<p>Are you sure?</p>',
+  buttons: [
+    { text: 'Cancel', action: () => modal.close() },
+    { text: 'Confirm', action: () => { /* ... */ } }
+  ]
+});
+modal.show();
+```
+
+## Page Development Workflow
+
+### Step 1: Create Page Directory
+
+```
+static/pages/my-page/
+├── index.html
+├── my-page.css
+└── my-page.js (optional)
+```
+
+### Step 2: Create HTML Structure
+
+```html
+<!DOCTYPE html>
+<html lang="en" data-theme="dark">
+<head>
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>My Page | Crypto Hub</title>
+  
+  <!-- Shared CSS -->
+  <link rel="stylesheet" href="/static/shared/css/design-system.css">
+  <link rel="stylesheet" href="/static/shared/css/layout.css">
+  <link rel="stylesheet" href="/static/shared/css/components.css">
+  
+  <!-- Page CSS -->
+  <link rel="stylesheet" href="/static/pages/my-page/my-page.css">
+</head>
+<body>
+  <div class="app-container">
+    <aside id="sidebar-container"></aside>
+    <header id="header-container"></header>
+    
+    <main class="main-content">
+      <div class="page-content">
+        <h1>My Page</h1>
+        <!-- Your content -->
+      </div>
+    </main>
+  </div>
+  
+  <script type="module">
+    import LayoutManager from '/static/shared/js/core/layout-manager.js';
+    import { ApiClient } from '/static/shared/js/core/api-client.js';
+    
+    // Initialize layout
+    await LayoutManager.init('my-page');
+    
+    // Your page logic
+    const client = new ApiClient();
+    const data = await client.get('/api/endpoint');
+    // ... update UI
+  </script>
+</body>
+</html>
+```
+
+### Step 3: Add to Navigation
+
+Update `/static/shared/layouts/sidebar.html`:
+
+```html
+<li>
+  <a href="/static/pages/my-page/index.html" 
+     class="nav-link" 
+     data-page="my-page">
+    My Page
+  </a>
+</li>
+```
+
+### Step 4: Register in Config
+
+Update `/static/shared/js/core/config.js`:
+
+```javascript
+export const PAGE_METADATA = [
+  // ... existing pages
+  { page: 'my-page', title: 'My Page | Crypto Hub', icon: 'star' }
+];
+```
+
+## Best Practices
+
+1. **Always use LayoutManager.init()** - Ensures layouts are injected
+2. **Use shared components** - Don't reinvent toast, modal, etc.
+3. **Follow naming conventions** - Page name matches directory name
+4. **Use API client** - Don't use raw fetch() for API calls
+5. **Handle loading states** - Use Loading component
+6. **Responsive design** - Test on mobile (sidebar auto-hides)
+7. **Error handling** - Use Toast for user feedback
+8. **Cache API calls** - Use ApiClient caching for performance
+
+## File Paths
+
+**Absolute paths (recommended):**
+
+- `/static/shared/js/core/layout-manager.js`
+- `/static/pages/dashboard/index.html`
+
+**Relative paths (from page directory):**
+
+- `../../shared/js/core/layout-manager.js`
+- `../dashboard/index.html`
+
+## Theme System
+
+The app supports dark/light themes:
+
+```javascript
+// Toggle theme
+LayoutManager.toggleTheme();
+
+// Get current theme
+const theme = document.documentElement.getAttribute('data-theme');
+```
+
+Theme is persisted in `localStorage` as `crypto_monitor_theme`.
+
+## API Status Monitoring
+
+LayoutManager automatically monitors API health:
+
+- Checks `/api/health` every 30 seconds
+- Updates status badge in header
+- Pauses when tab is hidden
+- Enters offline mode after 3 failures
+
+## Mobile Support
+
+- Sidebar auto-hides on screens < 1024px
+- Hamburger menu in header toggles sidebar
+- Touch-friendly components
+- Responsive grid layouts

UI_USAGE_SCRIPT.md

@@ -0,0 +1,715 @@
+# UI Structure Usage Script / Guide
+
+## How to Use the Application's UI Structure
+
+This document provides step-by-step instructions on how to work with the multi-page architecture, shared components, and layout system.
+
+---
+
+## Part 1: Understanding the Structure
+
+### Architecture Overview
+
+The application follows a **modular multi-page architecture**:
+
+1. **Pages** (`/static/pages/`) - Standalone page modules
+2. **Shared Components** (`/static/shared/`) - Reusable layouts, utilities, components
+3. **Layout Manager** - Dynamically injects header, sidebar, footer
+4. **API Client** - Centralized HTTP client with caching
+5. **Component System** - Toast, Modal, Table, Chart, Loading
+
+### Key Principles
+
+- Each page is self-contained but shares common layouts
+- Layouts are injected dynamically (not hardcoded in each page)
+- All API calls go through the centralized API client
+- Components are reusable across pages
+- Theme system (dark/light) is managed globally
+
+---
+
+## Part 2: Creating a New Page
+
+### Step-by-Step Process
+
+#### Step 1: Create Directory Structure
+
+```
+Location: /static/pages/your-page-name/
+Files needed:
+  - index.html (required)
+  - your-page-name.css (optional, for page-specific styles)
+  - your-page-name.js (optional, for page-specific logic)
+```
+
+#### Step 2: Create the HTML Template
+
+**Template Structure:**
+
+```html
+<!DOCTYPE html>
+<html lang="en" data-theme="dark">
+<head>
+  <!-- Meta tags -->
+  <meta charset="UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>Your Page Title | Crypto Hub</title>
+  
+  <!-- Shared CSS (always include these) -->
+  <link rel="stylesheet" href="/static/shared/css/design-system.css">
+  <link rel="stylesheet" href="/static/shared/css/layout.css">
+  <link rel="stylesheet" href="/static/shared/css/components.css">
+  <link rel="stylesheet" href="/static/shared/css/utilities.css">
+  
+  <!-- Page-specific CSS -->
+  <link rel="stylesheet" href="/static/pages/your-page-name/your-page-name.css">
+  
+  <!-- Error suppressor (suppresses browser warnings) -->
+  <script src="/static/shared/js/utils/error-suppressor.js"></script>
+</head>
+<body>
+  <!-- App Container (required structure) -->
+  <div class="app-container">
+    
+    <!-- Sidebar Container (LayoutManager will inject sidebar here) -->
+    <aside id="sidebar-container"></aside>
+    
+    <!-- Main Content Area -->
+    <main class="main-content">
+      
+      <!-- Header Container (LayoutManager will inject header here) -->
+      <header id="header-container"></header>
+      
+      <!-- Your Page Content -->
+      <div class="page-content">
+        <!-- Your page-specific content goes here -->
+        <h1>Your Page Title</h1>
+        <div id="your-content-container">
+          <!-- Content -->
+        </div>
+      </div>
+      
+    </main>
+  </div>
+  
+  <!-- Scripts (at end of body) -->
+  <script type="module">
+    // Import LayoutManager (required)
+    import LayoutManager from '/static/shared/js/core/layout-manager.js';
+    
+    // Import API Client (if you need API calls)
+    import { ApiClient } from '/static/shared/js/core/api-client.js';
+    
+    // Import components you need
+    import { Toast } from '/static/shared/js/components/toast.js';
+    import { Loading } from '/static/shared/js/components/loading.js';
+    
+    // Initialize Layout Manager (this injects header, sidebar, footer)
+    await LayoutManager.init('your-page-name');
+    
+    // Your page initialization code
+    async function initPage() {
+      // Show loading state
+      Loading.show();
+      
+      try {
+        // Fetch data using API client
+        const client = new ApiClient();
+        const data = await client.get('/api/your-endpoint');
+        
+        // Update UI with data
+        updateUI(data);
+        
+        // Show success message
+        Toast.success('Data loaded successfully');
+      } catch (error) {
+        // Show error message
+        Toast.error('Failed to load data: ' + error.message);
+      } finally {
+        // Hide loading state
+        Loading.hide();
+      }
+    }
+    
+    // Call initialization
+    initPage();
+  </script>
+</body>
+</html>
+```
+
+#### Step 3: Register the Page
+
+**A. Add to Sidebar Navigation**
+
+Edit: `/static/shared/layouts/sidebar.html`
+
+Add navigation link:
+
+```html
+<li>
+  <a href="/static/pages/your-page-name/index.html" 
+     class="nav-link" 
+     data-page="your-page-name">
+    <span class="icon">📊</span>
+    <span>Your Page Name</span>
+  </a>
+</li>
+```
+
+**B. Register in Config**
+
+Edit: `/static/shared/js/core/config.js`
+
+Add to `PAGE_METADATA` array:
+
+```javascript
+export const PAGE_METADATA = [
+  // ... existing pages
+  { 
+    page: 'your-page-name', 
+    title: 'Your Page Title | Crypto Hub', 
+    icon: 'star' 
+  }
+];
+```
+
+---
+
+## Part 3: Using Shared Components
+
+### Layout Manager
+
+**Purpose:** Manages shared layouts (header, sidebar, footer)
+
+**Basic Usage:**
+
+```javascript
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+
+// Initialize (injects layouts, sets active page)
+await LayoutManager.init('page-name');
+
+// Set active page in navigation
+LayoutManager.setActivePage('dashboard');
+
+// Update API status badge
+LayoutManager.updateApiStatus('online', '✓ System Active');
+LayoutManager.updateApiStatus('offline', '✗ Offline');
+LayoutManager.updateApiStatus('degraded', '⚠ Degraded');
+
+// Toggle theme
+LayoutManager.toggleTheme();
+
+// Get current theme
+const theme = document.documentElement.getAttribute('data-theme');
+```
+
+### API Client
+
+**Purpose:** Centralized HTTP client with caching and error handling
+
+**Basic Usage:**
+
+```javascript
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+
+const client = new ApiClient();
+
+// GET request with caching
+const data = await client.get('/api/market/top', {
+  cache: true,
+  ttl: 30000 // Cache for 30 seconds
+});
+
+// POST request
+const result = await client.post('/api/sentiment/analyze', {
+  text: 'Bitcoin is bullish!'
+});
+
+// PUT request
+await client.put('/api/settings', { theme: 'dark' });
+
+// DELETE request
+await client.delete('/api/resource/123');
+
+// With error handling
+try {
+  const data = await client.get('/api/endpoint');
+} catch (error) {
+  console.error('API Error:', error);
+  Toast.error('Failed to fetch data');
+}
+```
+
+### Polling Manager
+
+**Purpose:** Auto-refresh data with smart pause/resume
+
+**Usage:**
+
+```javascript
+import { PollingManager } from '/static/shared/js/core/polling-manager.js';
+
+// Create poller
+const poller = new PollingManager({
+  interval: 5000,        // Poll every 5 seconds
+  pauseOnHidden: true,   // Pause when tab is hidden
+  maxRetries: 3          // Max retries on failure
+});
+
+// Start polling
+poller.start(async () => {
+  const client = new ApiClient();
+  const data = await client.get('/api/market/top');
+  updateMarketData(data);
+});
+
+// Stop polling
+poller.stop();
+
+// Pause temporarily
+poller.pause();
+
+// Resume
+poller.resume();
+```
+
+### Toast Notifications
+
+**Purpose:** User feedback messages
+
+**Usage:**
+
+```javascript
+import { Toast } from '/static/shared/js/components/toast.js';
+
+// Success message
+Toast.success('Operation completed successfully');
+
+// Error message
+Toast.error('Failed to save changes');
+
+// Info message
+Toast.info('Processing your request...');
+
+// Warning message
+Toast.warning('Please check your input');
+
+// Custom message
+Toast.show('Custom message', 'info', 5000); // message, type, duration
+```
+
+### Modal Dialogs
+
+**Purpose:** Popup dialogs for confirmations, forms, etc.
+
+**Usage:**
+
+```javascript
+import { Modal } from '/static/shared/js/components/modal.js';
+
+// Simple modal
+const modal = new Modal({
+  title: 'Confirm Action',
+  content: '<p>Are you sure you want to proceed?</p>',
+  buttons: [
+    {
+      text: 'Cancel',
+      class: 'btn-secondary',
+      action: () => modal.close()
+    },
+    {
+      text: 'Confirm',
+      class: 'btn-primary',
+      action: () => {
+        // Perform action
+        performAction();
+        modal.close();
+      }
+    }
+  ]
+});
+modal.show();
+
+// Modal with form
+const formModal = new Modal({
+  title: 'Add Item',
+  content: `
+    <form id="add-form">
+      <input type="text" name="name" placeholder="Name" required>
+      <button type="submit">Add</button>
+    </form>
+  `,
+  onClose: () => console.log('Modal closed')
+});
+formModal.show();
+```
+
+### Loading Component
+
+**Purpose:** Show/hide loading states
+
+**Usage:**
+
+```javascript
+import { Loading } from '/static/shared/js/components/loading.js';
+
+// Show loading overlay
+Loading.show('Loading data...');
+
+// Hide loading
+Loading.hide();
+
+// Show loading in specific container
+Loading.showIn('#my-container', 'Loading...');
+
+// Hide loading in container
+Loading.hideIn('#my-container');
+```
+
+### Table Component
+
+**Purpose:** Data tables with sorting and filtering
+
+**Usage:**
+
+```javascript
+import { DataTable } from '/static/shared/js/components/table.js';
+
+const table = new DataTable('#table-container', {
+  columns: [
+    { key: 'name', label: 'Name', sortable: true },
+    { key: 'price', label: 'Price', sortable: true, formatter: (val) => `$${val}` },
+    { key: 'change', label: 'Change', sortable: true }
+  ],
+  data: marketData,
+  searchable: true,
+  pagination: true,
+  pageSize: 10
+});
+
+// Update data
+table.updateData(newData);
+
+// Refresh
+table.refresh();
+```
+
+### Chart Component
+
+**Purpose:** Chart.js wrapper for data visualization
+
+**Usage:**
+
+```javascript
+import { Chart } from '/static/shared/js/components/chart.js';
+
+const chart = new Chart('#chart-container', {
+  type: 'line',
+  data: {
+    labels: dates,
+    datasets: [{
+      label: 'Price',
+      data: prices,
+      borderColor: '#8B5CF6'
+    }]
+  },
+  options: {
+    responsive: true,
+    maintainAspectRatio: false
+  }
+});
+
+// Update chart data
+chart.updateData(newData);
+```
+
+---
+
+## Part 4: Common Patterns
+
+### Pattern 1: Page with Data Fetching
+
+```javascript
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+import { Toast } from '/static/shared/js/components/toast.js';
+import { Loading } from '/static/shared/js/components/loading.js';
+
+// Initialize
+await LayoutManager.init('my-page');
+
+// Fetch and display data
+async function loadData() {
+  Loading.show('Loading data...');
+  
+  try {
+    const client = new ApiClient();
+    const data = await client.get('/api/endpoint');
+    
+    renderData(data);
+    Toast.success('Data loaded');
+  } catch (error) {
+    Toast.error('Failed to load: ' + error.message);
+  } finally {
+    Loading.hide();
+  }
+}
+
+function renderData(data) {
+  const container = document.getElementById('data-container');
+  container.innerHTML = data.map(item => `
+    <div class="data-item">
+      <h3>${item.name}</h3>
+      <p>${item.description}</p>
+    </div>
+  `).join('');
+}
+
+// Load on page load
+loadData();
+```
+
+### Pattern 2: Page with Auto-Refresh
+
+```javascript
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+import { PollingManager } from '/static/shared/js/core/polling-manager.js';
+
+await LayoutManager.init('my-page');
+
+const client = new ApiClient();
+const poller = new PollingManager({ interval: 10000 });
+
+async function refreshData() {
+  try {
+    const data = await client.get('/api/endpoint', { cache: false });
+    updateUI(data);
+  } catch (error) {
+    console.error('Refresh failed:', error);
+  }
+}
+
+// Start auto-refresh
+poller.start(refreshData);
+
+// Initial load
+refreshData();
+
+// Cleanup on page unload
+window.addEventListener('beforeunload', () => {
+  poller.stop();
+});
+```
+
+### Pattern 3: Form Submission
+
+```javascript
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+import { Toast } from '/static/shared/js/components/toast.js';
+import { Loading } from '/static/shared/js/components/loading.js';
+
+document.getElementById('my-form').addEventListener('submit', async (e) => {
+  e.preventDefault();
+  
+  const formData = new FormData(e.target);
+  const data = Object.fromEntries(formData);
+  
+  Loading.show('Submitting...');
+  
+  try {
+    const client = new ApiClient();
+    const result = await client.post('/api/submit', data);
+    
+    Toast.success('Submitted successfully!');
+    e.target.reset();
+  } catch (error) {
+    Toast.error('Submission failed: ' + error.message);
+  } finally {
+    Loading.hide();
+  }
+});
+```
+
+### Pattern 4: Interactive Table with Actions
+
+```javascript
+import { DataTable } from '/static/shared/js/components/table.js';
+import { Modal } from '/static/shared/js/components/modal.js';
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+
+const table = new DataTable('#table-container', {
+  columns: [
+    { key: 'name', label: 'Name' },
+    { key: 'status', label: 'Status' },
+    { 
+      key: 'actions', 
+      label: 'Actions',
+      render: (row) => `
+        <button onclick="editItem(${row.id})">Edit</button>
+        <button onclick="deleteItem(${row.id})">Delete</button>
+      `
+    }
+  ],
+  data: items
+});
+
+async function deleteItem(id) {
+  const modal = new Modal({
+    title: 'Confirm Delete',
+    content: '<p>Are you sure?</p>',
+    buttons: [
+      { text: 'Cancel', action: () => modal.close() },
+      {
+        text: 'Delete',
+        action: async () => {
+          const client = new ApiClient();
+          await client.delete(`/api/items/${id}`);
+          table.refresh();
+          modal.close();
+        }
+      }
+    ]
+  });
+  modal.show();
+}
+```
+
+---
+
+## Part 5: File Paths Reference
+
+### Absolute Paths (Recommended)
+
+```
+/static/shared/js/core/layout-manager.js
+/static/shared/css/design-system.css
+/static/pages/dashboard/index.html
+```
+
+### Relative Paths (From Page Directory)
+
+```
+../../shared/js/core/layout-manager.js
+../../shared/css/design-system.css
+../dashboard/index.html
+```
+
+### Import Statements
+
+```javascript
+// ES6 Modules (recommended)
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+
+// Dynamic imports
+const { Toast } = await import('/static/shared/js/components/toast.js');
+```
+
+---
+
+## Part 6: Best Practices Checklist
+
+### ✅ Do's
+
+- Always use `LayoutManager.init()` in every page
+- Use `ApiClient` for all API calls (don't use raw `fetch()`)
+- Show loading states with `Loading` component
+- Provide user feedback with `Toast` notifications
+- Handle errors gracefully with try/catch
+- Use shared CSS classes from design system
+- Follow the page template structure
+- Register new pages in config and sidebar
+- Use absolute paths for imports
+- Clean up polling/intervals on page unload
+
+### ❌ Don'ts
+
+- Don't hardcode layouts in pages (use LayoutManager)
+- Don't use raw `fetch()` for API calls
+- Don't create duplicate components (use shared ones)
+- Don't forget error handling
+- Don't use inline styles (use CSS classes)
+- Don't forget to register pages in navigation
+- Don't use relative paths that break on different routes
+- Don't forget to stop polling/intervals
+
+---
+
+## Part 7: Troubleshooting
+
+### Layout Not Showing
+
+- Check that `LayoutManager.init()` is called
+- Verify containers exist: `#sidebar-container`, `#header-container`
+- Check browser console for errors
+- Verify file paths are correct
+
+### API Calls Failing
+
+- Check that `ApiClient` is imported correctly
+- Verify endpoint URLs are correct
+- Check network tab for actual requests
+- Verify CORS settings if calling external APIs
+
+### Components Not Working
+
+- Check that component scripts are imported
+- Verify component initialization code
+- Check browser console for errors
+- Ensure CSS is loaded
+
+### Navigation Not Highlighting
+
+- Verify page name matches `data-page` attribute
+- Check that `LayoutManager.setActivePage()` is called
+- Verify page is registered in `PAGE_METADATA`
+
+---
+
+## Part 8: Quick Reference
+
+### Required Imports for Every Page
+
+```javascript
+import LayoutManager from '/static/shared/js/core/layout-manager.js';
+await LayoutManager.init('page-name');
+```
+
+### Common Component Imports
+
+```javascript
+import { ApiClient } from '/static/shared/js/core/api-client.js';
+import { Toast } from '/static/shared/js/components/toast.js';
+import { Loading } from '/static/shared/js/components/loading.js';
+import { Modal } from '/static/shared/js/components/modal.js';
+import { PollingManager } from '/static/shared/js/core/polling-manager.js';
+```
+
+### Required HTML Structure
+
+```html
+<div class="app-container">
+  <aside id="sidebar-container"></aside>
+  <header id="header-container"></header>
+  <main class="main-content">
+    <div class="page-content">
+      <!-- Your content -->
+    </div>
+  </main>
+</div>
+```
+
+### Required CSS Imports
+
+```html
+<link rel="stylesheet" href="/static/shared/css/design-system.css">
+<link rel="stylesheet" href="/static/shared/css/layout.css">
+<link rel="stylesheet" href="/static/shared/css/components.css">
+```
+
+---
+
+This guide provides the complete framework for working with the UI structure. Follow these patterns and practices to maintain consistency and leverage the shared component system effectively.