Documentation/SCHEMES_API_GUIDE.md

# Scheme Display API - Documentation

## Overview

This document describes the new API endpoints for displaying government schemes on the frontend with filtering, pagination, and search capabilities.

---

## 🎯 New API Endpoints

### 1. **GET /schemes/all** - Get All Schemes with Filtering

Retrieves a paginated list of schemes with optional filtering.

#### Request Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `page` | integer | No | 1 | Page number (starts from 1) |
| `page_size` | integer | No | 10 | Number of schemes per page (max: 100) |
| `state` | string | No | null | Filter by state name |
| `category` | string | No | null | Filter by scheme category |
| `level` | string | No | null | Filter by level (Central/State) |
| `search` | string | No | null | Search text (searches in name, details, benefits, tags) |

#### Response Format

```json
{
  "total": 3402,
  "page": 1,
  "page_size": 10,
  "total_pages": 341,
  "schemes": [
    {
      "scheme_name": "Scheme Name",
      "slug": "scheme-slug",
      "details": "Detailed description...",
      "benefits": "Benefits offered...",
      "eligibility": "Eligibility criteria...",
      "application": "How to apply...",
      "documents": "Required documents...",
      "level": "State/Central",
      "scheme_category": "Category name",
      "tags": "tag1, tag2"
    }
  ]
}
```

#### Example Requests

```bash
# Get first page of all schemes
GET http://127.0.0.1:8000/schemes/all

# Get schemes for specific state
GET http://127.0.0.1:8000/schemes/all?state=Karnataka&page=1&page_size=20

# Filter by category
GET http://127.0.0.1:8000/schemes/all?category=Agriculture

# Filter by level
GET http://127.0.0.1:8000/schemes/all?level=Central

# Search with text
GET http://127.0.0.1:8000/schemes/all?search=farmer

# Combined filters
GET http://127.0.0.1:8000/schemes/all?state=Maharashtra&category=Education&page=1&page_size=15
```

---

### 2. **GET /schemes/{slug}** - Get Scheme Details by Slug

Retrieves detailed information about a specific scheme.

#### Request Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `slug` | string | Yes | - | Unique scheme identifier |
| `language` | string | No | "en" | Language code for translation |

#### Response Format

```json
{
  "scheme_name": "Scheme Name",
  "slug": "scheme-slug",
  "details": "Full details...",
  "benefits": "Complete benefits...",
  "eligibility": "Full eligibility criteria...",
  "application": "Complete application process...",
  "documents": "All required documents...",
  "level": "State/Central",
  "scheme_category": "Category",
  "tags": "tags"
}
```

#### Example Requests

```bash
# Get scheme in English
GET http://127.0.0.1:8000/schemes/astpss

# Get scheme in Hindi
GET http://127.0.0.1:8000/schemes/astpss?language=hi

# Get scheme in Telugu
GET http://127.0.0.1:8000/schemes/pmsfbcs?language=te
```

---

### 3. **GET /schemes/categories** - Get All Categories

Retrieves all unique scheme categories.

#### Response Format

```json
{
  "categories": [
    "Agriculture,Rural & Environment",
    "Business & Entrepreneurship",
    "Education & Learning",
    "Health & Wellness",
    "Social welfare & Empowerment",
    "Women and Child"
  ]
}
```

#### Example Request

```bash
GET http://127.0.0.1:8000/schemes/categories
```

---

### 4. **GET /schemes/stats** - Get Scheme Statistics

Retrieves statistics about all schemes.

#### Response Format

```json
{
  "total_schemes": 3402,
  "by_level": {
    "State": 2850,
    "Central": 552
  },
  "by_category": {
    "Social welfare & Empowerment": 856,
    "Business & Entrepreneurship": 642,
    "Education & Learning": 432,
    "Agriculture,Rural & Environment": 389,
    "Health & Wellness": 267
  },
  "total_categories": 24
}
```

#### Example Request

```bash
GET http://127.0.0.1:8000/schemes/stats
```

---

## 🎨 Frontend Implementation Examples

### React/Next.js Example

```javascript
// Fetch schemes with filters
const fetchSchemes = async (filters) => {
  const params = new URLSearchParams({
    page: filters.page || 1,
    page_size: filters.pageSize || 10,
    ...(filters.state && { state: filters.state }),
    ...(filters.category && { category: filters.category }),
    ...(filters.level && { level: filters.level }),
    ...(filters.search && { search: filters.search })
  });
  
  const response = await fetch(
    `http://127.0.0.1:8000/schemes/all?${params}`
  );
  return await response.json();
};

// Fetch single scheme
const fetchScheme = async (slug, language = 'en') => {
  const response = await fetch(
    `http://127.0.0.1:8000/schemes/${slug}?language=${language}`
  );
  return await response.json();
};

// Usage in component
const SchemesList = () => {
  const [schemes, setSchemes] = useState([]);
  const [filters, setFilters] = useState({
    page: 1,
    pageSize: 10,
    state: '',
    category: '',
    search: ''
  });

  useEffect(() => {
    fetchSchemes(filters).then(data => {
      setSchemes(data.schemes);
    });
  }, [filters]);

  return (
    <div>
      {/* Filter UI */}
      {/* Scheme cards */}
    </div>
  );
};
```

---

## 🔍 Search & Filter Recommendations

### **RECOMMENDATION: Implement Filters on BACKEND**

#### ✅ Why Backend Filtering is Better:

1. **Performance**
   - Only sends required data over network
   - Reduces bandwidth usage
   - Faster page loads (10 schemes vs 3402 schemes)

2. **Scalability**
   - Database can grow without affecting frontend performance
   - Efficient pagination
   - Optimized queries

3. **Consistency**
   - Same filtering logic across all platforms (web, mobile, etc.)
   - Centralized business logic
   - Easier to maintain

4. **User Experience**
   - Faster search results
   - Real-time filtering without loading all data
   - Better mobile experience

5. **SEO Benefits**
   - Server-side rendering possible
   - Better indexing of individual scheme pages
   - Faster initial page load

#### ❌ Why Avoid Frontend Filtering:

1. **Performance Issues**
   - Loading 3402+ schemes slows down initial page load
   - Memory consumption on client devices
   - Poor mobile performance

2. **Network Overhead**
   - Large JSON payload on every page load
   - Wasted bandwidth for unused data

3. **Maintenance**
   - Duplicate filtering logic on frontend and backend
   - Harder to debug issues

---

## 📊 Backend Filtering Implementation

### Current Implementation Features:

#### 1. **State Filtering**
```python
# Searches in details and eligibility fields
if state and state != "All States":
    filtered_df = filtered_df[
        filtered_df['details'].str.contains(state, case=False, na=False) |
        filtered_df['eligibility'].str.contains(state, case=False, na=False)
    ]
```

#### 2. **Category Filtering**
```python
# Case-insensitive partial matching
if category:
    filtered_df = filtered_df[
        filtered_df['schemeCategory'].str.contains(category, case=False, na=False)
    ]
```

#### 3. **Level Filtering**
```python
# Exact match for Central/State
if level:
    filtered_df = filtered_df[
        filtered_df['level'].str.lower() == level.lower()
    ]
```

#### 4. **Text Search**
```python
# Searches across multiple fields
if search:
    search_mask = (
        filtered_df['scheme_name'].str.contains(search, case=False, na=False) |
        filtered_df['details'].str.contains(search, case=False, na=False) |
        filtered_df['benefits'].str.contains(search, case=False, na=False) |
        filtered_df['tags'].str.contains(search, case=False, na=False)
    )
```

#### 5. **Pagination**
```python
# Efficient slicing
start_idx = (page - 1) * page_size
end_idx = start_idx + page_size
paginated_df = filtered_df.iloc[start_idx:end_idx]
```

---

## 🎯 Recommended Frontend Architecture

### Option 1: Server-Side Rendering (SSR) - **Best for SEO**

```javascript
// Next.js Example
export async function getServerSideProps(context) {
  const { page, state, category } = context.query;
  
  const response = await fetch(
    `http://127.0.0.1:8000/schemes/all?page=${page || 1}&state=${state || ''}&category=${category || ''}`
  );
  const data = await response.json();
  
  return {
    props: { schemes: data }
  };
}
```

### Option 2: Client-Side with Debouncing - **Good UX**

```javascript
// React Hook for debounced search
import { useState, useEffect } from 'react';
import { debounce } from 'lodash';

const useSchemeSearch = () => {
  const [filters, setFilters] = useState({});
  const [schemes, setSchemes] = useState([]);
  const [loading, setLoading] = useState(false);

  const debouncedSearch = debounce(async (filters) => {
    setLoading(true);
    const data = await fetchSchemes(filters);
    setSchemes(data.schemes);
    setLoading(false);
  }, 500); // Wait 500ms after user stops typing

  useEffect(() => {
    debouncedSearch(filters);
  }, [filters]);

  return { schemes, loading, setFilters };
};
```

### Option 3: Hybrid Approach - **Balanced**

```javascript
// Initial load: Server-side
// Filters: Client-side API calls
// Search: Debounced backend calls

const SchemeList = ({ initialSchemes }) => {
  const [schemes, setSchemes] = useState(initialSchemes);
  
  const handleFilterChange = async (newFilters) => {
    const data = await fetchSchemes(newFilters);
    setSchemes(data.schemes);
  };
  
  return (
    <div>
      <Filters onChange={handleFilterChange} />
      <SchemeCards schemes={schemes} />
    </div>
  );
};
```

---

## 🚀 Performance Optimization Tips

### 1. **Caching**

```javascript
// Frontend caching with React Query
import { useQuery } from '@tanstack/react-query';

const useSchemes = (filters) => {
  return useQuery({
    queryKey: ['schemes', filters],
    queryFn: () => fetchSchemes(filters),
    staleTime: 5 * 60 * 1000, // Cache for 5 minutes
  });
};
```

### 2. **Lazy Loading**

```javascript
// Load more schemes on scroll
const SchemeList = () => {
  const [page, setPage] = useState(1);
  
  const handleScroll = () => {
    if (/* bottom of page */) {
      setPage(prev => prev + 1);
    }
  };
  
  // Fetch next page and append
};
```

### 3. **Virtual Scrolling**

```javascript
// Use react-window for large lists
import { FixedSizeList } from 'react-window';

<FixedSizeList
  height={600}
  itemCount={schemes.length}
  itemSize={120}
>
  {SchemeCard}
</FixedSizeList>
```

---

## 📱 Mobile Optimization

### Recommended Approach for Mobile:

1. **Smaller Page Size**: Use `page_size=5` or `page_size=10` on mobile
2. **Lazy Loading**: Load more on scroll instead of pagination buttons
3. **Simplified Filters**: Show essential filters only
4. **Backend Filtering**: Critical for mobile performance

```javascript
// Detect device and adjust
const isMobile = window.innerWidth < 768;
const pageSize = isMobile ? 5 : 10;

fetchSchemes({ page: 1, page_size: pageSize });
```

---

## 🔐 Error Handling

### Backend Errors

```javascript
const fetchSchemes = async (filters) => {
  try {
    const response = await fetch(`/schemes/all?${params}`);
    
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    
    return await response.json();
  } catch (error) {
    console.error('Failed to fetch schemes:', error);
    // Show error toast/message to user
    return { schemes: [], total: 0 };
  }
};
```

---

## 📈 Testing the API

### Using cURL

```bash
# Get all schemes
curl http://127.0.0.1:8000/schemes/all

# Filter by state
curl "http://127.0.0.1:8000/schemes/all?state=Karnataka"

# Search
curl "http://127.0.0.1:8000/schemes/all?search=farmer"

# Get specific scheme
curl http://127.0.0.1:8000/schemes/astpss

# Get statistics
curl http://127.0.0.1:8000/schemes/stats
```

### Using Python

```python
import requests

# Get schemes
response = requests.get(
    'http://127.0.0.1:8000/schemes/all',
    params={
        'page': 1,
        'page_size': 10,
        'state': 'Karnataka',
        'category': 'Agriculture'
    }
)
data = response.json()
print(f"Found {data['total']} schemes")
```

---

## 📝 Summary

### ✅ What We Built:

1. **4 new API endpoints** for scheme management
2. **Backend filtering** (state, category, level, search)
3. **Pagination** support
4. **Multi-language** scheme details
5. **Statistics** endpoint

### ✅ Recommendations:

1. **Use BACKEND filtering** - Better performance, scalability, UX
2. **Implement pagination** - Don't load all 3402 schemes at once
3. **Add debouncing** - For search input (500ms delay)
4. **Cache results** - Use React Query or SWR
5. **Lazy load** - For mobile and infinite scroll

### 🎯 Best Practice:

**Backend handles filtering → Frontend displays results → User gets fast, smooth experience**

---

## 🆘 Support

For API documentation and interactive testing:
- Swagger UI: http://127.0.0.1:8000/docs
- ReDoc: http://127.0.0.1:8000/redoc