Documentation/API_DOCUMENTATION.md

# Government Schemes RAG API Documentation (Multilingual)

## Overview
FastAPI-based REST API for querying Indian Government Schemes using Retrieval-Augmented Generation (RAG) with **support for 13+ Indian languages**.

## Base URL
```

http://127.0.0.1:8000

```

## Key Features
- ✅ Multilingual support (13+ Indian languages)
- ✅ Automatic translation (Input & Output)
- ✅ Text-to-Speech capability (optional)
- ✅ RAG-powered intelligent search
- ✅ 3400+ government schemes database

## API Endpoints

### 1. Root Endpoint
**GET /** 

Returns API information, version, and supported languages.

**Response:**
```json

{

  "message": "Government Schemes RAG API with Multilingual Support",

  "version": "2.0.0",

  "supported_languages": {

    "en": "English",

    "hi": "Hindi",

    "te": "Telugu",

    "ta": "Tamil",

    "ml": "Malayalam",

    "kn": "Kannada",

    "bn": "Bengali",

    "mr": "Marathi",

    "gu": "Gujarati",

    "pa": "Punjabi",

    "ur": "Urdu",

    "or": "Odia",

    "as": "Assamese"

  },

  "endpoints": {

    "POST /query": "Query government schemes with translation support",

    "GET /states": "Get list of Indian states",

    "GET /languages": "Get list of supported languages",

    "GET /health": "Health check"

  }

}

```

---

### 2. Health Check
**GET /health**

Check if the API and RAG system are running properly.

**Response:**
```json

{

  "status": "healthy",

  "rag_system": "initialized"

}

```

---

### 3. Get Supported Languages
**GET /languages**

Get list of all supported languages for translation.

**Response:**
```json

{

  "languages": {

    "en": "English",

    "hi": "Hindi",

    "te": "Telugu",

    "ta": "Tamil",

    "ml": "Malayalam",

    "kn": "Kannada",

    "bn": "Bengali",

    "mr": "Marathi",

    "gu": "Gujarati",

    "pa": "Punjabi",

    "ur": "Urdu",

    "or": "Odia",

    "as": "Assamese"

  }

}

```

---

### 4. Get States
**GET /states**

Get list of all Indian states and union territories.

**Response:**
```json

{

  "states": [

    "All States",

    "Andhra Pradesh",

    "Arunachal Pradesh",

    ...

  ]

}

```

---

### 5. Query Schemes (with Multilingual Support)
**POST /query**

Query government schemes in any supported language. The API automatically translates the input to English, processes it through the RAG system, and returns the answer in the requested language.

**Request Body:**
```json

{

  "question": "స్కాలర్‌షిప్ల గురించి చెప్పండి",  // Question in any language

  "state": "Telangana",  // Optional

  "language": "te"  // Language code (default: "en")

}

```

**Response:**
```json

{

  "answer": "తెలంగాణలో అందుబాటులో ఉన్న స్కాలర్‌షిప్‌ల గురించి...",

  "sources": [

    "Scheme Name: Pre-Matric Scholarship for Backward Class Students...",

    "Scheme Name: Post-Matric Scholarship Scheme...",

    "Scheme Name: Merit-cum-Means Scholarship..."

  ]

}

```

**Note:** Audio is NOT automatically generated. Use the `/generate-audio` endpoint when the user clicks the speaker button.

**Translation Flow:**
```

User Question (Telugu) → Translate to English → RAG Processing → 

English Answer → Translate to Telugu → Return to User

```

---

### 6. Generate Audio (On-Demand)
**POST /generate-audio**

Generate audio from text. This endpoint should be called ONLY when the user clicks the "Play Audio" or speaker button on the UI.

**Request Body:**
```json

{

  "text": "తెలంగాణలో అందుబాటులో ఉన్న స్కాలర్‌షిప్‌ల గురించి...",

  "language": "te"  // Language code (default: "en")

}

```

**Response:**
```json

{

  "audio": "base64_encoded_mp3_audio_data"

}

```

**Usage Flow:**
```

1. User submits question → Receive answer (fast, no audio)

2. User clicks speaker button → Call /generate-audio → Play audio

```

**Error Response (400 - Empty Text):**
```json

{

  "detail": "Text cannot be empty"

}

```

**Error Response (400 - Unsupported Language):**
```json

{

  "detail": "Unsupported language. Supported: ['en', 'hi', 'te', 'ta', ...]"

}

```

**Error Response (400 - Empty Question):**
```json

{

  "detail": "Question cannot be empty"

}

```

**Error Response (400 - Unsupported Language):**
```json

{

  "detail": "Unsupported language. Supported: ['en', 'hi', 'te', 'ta', ...]"

}

```

**Error Response (500):**
```json

{

  "detail": "Error processing query: [error message]"

}

```

---

## Interactive API Documentation

FastAPI automatically generates interactive API documentation:

- **Swagger UI**: http://127.0.0.1:8000/docs
- **ReDoc**: http://127.0.0.1:8000/redoc

These interfaces allow you to:
- View all endpoints
- See request/response schemas
- Test API calls directly from the browser
- Download OpenAPI specification

---

## Usage Examples

### Using cURL

```bash

# Health check

curl http://127.0.0.1:8000/health



# Get supported languages

curl http://127.0.0.1:8000/languages



# Get states

curl http://127.0.0.1:8000/states



# Query in English

curl -X POST http://127.0.0.1:8000/query \

  -H "Content-Type: application/json" \

  -d '{

    "question": "What scholarships are available for SC students?",

    "state": "Karnataka",

    "language": "en"

  }'



# Query in Hindi

curl -X POST http://127.0.0.1:8000/query \

  -H "Content-Type: application/json" \

  -d '{

    "question": "छात्रवृत्ति के बारे में बताएं",

    "language": "hi"

  }'



# Query in Telugu

curl -X POST http://127.0.0.1:8000/query \

  -H "Content-Type: application/json" \

  -d '{

    "question": "స్కాలర్‌షిప్ల గురించి చెప్పండి",

    "state": "Telangana",

    "language": "te"

  }'



# Generate audio (when user clicks speaker button)

curl -X POST http://127.0.0.1:8000/generate-audio \

  -H "Content-Type: application/json" \

  -d '{

    "text": "తెలంగాణలో అందుబాటులో ఉన్న స్కాలర్‌షిప్‌లు...",

    "language": "te"

  }'

```

### Using Python requests

```python

import requests



# Query in English

response = requests.post(

    "http://127.0.0.1:8000/query",

    json={

        "question": "My daughter is studying in 9th standard. What schemes are applicable?",

        "state": "Maharashtra",

        "language": "en"

    }

)



data = response.json()

print(data["answer"])



# Query in Hindi

response_hindi = requests.post(

    "http://127.0.0.1:8000/query",

    json={

        "question": "मुझे छात्रवृत्ति चाहिए",

        "language": "hi"

    }

)



hindi_data = response_hindi.json()

print(hindi_data["answer"])  # Answer will be in Hindi



# Generate audio on-demand (when user clicks speaker button)

audio_response = requests.post(

    "http://127.0.0.1:8000/generate-audio",

    json={

        "text": hindi_data["answer"],

        "language": "hi"

    }

)



audio_data = audio_response.json()

# audio_data["audio"] contains base64 encoded MP3

```

### Using JavaScript fetch

```javascript

// Query in English

const response = await fetch('http://127.0.0.1:8000/query', {

  method: 'POST',

  headers: {

    'Content-Type': 'application/json',

  },

  body: JSON.stringify({

    question: 'What schemes are available for girl child education?',

    state: 'All States',

    language: 'en'

  })

});



const data = await response.json();

console.log(data.answer);



// Query in Telugu

const responseTelugu = await fetch('http://127.0.0.1:8000/query', {

  method: 'POST',

  headers: {

    'Content-Type': 'application/json',

  },

  body: JSON.stringify({

    question: 'బాలికల విద్య కోసం ఏ పథకాలు ఉన్నాయి?',

    language: 'te'

  })

});



const teluguData = await responseTelugu.json();

console.log(teluguData.answer);  // Answer in Telugu



// Generate audio when user clicks speaker button

const playAudio = async (text, language) => {

  const audioResponse = await fetch('http://127.0.0.1:8000/generate-audio', {

    method: 'POST',

    headers: {

      'Content-Type': 'application/json',

    },

    body: JSON.stringify({

      text: text,

      language: language

    })

  });

  

  const audioData = await audioResponse.json();

  const audio = new Audio(`data:audio/mp3;base64,${audioData.audio}`);

  audio.play();

};



// Usage: Call when user clicks speaker button

// playAudio(teluguData.answer, 'te');

```

### Using React (Frontend Integration)

```jsx

import React, { useState } from 'react';



function SchemeQuery() {

  const [language, setLanguage] = useState('en');

  const [question, setQuestion] = useState('');

  const [answer, setAnswer] = useState('');

  const [audioLoading, setAudioLoading] = useState(false);



  const handleSubmit = async (e) => {

    e.preventDefault();

    

    const response = await fetch('http://127.0.0.1:8000/query', {

      method: 'POST',

      headers: { 'Content-Type': 'application/json' },

      body: JSON.stringify({

        question: question,

        language: language

      })

    });

    

    const data = await response.json();

    setAnswer(data.answer);

  };



  // Called only when user clicks speaker button

  const playAudio = async () => {

    if (!answer) return;

    

    setAudioLoading(true);

    try {

      const response = await fetch('http://127.0.0.1:8000/generate-audio', {

        method: 'POST',

        headers: { 'Content-Type': 'application/json' },

        body: JSON.stringify({

          text: answer,

          language: language

        })

      });

      

      const data = await response.json();

      const audio = new Audio(`data:audio/mp3;base64,${data.audio}`);

      audio.play();

    } catch (error) {

      console.error('Audio generation failed:', error);

    } finally {

      setAudioLoading(false);

    }

  };



  return (

    <div>

      <select value={language} onChange={e => setLanguage(e.target.value)}>

        <option value="en">English</option>

        <option value="hi">Hindi</option>

        <option value="te">Telugu</option>

        <option value="ta">Tamil</option>

      </select>

      

      <form onSubmit={handleSubmit}>

        <input 

          value={question}

          onChange={e => setQuestion(e.target.value)}

          placeholder="Ask your question..."

        />

        <button type="submit">Ask</button>

      </form>

      

      {answer && (

        <div>

          <p>{answer}</p>

          <button onClick={playAudio} disabled={audioLoading}>

            {audioLoading ? '⏳ Generating...' : '🔊 Play Audio'}

          </button>

        </div>

      )}

    </div>

  );

}

```
    

    const response = await fetch('http://127.0.0.1:8000/query', {

      method: 'POST',

      headers: { 'Content-Type': 'application/json' },

      body: JSON.stringify({

        question: question,

        language: language

      })

    });

    

    const data = await response.json();

    setAnswer(data.answer);

  };


  return (
    <div>

      <select value={language} onChange={e => setLanguage(e.target.value)}>

        <option value="en">English</option>

        <option value="hi">Hindi</option>

        <option value="te">Telugu</option>

        <option value="ta">Tamil</option>

      </select>

      

      <form onSubmit={handleSubmit}>

        <input 

          value={question}

          onChange={e => setQuestion(e.target.value)}

          placeholder="Ask your question..."

        />

        <button type="submit">Ask</button>

      </form>

      

      {answer && <div>{answer}</div>}

    </div>

  );

}

```


### Using Postman

1. **Method**: POST
2. **URL**: `http://127.0.0.1:8000/query`
3. **Headers**: 
   - `Content-Type: application/json`
4. **Body** (raw JSON):

**English:**
```json

{

  "question": "What are the schemes for construction workers?",

  "state": "Karnataka",

  "language": "en"

}

```

**Hindi:**
```json

{

  "question": "निर्माण श्रमिकों के लिए क्या योजनाएं हैं?",

  "language": "hi"

}

```

**Telugu:**
```json

{

  "question": "నిర్మాణ కార్మికులకు ఏ పథకాలు ఉన్నాయి?",

  "state": "Telangana",

  "language": "te"

}

```

---

## Running the API

### Start the Server

```bash

# Activate virtual environment

.venv\Scripts\activate



# Run the API

python app.py

```

The API will start on `http://0.0.0.0:8000`

### Testing the API

Run the test script:
```bash

python test_api.py

```

---

## CORS Configuration

The API is configured to accept requests from any origin (`allow_origins=["*"]`).

⚠️ **For production**, update the CORS settings in `app.py`:

```python

app.add_middleware(

    CORSMiddleware,

    allow_origins=["https://yourdomain.com"],  # Specify allowed origins

    allow_credentials=True,

    allow_methods=["*"],

    allow_headers=["*"],

)

```

---

## Data Source

The API uses `updated_data.csv` containing **3400+ government schemes** across categories:
- Education & Learning
- Social Welfare & Empowerment
- Health & Wellness
- Business & Entrepreneurship
- Women and Child
- And more...

---

## Technology Stack

- **Framework**: FastAPI 0.104.1
- **LLM**: Groq API (llama-3.3-70b-versatile)
- **Embeddings**: HuggingFace sentence-transformers/all-MiniLM-L6-v2
- **Vector DB**: ChromaDB
- **RAG Framework**: LangChain 0.1.0
- **Translation**: deep-translator 1.11.4 (Google Translate)
- **Text-to-Speech**: gTTS 2.5.0 (Google Text-to-Speech)
- **Server**: Uvicorn

---

## Multilingual Features

### Translation Process

1. **Input Translation**: User's question in any Indian language → English
2. **RAG Processing**: English query → Vector search → LLM inference → English answer
3. **Output Translation**: English answer → User's selected language

### Supported Language Codes

| Code | Language | Code | Language |
|------|----------|------|----------|
| `en` | English | `ml` | Malayalam |
| `hi` | Hindi | `kn` | Kannada |
| `te` | Telugu | `bn` | Bengali |
| `ta` | Tamil | `mr` | Marathi |
| `gu` | Gujarati | `pa` | Punjabi |
| `ur` | Urdu | `or` | Odia |
| `as` | Assamese | | |

### Text-to-Speech (Optional)

To enable audio responses, uncomment the following lines in `app.py`:

```python

# Line ~280 in app.py

audio_base64 = TranslationService.text_to_speech(final_answer, request.language)

```

When enabled, the API will return base64-encoded MP3 audio in the `audio` field.

---

## Testing the Multilingual API

### Using the Test Script

```bash

# Make sure the server is running first

python app.py



# In another terminal

python test_translation.py

```

The test script will:
1. Verify language endpoint
2. Test queries in English, Hindi, Telugu, Tamil, and Malayalam
3. Display translated responses

### Manual Testing Checklist

- [ ] Test each supported language
- [ ] Verify translations are accurate
- [ ] Check source citations are included
- [ ] Test with state filters
- [ ] Test error handling (empty questions, invalid languages)
- [ ] Verify CORS headers for frontend integration

---

## Rate Limits

Currently, there are no rate limits implemented. The API uses Groq's free tier which has its own rate limits.

For production deployment, consider implementing:
- Request rate limiting
- Authentication/API keys
- Caching for common queries

---

## Error Handling

- **400 Bad Request**: Invalid or empty question
- **500 Internal Server Error**: Processing error (check GROQ_API_KEY)

---

## Performance Notes

- First query may take 3-5 seconds (vector search + LLM inference)
- Subsequent queries are faster (~1-2 seconds)
- ChromaDB is persisted to disk (./chroma_db/) for faster restarts

- 3400 schemes are chunked into ~12,000-15,000 text segments



---



## Deployment



### Local Development

```bash

uvicorn app:app --reload --host 0.0.0.0 --port 8000

```



### Production (with Gunicorn)

```bash

pip install gunicorn

gunicorn app:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000

```



### Docker (Optional)

Create a `Dockerfile`:

```dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

```



---



## Support



For issues or questions:

1. Check API docs at `/docs`

2. Review logs in terminal

3. Verify `.env` file has valid `GROQ_API_KEY`

4. Ensure `updated_data.csv` is present

---

## Example Queries (Multilingual)

### English
- "My daughter is studying in 9th standard. What schemes are applicable to her?"
- "What scholarships are available for SC/ST students?"
- "What are the schemes for construction workers?"
- "Tell me about Beti Bachao Beti Padhao scheme"

### Hindi (हिंदी)
- "मेरी बेटी 9वीं कक्षा में पढ़ती है। उसके लिए कौन सी योजनाएं हैं?"
- "SC/ST छात्रों के लिए क्या छात्रवृत्ति उपलब्ध है?"
- "निर्माण श्रमिकों के लिए क्या योजनाएं हैं?"
- "बेटी बचाओ बेटी पढ़ाओ योजना के बारे में बताएं"

### Telugu (తెలుగు)
- "నా కూతురు 9వ తరగతి చదువుతోంది. ఆమెకు ఏ పథకాలు వర్తిస్తాయి?"
- "SC/ST విద్యార్థులకు ఏ స్కాలర్‌షిప్‌లు అందుబాటులో ఉన్నాయి?"
- "నిర్మాణ కార్మికులకు ఏ పథకాలు ఉన్నాయి?"

### Tamil (தமிழ்)
- "என் மகள் 9வது வகுப்பு படிக்கிறாள். அவளுக்கு என்ன திட்டங்கள் பொருந்தும்?"
- "SC/ST மாணவர்களுக்கு என்ன உதவித்தொகை கிடைக்கும்?"

### Malayalam (മലയാളം)
- "എന്റെ മകൾ 9-ാം ക്ലാസിൽ പഠിക്കുന്നു. അവൾക്ക് എന്തെല്ലാം പദ്ധതികൾ ബാധകമാണ്?"
- "SC/ST വിദ്യാർത്ഥികൾക്ക് എന്ത് സ്കോളർഷിപ്പുകൾ ലഭ്യമാണ്?"

---

## Frontend Integration Guide

For detailed React integration instructions, see: **MULTILINGUAL_INTEGRATION_GUIDE.md**

Key points for frontend developers:
1. Always send `language` parameter with queries
2. Backend handles ALL translation - no frontend translation needed
3. Use Web Speech API for voice input (browser native)
4. Use Speech Synthesis API for voice output (browser native)
5. Display loading states during translation/query processing

---

## Performance & Optimization

### Response Times
- **Translation**: ~0.5-1 second per translation
- **RAG Query**: ~2-3 seconds
- **Total**: ~3-5 seconds for multilingual queries
- **English-only**: ~2-3 seconds (no translation overhead)

### Optimization Tips
1. **Cache translations** for common queries
2. **Lazy load audio** - only generate when user clicks "Play"
3. **Use connection pooling** for API calls
4. **Implement request debouncing** in frontend
5. **Add response caching** for identical queries

### Scaling Considerations
- Translation uses free Google Translate API (via deep-translator)
- No rate limits on translation service currently
- Groq API has free tier limits (check console.groq.com)
- Consider premium APIs for production (Azure Translator, Google Cloud Translation)