Hugging Face's logo

Abhisingh-18
/
Rag-Model

Model card Files
xet
 Community
Rag-Model / HOW_IT_WORKS.md
Abhisingh-18's picture
Abhisingh-18
Mirror of github.com/Abhisingh18/Rag-Model
9ce3455 verified
preview code
|
Raw
History Blame Contribute Delete
12.7 kB

🎯 RAG Document Analysis System - Complete Working Guide

📋 System Overview

Yeh ek AI-powered Training Data Analysis System hai jo disaster management training data ko analyze karta hai aur professional reports generate karta hai.

🏗️ Architecture (Kaise Kaam Karta Hai)

1. Frontend (React + Vite)

Location: frontend/ folder

Main Components:

  • App.jsx - Main application, routing handle karta hai
  • FileUpload.jsx - File upload interface (drag & drop)
  • AnalysisDashboard.jsx - Analysis results dikhata hai
  • ReportPreview.jsx - PDF report preview

Flow:

User uploads CSV/Excel → FileUpload component → Backend API → Analysis → Dashboard shows results

2. Backend (Node.js + Express)

Location: backend/ folder

Main Files:

server.js

  • Express server start karta hai (Port 5000)
  • MongoDB connection (optional)
  • Routes setup karta hai
  • CORS enable karta hai

routes/upload.js

Upload aur analysis ka main logic:

1. File upload (Multer)
2. File parse (CSV/Excel → Text)
3. AI Analysis (Local rule-based)
4. PDF Report generation
5. Response send to frontend

services/documentParser.js

Files ko parse karta hai:

  • CSV files → csv-parser use karke
  • Excel files → xlsx library use karke
  • PDF files → pdf-parse use karke

services/analysisService.js

LOCAL AI ANALYSIS (No external API needed!):

analyzeTrainingData(rawData) {
    // 1. Parse CSV data
    // 2. Calculate metrics:
    //    - Total trainings
    //    - Total participants
    //    - State-wise coverage
    //    - Theme distribution
    //    - Completion rates
    
    // 3. Generate insights:
    //    - Top performing states
    //    - Popular themes
    //    - Coverage gaps
    
    // 4. Gap Analysis:
    //    - Underserved states
    //    - Missing themes
    //    - Critical gaps
    
    // 5. Recommendations:
    //    - Based on data patterns
    //    - Actionable suggestions
}

Important: Yeh 100% FREE hai - koi external AI API nahi use hota!

services/reportGenerator.js

Professional PDF report banata hai:

generateReport() {
    1. Cover Page (Government format)
    2. Executive Summary
    3. Key Metrics (5 cards)
    4. Detailed Analysis (charts data)
    5. Gap Analysis
    6. Business Insights
    7. Recommendations
    8. Footer
}

Uses PDFKit library for PDF generation.

🔄 Complete Data Flow

Step 1: File Upload 

User → Drag CSV file → FileUpload component
     → Axios POST to /api/upload
     → Multer saves file to backend/uploads/

Step 2: File Parsing 

Backend → documentParser.js
        → Reads CSV/Excel
        → Converts to text format
        → Returns structured data

Step 3: AI Analysis (LOCAL) 

analysisService.js:
1. Parse rows from CSV
2. Count trainings, participants
3. Group by state, theme
4. Calculate percentages
5. Identify gaps (states with < 5 trainings)
6. Generate insights from patterns
7. Create recommendations

Example Analysis Logic:

// Count trainings per state
const stateWiseCoverage = {};
rows.forEach(row => {
    const state = row.State;
    stateWiseCoverage[state] = (stateWiseCoverage[state] || 0) + 1;
});

// Find underserved states
const underservedStates = Object.entries(stateWiseCoverage)
    .filter(([state, count]) => count < 5)
    .map(([state]) => state);

Step 4: PDF Generation 

reportGenerator.js:
→ Creates PDFDocument
→ Adds sections with colors, boxes
→ Saves to backend/reports/
→ Returns file path

Step 5: Response to Frontend 

{
    "success": true,
    "analysis": {
        "totalTrainings": 45,
        "totalParticipants": 1250,
        "stateWiseCoverage": {...},
        "themeDistribution": {...},
        "keyInsights": [...],
        "recommendations": [...],
        "gapAnalysis": {...}
    },
    "executiveSummary": "...",
    "reportUrl": "/reports/NDMA_Training_Report_123456.pdf"
}

Step 6: Dashboard Display 

AnalysisDashboard.jsx:
→ Shows 4 stat cards (top)
→ Shows charts (left side)
→ Shows business insights (right sidebar)
→ Shows recommendations (full width bottom)

📊 Dashboard Layout 

┌─────────────────────────────────────────────────┐
│  [Trainings] [Participants] [Rate] [States]    │ ← Stats (horizontal)
├──────────────────────────┬──────────────────────┤
│  📊 Theme Chart          │  💡 Business         │
│  📍 State Chart          │     Insights         │
│  ⚠️ Gap Analysis         │  (Sidebar)           │
├──────────────────────────┴──────────────────────┤
│  ✅ Recommendations (Full Width - 3 columns)    │
└─────────────────────────────────────────────────┘

🎨 UI Features

Premium Design:

  • Whitish gradient background
  • Glass morphism cards
  • Hover animations
  • Color-coded sections
  • Professional typography (Inter font)

Interactive Elements:

  • Hover effects on cards
  • Smooth transitions
  • Responsive layout
  • Clean spacing

📄 PDF Report Structure 

1. Cover Page
   - Government header
   - Title in colored box
   - Report details
   - Confidentiality notice

2. Executive Summary
   - Blue background box
   - Summary text

3. Key Metrics (5 cards)
   - 🎯 Total Trainings
   - 👥 Participants
   - 📈 Completion Rate
   - 📍 States
   - 🎓 Themes

4. Detailed Analysis
   - Theme distribution list
   - Top 10 states list

5. Gap Analysis
   - Underserved states
   - Critical gaps

6. Business Insights
   - Numbered insights

7. Recommendations
   - Strategic actions

8. Footer
   - Copyright notice

🔑 Key Technologies

Frontend:

  • React 18
  • Vite (build tool)
  • Axios (API calls)
  • Recharts (charts)
  • React Dropzone (file upload)
  • Lucide React (icons)

Backend:

  • Node.js
  • Express.js
  • Multer (file upload)
  • csv-parser (CSV parsing)
  • xlsx (Excel parsing)
  • PDFKit (PDF generation)
  • Mongoose (MongoDB - optional)

💾 Data Storage

Files:

  • Uploads: backend/uploads/ (temporary)
  • Reports: backend/reports/ (generated PDFs)

MongoDB (Optional): 

Document Schema:
{
    originalName: String,
    fileType: String,
    uploadDate: Date,
    extractedText: String,
    analysis: Object,
    executiveSummary: String,
    reportUrl: String
}

Note: System works WITHOUT MongoDB too!

🚀 How to Use

1. Start Servers: 

# Terminal 1 - Backend
cd backend
npm run dev

# Terminal 2 - Frontend
cd frontend
npm run dev

2. Open Browser: 

http://localhost:5173

3. Upload File:

  • Click or drag CSV/Excel file
  • Supported: .csv, .xlsx, .xls
  • Max size: 10MB

4. View Results:

  • Dashboard shows automatically
  • Charts, insights, recommendations
  • Download PDF report

📁 File Structure 

Rag Model/
├── backend/
│   ├── models/
│   │   └── Document.js          # MongoDB schema
│   ├── routes/
│   │   ├── upload.js            # Upload & analysis routes
│   │   └── analysis.js          # Additional routes
│   ├── services/
│   │   ├── documentParser.js    # File parsing
│   │   ├── analysisService.js   # AI analysis (LOCAL)
│   │   └── reportGenerator.js   # PDF generation
│   ├── uploads/                 # Uploaded files
│   ├── reports/                 # Generated PDFs
│   ├── server.js                # Main server
│   ├── package.json
│   └── .env                     # Environment variables
│
├── frontend/
│   ├── src/
│   │   ├── components/
│   │   │   ├── FileUpload.jsx
│   │   │   ├── AnalysisDashboard.jsx
│   │   │   └── ReportPreview.jsx
│   │   ├── App.jsx
│   │   ├── main.jsx
│   │   └── index.css            # Styles
│   ├── index.html
│   ├── package.json
│   └── vite.config.js
│
├── test_data_1_recent.csv       # Sample data
├── test_data_2_detailed.csv     # Sample data
└── README.md

🎯 Key Features

1. FREE AI Analysis

  • No API keys needed
  • No quota limits
  • 100% local processing
  • Fast & reliable

2. Professional Dashboard

  • Real-time data visualization
  • Interactive charts
  • Color-coded insights
  • Responsive design

3. Government-Format PDF

  • Official layout
  • Colored sections
  • Comprehensive data
  • Professional formatting

4. Easy to Use

  • Drag & drop upload
  • Automatic analysis
  • One-click PDF download
  • Clean interface

🔧 Environment Variables

backend/.env:

PORT=5000
MONGODB_URI=mongodb://localhost:27017/rag-document-analysis
CLIENT_URL=http://localhost:5173

Note: MongoDB is optional!

📊 Sample Data Format

CSV Structure:

Training_ID,Theme,State,Participants,Duration,Completion_Rate,Date
T001,Earthquake,Delhi,45,3,95%,2024-10-15
T002,Flood,Maharashtra,60,2,88%,2024-10-20
...

Required Columns:

  • Training_ID
  • Theme (disaster type)
  • State
  • Participants (number)
  • Duration (days)
  • Completion_Rate (%)
  • Date

✅ System Status

Currently Running:

  • ✅ Backend: http://localhost:5000
  • ✅ Frontend: http://localhost:5173
  • ✅ File Upload: Working
  • ✅ Analysis: Local (FREE)
  • ✅ PDF Generation: Working
  • ✅ Dashboard: Professional UI

No Changes Needed - System is READY! 🎉

🎓 How Analysis Works (Technical)

Data Extraction: 

1. Read CSV rows
2. Parse each row into object
3. Extract fields: Theme, State, Participants, etc.

Metric Calculation: 

totalTrainings = rows.length
totalParticipants = sum of all Participants
stateWiseCoverage = group by State, count
themeDistribution = group by Theme, count
averageCompletionRate = average of Completion_Rate

Insight Generation: 

// Example: Top performing state
const topState = Object.entries(stateWiseCoverage)
    .sort((a, b) => b[1] - a[1])[0];

insight = `${topState[0]} leads with ${topState[1]} trainings`;

Gap Identification: 

// States with < 5 trainings
underservedStates = states.filter(count < 5);

// Themes with < 3 trainings
underservedThemes = themes.filter(count < 3);

Recommendations: 

// Based on gaps
if (underservedStates.length > 0) {
    recommendation = "Increase training coverage in: " + underservedStates.join(", ");
}

🎨 UI Color Scheme

  • Primary: #3b82f6 (Blue)
  • Secondary: #10b981 (Green)
  • Accent: #f59e0b (Orange)
  • Warning: #dc2626 (Red)
  • Background: #f8fafc (Light)
  • Text: #1f2937 (Dark Gray)

📱 Browser Compatibility

  • ✅ Chrome
  • ✅ Edge
  • ✅ Firefox
  • ✅ Safari

🔐 Security

  • File type validation
  • File size limits (10MB)
  • CORS enabled for localhost
  • No sensitive data stored (optional MongoDB)

🎉 Summary

Yeh system:

  1. CSV/Excel files upload karta hai
  2. Data ko parse karta hai
  3. LOCAL AI se analysis karta hai (FREE!)
  4. Professional dashboard dikhata hai
  5. Government-format PDF report banata hai

Sab kuch READY hai - koi change nahi chahiye! ✅

Test karne ke liye:

  1. Browser mein http://localhost:5173 kholo
  2. test_data_1_recent.csv upload karo
  3. Dashboard dekho
  4. PDF download karo

System 100% working hai! 🚀