Merge pull request #25 from cyberalertnepal/Testing

Implement rate limiting, Nepali support, and RAG pipeline fixes

.env-example

@@ -1,2 +1,34 @@
 MY_SECRET_TOKEN="SECRET_CODE_TOKEN"
 
+# CHROMA_HOST = "localhost" (Host gareko address rakhney)
+
+
+# EXAMPLE CONFIGURATIONS FOR DIFFERENT PROVIDERS(Use only one at once)
+# ===========================================
+
+# FOR OPENAI:(PAID)
+# LLM_PROVIDER=openai
+# LLM_API_KEY=sk-your-openai-api-key
+# LLM_MODEL=gpt-3.5-turbo
+# # Other options: gpt-4, gpt-4-turbo-preview, etc.
+
+# FOR GROQ:(FREE: BABAL XA-> prefer this)
+# LLM_PROVIDER=groq
+# LLM_API_KEY=gsk_your-groq-api-key
+# LLM_MODEL=llama-3.3-70b-versatile
+# # Other options: llama-3.1-70b-versatile, mixtral-8x7b-32768, etc.
+
+# FOR OPENROUTER:(FREE: LASTAI RATE LIMIT LAGAUXA)
+# LLM_PROVIDER=openrouter
+# LLM_API_KEY=sk-or-your-openrouter-api-key
+# LLM_MODEL=meta-llama/llama-3.1-8b-instruct:free
+# # Other options: anthropic/claude-3-haiku, google/gemma-7b-it, etc.
+
+# ===========================================
+# ADVANCED CONFIGURATION
+# ===========================================
+# Temperature (0.0 to 1.0) - controls randomness
+# LLM_TEMPERATURE=0.1
+
+# Maximum tokens for response
+# LLM_MAX_TOKENS=4096

.gitignore

@@ -66,3 +66,6 @@ notebooks
 np_text_model/classifier/sentencepiece.bpe.model
 np_text_model/classifier/tokenizer.json
 
+# vector database
+chroma_data
+chroma_database

README.md

@@ -1,3 +1,15 @@
+---
+title: Testing AI Contain
+emoji: 🤖
+colorFrom: blue
+colorTo: green
+sdk: docker
+sdk_version: "latest"
+app_file: app.py
+pinned: false
+---
+
+# AI-Contain-Checker
 # AI-Content-Checker
 
 A modular AI content detection system with support for **image classification**, **image edit detection**, **Nepali text classification**, and **general text classification**. Built for performance and extensibility, it is ideal for detecting AI-generated content in both visual and textual forms.
@@ -150,3 +162,4 @@ AI-Checker/
 ## 📄 License
 
 See full license terms here: [`LICENSE.md`](license.md)
+

READMEs.md

@@ -0,0 +1,152 @@
+# AI-Contain-Checker
+
+A modular AI content detection system with support for **image classification**, **image edit detection**, **Nepali text classification**, and **general text classification**. Built for performance and extensibility, it is ideal for detecting AI-generated content in both visual and textual forms.
+
+
+## 🌟 Features
+
+### 🖼️ Image Classifier
+
+* **Purpose**: Classifies whether an image is AI-generated or a real-life photo.
+* **Model**: Fine-tuned **InceptionV3** CNN.
+* **Dataset**: Custom curated dataset with **\~79,950 images** for binary classification.
+* **Location**: [`features/image_classifier`](features/image_classifier)
+* **Docs**: [`docs/features/image_classifier.md`](docs/features/image_classifier.md)
+
+### 🖌️ Image Edit Detector
+
+* **Purpose**: Detects image tampering or post-processing.
+* **Techniques Used**:
+
+  * **Error Level Analysis (ELA)**: Visualizes compression artifacts.
+  * **Fast Fourier Transform (FFT)**: Detects unnatural frequency patterns.
+* **Location**: [`features/image_edit_detector`](features/image_edit_detector)
+* **Docs**:
+
+  * [ELA](docs/detector/ELA.md)
+  * [FFT](docs/detector/fft.md )
+  * [Metadata Analysis](docs/detector/meta.md)
+  * [Backend Notes](docs/detector/note-for-backend.md)
+
+### 📝 Nepali Text Classifier
+
+* **Purpose**: Determines if Nepali text content is AI-generated or written by a human.
+* **Model**: Based on `XLMRClassifier` fine-tuned on Nepali language data.
+* **Dataset**: Scraped dataset of **\~18,000** Nepali texts.
+* **Location**: [`features/nepali_text_classifier`](features/nepali_text_classifier)
+* **Docs**: [`docs/features/nepali_text_classifier.md`](docs/features/nepali_text_classifier.md)
+
+### 🌐 English Text Classifier
+
+* **Purpose**: Detects if English text is AI-generated or human-written.
+* **Pipeline**:
+
+  * Uses **GPT2 tokenizer** for input preprocessing.
+  * Custom binary classifier to differentiate between AI and human-written content.
+* **Location**: [`features/text_classifier`](features/text_classifier)
+* **Docs**: [`docs/features/text_classifier.md`](docs/features/text_classifier.md)
+
+---
+
+## 🗂️ Project Structure
+
+```bash
+AI-Checker/
+│
+├── app.py                  # Main FastAPI entry point
+├── config.py               # Configuration settings
+├── Dockerfile              # Docker build script
+├── Procfile                # Deployment file for Heroku or similar
+├── requirements.txt        # Python dependencies
+├── README.md               # You are here 📘
+│
+├── features/               # Core detection modules
+│   ├── image_classifier/
+│   ├── image_edit_detector/
+│   ├── nepali_text_classifier/
+│   └── text_classifier/
+│
+├── docs/                   # Internal and API documentation
+│   ├── api_endpoints.md
+│   ├── deployment.md
+│   ├── detector/
+│   │   ├── ELA.md
+│   │   ├── fft.md
+│   │   ├── meta.md
+│   │   └── note-for-backend.md
+│   ├── functions.md
+│   ├── nestjs_integration.md
+│   ├── security.md
+│   ├── setup.md
+│   └── structure.md
+│
+├── IMG_Models/             # Saved image classifier model(s)
+│   └── latest-my_cnn_model.h5
+│
+├── notebooks/              # Experimental and debug notebooks
+├── static/                 # Static assets if needed
+└── test.md                 # Test notes
+````
+
+---
+
+## 📚 Documentation Links
+
+* [API Endpoints](docs/api_endpoints.md)
+* [Deployment Guide](docs/deployment.md)
+* [Detector Documentation](docs/detector/)
+
+  * [Error Level Analysis (ELA)](docs/detector/ELA.md)
+  * [Fast Fourier Transform (FFT)](docs/detector/fft.md)
+  * [Metadata Analysis](docs/detector/meta.md)
+  * [Backend Notes](docs/detector/note-for-backend.md)
+* [Functions Overview](docs/functions.md)
+* [NestJS Integration Guide](docs/nestjs_integration.md)
+* [Security Details](docs/security.md)
+* [Setup Instructions](docs/setup.md)
+* [Project Structure](docs/structure.md)
+
+---
+
+## 🚀 Usage
+
+1. **Install dependencies**
+
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+2. **Run the API**
+
+   ```bash
+   uvicorn app:app --reload
+   ```
+
+3. **Build Docker (optional)**
+
+   ```bash
+   docker build -t ai-contain-checker .
+   docker run -p 8000:8000 ai-contain-checker
+   ```
+
+---
+
+## 🔐 Security & Integration
+
+* **Token Authentication** and **IP Whitelisting** supported.
+* NestJS integration guide: [`docs/nestjs_integration.md`](docs/nestjs_integration.md)
+* Rate limiting handled using `slowapi`.
+
+---
+
+## 🛡️ Future Plans
+
+* Add **video classifier** module.
+* Expand dataset for **multilingual** AI content detection.
+* Add **fine-tuning UI** for models.
+
+---
+
+## 📄 License
+
+See full license terms here: [`LICENSE.md`](license.md)

docs/detector/ai_human_image_checker.md

@@ -0,0 +1,132 @@
+Real vs. Fake Image Classification for Production Pipeline
+==========================================================
+
+1\. Business Problem
+--------------------
+
+This project addresses the critical business need to automatically identify and flag manipulated or synthetically generated images. By accurately classifying images as **"real"** or **"fake,"** we can enhance the integrity of our platform, prevent the spread of misinformation, and protect our users from fraudulent content. This solution is designed for integration into our production pipeline to process images in real-time.
+
+2\. Solution Overview
+---------------------
+
+This solution leverages OpenAI's CLIP (Contrastive Language-Image Pre-Training) model to differentiate between real and fake images. The system operates as follows:
+
+1.  **Feature Extraction:** A pre-trained CLIP model ('ViT-L/14') converts input images into 768-dimensional feature vectors.
+    
+2.  **Classification:** A Support Vector Machine (SVM) model, trained on our internal dataset of real and fake images, classifies the feature vectors.
+    
+3.  **Deployment:** The trained model is deployed as a service that can be integrated into our production image processing pipeline.
+    
+
+The model has achieved an accuracy of **98.29%** on our internal test set, demonstrating its effectiveness in distinguishing between real and fake images.
+
+3\. Getting Started
+-------------------
+
+### 3.1. Dependencies
+
+To ensure a reproducible environment, all dependencies are listed in the requirements.txt file. Install them using pip:
+
+```bash
+pip install -r requirements.txt
+```
+
+**requirements.txt**:
+ - numpy 
+ - Pillow  
+ - torch  
+ - clip-by-openai  
+ - scikit-learn  
+ - tqdm  
+ - seaborn  
+ - matplotlib
+
+### 3.2. Data Preparation
+
+The model was trained on a dataset of real and fake images obtained form kaggle the dataset link is https://www.kaggle.com/datasets/tristanzhang32/ai-generated-images-vs-real-images/data$0.
+
+### 3.3. Usage
+
+#### 3.3.1. Feature Extraction
+
+To extract features from a new dataset, run the following command:
+
+```
+ python extract_features.py --data_dir /path/to/your/data --output_file features.npz
+ ```   
+
+#### 3.3.2. Model Training
+
+To retrain the SVM model on a new set of extracted features, run:
+
+```
+python train_model.py --features_file features.npz --model_output_path model.joblib   
+```
+
+#### 3.3.3. Inference
+
+To classify a single image using the trained model, use the provided inference script:
+```
+ python classify.py --image_path /path/to/your/image.jpg --model_path model.joblib  
+ ```
+
+4\. Production Deployment
+-------------------------
+
+The image classification model is deployed as a microservice. The service exposes an API endpoint that accepts an image and returns a classification result ("real" or "fake").
+
+### 4.1. API Specification
+
+*   **Endpoint:** /classify
+    
+*   **Method:** POST
+    
+*   **Request Body:** multipart/form-data with a single field image.
+    
+*   **Response:**
+    
+    *   JSON{ "classification": "real", "confidence": 0.95}
+        
+    *   JSON{ "error": "Error message"}
+        
+
+### 4.2. Scalability and Monitoring
+
+The service is deployed in a containerized environment (e.g., Docker) and managed by an orchestrator (e.g., Kubernetes) to ensure scalability and high availability. Monitoring and logging are in place to track model performance, API latency, and error rates.
+
+5\. Model Versioning
+--------------------
+
+We use a combination of Git for code versioning and a model registry for tracking trained model artifacts. Each model is versioned and associated with the commit hash of the code that produced it. The current production model is **v1.2.0**.
+
+6\. Testing
+-----------
+
+The project includes a suite of tests to ensure correctness and reliability:
+
+*   **Unit tests:** To verify individual functions and components.
+    
+*   **Integration tests:** To test the interaction between different parts of the system.
+    
+*   **Model evaluation tests:** To continuously monitor model performance on a golden dataset.
+    
+
+To run the tests, execute:
+``` 
+pytest   
+```
+
+7\. Future Work
+---------------
+
+*   **Explore more advanced classifiers:** Investigate the use of neural network-based classifiers on top of CLIP features.
+    
+*   **Fine-tune the CLIP model:** For even better performance, we can fine-tune the CLIP model on our specific domain of images.
+    
+*   **Expand the training dataset:** Continuously augment the training data with new examples of real and fake images to improve the model's robustness.
+    
+
+8\. Contact/Support
+-------------------
+
+For any questions or issues regarding this project, please contact the Machine Learning team at [your-team-email@yourcompany.com](mailto:your-team-email@yourcompany.com) .

features/ai_human_image_classifier/controller.py

@@ -0,0 +1,35 @@
+from typing import IO
+from preprocessor import preprocessor
+from inferencer import inferencer
+
+class ClassificationController:
+    """
+    Controller to handle the image classification logic.
+    """
+    def classify_image(self, image_file: IO) -> dict:
+        """
+        Orchestrates the classification of a single image file.
+
+        Args:
+            image_file (IO): The image file to classify.
+
+        Returns:
+            dict: The classification result.
+        """
+        try:
+            # Step 1: Preprocess the image
+            image_tensor = preprocessor.process(image_file)
+
+            # Step 2: Perform inference
+            result = inferencer.predict(image_tensor)
+
+            return result
+        except ValueError as e:
+            # Handle specific errors like invalid images
+            return {"error": str(e)}
+        except Exception as e:
+            # Handle unexpected errors
+            print(f"An unexpected error occurred: {e}")
+            return {"error": "An internal error occurred during classification."}
+
+controller = ClassificationController()

features/ai_human_image_classifier/inferencer.py

@@ -0,0 +1,48 @@
+import torch
+import numpy as np
+from model_loader import models
+
+class Inferencer:
+
+    def __init__(self):
+        self.clip_model = models.clip_model
+        self.svm_model = models.svm_model
+
+    @torch.no_grad()
+    def predict(self, image_tensor:torch.Tensor) -> dict:
+        """
+        Takes a preprocessed image tensor and returns the classification result.
+
+        Args:
+            image_tensor (torch.Tensor): The preprocessed image tensor.
+
+        Returns:
+            dict: A dictionary containing the classification label and confidence score.
+        """
+
+        image_features = self.clip_model.encode_image(image_tensor)
+        image_features_np = image_features.cpu().numpy()
+
+        prediction = self.svm_model.predict(image_features_np)[0]
+
+        if hasattr(self.svm_model, "predict_proba"):
+            # If yes, use predict_proba for a true confidence score
+            confidence_scores = self.svm_model.predict_proba(image_features_np)[0]
+            confidence = float(np.max(confidence_scores))
+        else:
+            # If no, use decision_function as a fallback confidence measure.
+            # The absolute value of the decision function score indicates confidence.
+            # We can apply a sigmoid function to scale it to a [0, 1] range for consistency.
+            decision_score = self.svm_model.decision_function(image_features_np)[0]
+            confidence = 1 / (1 + np.exp(-np.abs(decision_score)))
+            confidence = float(confidence)
+
+        label_map = {0: 'real', 1: 'fake'}
+        classification_label = label_map.get(prediction, "unknown")
+
+        return {
+            "classification": classification_label,
+            "confidence": confidence
+        }
+
+inferencer = Inferencer()

features/ai_human_image_classifier/main.py

@@ -0,0 +1,27 @@
+from fastapi import FastAPI
+from routes import router as api_router
+
+# Initialize the FastAPI app
+app = FastAPI(
+    title="Real vs. Fake Image Classification API",
+    description="An API to classify images as real or fake using OpenAI's CLIP and an SVM model.",
+    version="1.0.0"
+)
+
+# Include the API router
+# All routes defined in routes.py will be available under the /api prefix
+app.include_router(api_router, prefix="/api", tags=["Classification"])
+
+@app.get("/", tags=["Root"])
+async def read_root():
+    """
+    A simple root endpoint to confirm the API is running.
+    """
+    return {"message": "Welcome to the Image Classification API. Go to /docs for the API documentation."}
+
+
+# To run this application:
+# 1. Make sure you have all dependencies from requirements.txt installed.
+# 2. Make sure the 'svm_model.joblib' file is in the same directory.
+# 3. Run the following command in your terminal:
+#    uvicorn main:app --reload

features/ai_human_image_classifier/model_loader.py

@@ -0,0 +1,80 @@
+import clip
+import torch
+import joblib
+from pathlib import Path
+from huggingface_hub import hf_hub_download
+
+class ModelLoader:
+    """
+    A class to load and hold the machine learning models.
+    This ensures that models are loaded only once.
+    """
+    def __init__(self, clip_model_name: str, svm_repo_id: str, svm_filename: str):
+        """
+        Initializes the ModelLoader and loads the models.
+
+        Args:
+            clip_model_name (str): The name of the CLIP model to load (e.g., 'ViT-L/14').
+            svm_repo_id (str): The repository ID on Hugging Face (e.g., 'rhnsa/ai_human_image_detector').
+            svm_filename (str): The name of the model file in the repository (e.g., 'model.joblib').
+        """
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        print(f"Using device: {self.device}")
+
+        self.clip_model, self.clip_preprocess = self._load_clip_model(clip_model_name)
+        self.svm_model = self._load_svm_model(repo_id=svm_repo_id, filename=svm_filename)
+        print("Models loaded successfully.")
+
+    def _load_clip_model(self, model_name: str):
+        """
+        Loads the specified CLIP model and its preprocessor.
+        
+        Args:
+            model_name (str): The name of the CLIP model.
+        
+        Returns:
+            A tuple containing the loaded CLIP model and its preprocess function.
+        """
+        try:
+            model, preprocess = clip.load(model_name, device=self.device)
+            return model, preprocess
+        except Exception as e:
+            print(f"Error loading CLIP model: {e}")
+            raise
+
+    def _load_svm_model(self, repo_id: str, filename: str):
+        """
+        Downloads and loads the SVM model from a Hugging Face Hub repository.
+
+        Args:
+            repo_id (str): The repository ID on Hugging Face.
+            filename (str): The name of the model file in the repository.
+
+        Returns:
+            The loaded SVM model object.
+        """
+        print(f"Downloading SVM model from Hugging Face repo: {repo_id}")
+        try:
+            # Download the model file from the Hub. It returns the cached path.
+            model_path = hf_hub_download(repo_id=repo_id, filename=filename)
+            print(f"SVM model downloaded to: {model_path}")
+            
+            # Load the model from the downloaded path
+            svm_model = joblib.load(model_path)
+            return svm_model
+        except Exception as e:
+            print(f"Error downloading or loading SVM model from Hugging Face: {e}")
+            raise
+
+# --- Global Model Instance ---
+# This creates a single instance of the models that can be imported by other modules.
+CLIP_MODEL_NAME = 'ViT-L/14'
+SVM_REPO_ID = 'rhnsa/ai_human_image_detector'
+SVM_FILENAME = 'svm_model_real.joblib' # The name of your model file in the Hugging Face repo
+
+# This instance will be created when the application starts.
+models = ModelLoader(
+    clip_model_name=CLIP_MODEL_NAME, 
+    svm_repo_id=SVM_REPO_ID, 
+    svm_filename=SVM_FILENAME
+)

features/ai_human_image_classifier/preprocessor.py

@@ -0,0 +1,34 @@
+from PIL import Image
+import torch
+from typing import IO
+from model_loader import models
+
+class ImagePreprocessor:
+
+    def __init__(self):
+        self.preprocess = models.clip_preprocess
+        self.device = models.device
+
+    def process(self, image_file: IO) -> torch.Tensor:
+        """
+        Opens an image file, preprocesses it, and returns it as a tensor.
+
+        Args:
+            image_file (IO): The image file object (e.g., from a file upload).
+
+        Returns:
+            torch.Tensor: The preprocessed image as a tensor, ready for the model.
+        """
+        try:
+            # Open the image from the file-like object
+            image = Image.open(image_file).convert("RGB")
+        except Exception as e:
+            print(f"Error opening image: {e}")
+            # You might want to raise a custom exception here
+            raise ValueError("Invalid or corrupted image file.")
+
+        # Apply the CLIP preprocessing transformations and move to the correct device
+        image_tensor = self.preprocess(image).unsqueeze(0).to(self.device)
+        return image_tensor
+    
+preprocessor = ImagePreprocessor()

features/ai_human_image_classifier/routes.py

@@ -0,0 +1,44 @@
+from fastapi import APIRouter, File, UploadFile, HTTPException, status
+from fastapi.responses import JSONResponse
+from controller import controller
+
+from fastapi import Request, Depends
+from fastapi.security import HTTPBearer
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+
+
+router = APIRouter()
+limiter = Limiter(key_func=get_remote_address)
+security = HTTPBearer()
+# Create an API router
+router = APIRouter()
+
+@router.post("/classify", summary="Classify an image as Real or Fake")
+async def classify_image_endpoint(image: UploadFile = File(...)):
+    """
+    Accepts an image file and classifies it as 'real' or 'fake'.
+
+    - **image**: The image file to be classified (e.g., JPEG, PNG).
+
+    Returns a JSON object with the classification and a confidence score.
+    """
+    # Check for a valid image content type
+    if not image.content_type.startswith("image/"):
+        raise HTTPException(
+            status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
+            detail="Unsupported file type. Please upload an image (e.g., JPEG, PNG)."
+        )
+
+    # The controller expects a file-like object, which `image.file` provides
+    result = controller.classify_image(image.file)
+
+    if "error" in result:
+        # If the controller returned an error, forward it as an HTTP exception
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=result["error"]
+        )
+    
+    return JSONResponse(content=result, status_code=status.HTTP_200_OK)
+

features/nepali_text_classifier/preprocess.py

@@ -1,9 +1,9 @@
-import fitz  # PyMuPDF
+# import fitz  # PyMuPDF
 import docx
 from io import BytesIO
 import logging
 from fastapi import HTTPException
-
+from pypdf import PdfReader  
 
 def parse_docx(file: BytesIO):
     doc = docx.Document(file)
@@ -15,11 +15,10 @@ def parse_docx(file: BytesIO):
 
 def parse_pdf(file: BytesIO):
     try:
-        doc = fitz.open(stream=file, filetype="pdf")
+        doc = PdfReader(file)
         text = ""
-        for page_num in range(doc.page_count):
-            page = doc.load_page(page_num)
-            text += page.get_text()
+        for page in doc.pages:
+            text += page.extract_text()
         return text 
     except Exception as e:
         logging.error(f"Error while processing PDF: {str(e)}")

features/rag_chatbot/controller.py

@@ -0,0 +1,182 @@
+import os
+import asyncio
+import logging
+from io import BytesIO
+from typing import Dict, Any
+
+from fastapi import HTTPException, UploadFile, status, Depends
+from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
+
+from .rag_pipeline import route_and_process_query, add_document_to_rag, check_system_health
+from .document_handler import extract_text_from_file
+
+# Configure logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+security = HTTPBearer()
+
+# Supported file types
+SUPPORTED_CONTENT_TYPES = {
+    "application/pdf",
+    "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+    "text/plain"
+}
+
+MAX_FILE_SIZE = 100 * 1024 * 1024  # 100MB
+
+async def verify_token(credentials: HTTPAuthorizationCredentials = Depends(security)):
+    """Verify Bearer token from Authorization header."""
+    token = credentials.credentials
+    expected_token = os.getenv("MY_SECRET_TOKEN")
+    
+    if not expected_token:
+        logger.error("MY_SECRET_TOKEN not configured")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Server configuration error"
+        )
+    
+    if token != expected_token:
+        logger.warning(f"Invalid token attempt: {token[:10]}...")
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Invalid or expired token"
+        )
+    return token
+
+async def handle_rag_query(query: str) -> Dict[str, Any]:
+    """Handle an incoming query by routing it and getting the appropriate answer."""
+    
+    # Input validation
+    if not query or not query.strip():
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST, 
+            detail="Query cannot be empty"
+        )
+    
+    if len(query) > 1000:  # Reasonable limit
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="Query too long. Please limit to 1000 characters."
+        )
+    
+    try:
+        logger.info(f"Processing query: {query[:50]}...")
+        
+        # Process query in thread pool 
+        response = await asyncio.to_thread(route_and_process_query, query)
+        
+        logger.info(f"Query processed successfully. Route: {response.get('route', 'Unknown')}")
+        return response
+        
+    except Exception as e:
+        logger.error(f"Error processing query: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Error processing your query. Please try again."
+        )
+
+async def handle_document_upload(file: UploadFile) -> Dict[str, str]:
+    """Handle uploading a document to the RAG's vector store."""
+    
+    # File validation
+    if not file.filename:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="No file provided"
+        )
+    
+    if file.content_type not in SUPPORTED_CONTENT_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
+            detail=f"Unsupported file type: {file.content_type}. "
+                   f"Supported types: {', '.join(SUPPORTED_CONTENT_TYPES)}"
+        )
+    
+    # Check file size
+    contents = await file.read()
+    if len(contents) > MAX_FILE_SIZE:
+        raise HTTPException(
+            status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
+            detail=f"File too large. Maximum size: {MAX_FILE_SIZE / (1024*1024):.1f}MB"
+        )
+    
+    # Reset file pointer
+    await file.seek(0)
+    
+    try:
+        logger.info(f"Processing file upload: {file.filename}")
+        
+        # Extract text from file
+        text = await extract_text_from_file(file)
+        
+        if not text or not text.strip():
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail="The file appears to be empty or could not be read."
+            )
+        
+        if len(text) < 50:  # Too short to be meaningful
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail="The extracted text is too short to be meaningful."
+            )
+        
+        # Add to RAG system
+        success = await asyncio.to_thread(
+            add_document_to_rag, 
+            text, 
+            {
+                "source": file.filename,
+                "content_type": file.content_type,
+                "size": len(contents)
+            }
+        )
+        
+        if not success:
+            raise HTTPException(
+                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                detail="Failed to add document to the knowledge base"
+            )
+        
+        logger.info(f"Successfully processed file: {file.filename}")
+        
+        return {
+            "message": f"Successfully uploaded and processed '{file.filename}'. "
+                      f"It is now available for querying.",
+            "filename": file.filename,
+            "text_length": len(text),
+            "content_type": file.content_type
+        }
+
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Error processing file {file.filename}: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Error processing the file. Please try again."
+        )
+
+async def handle_health_check() -> Dict[str, Any]:
+    """Handle health check requests."""
+    try:
+        health_status = await asyncio.to_thread(check_system_health)
+        
+        if health_status["status"] == "unhealthy":
+            raise HTTPException(
+                status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+                detail="Service is currently unhealthy"
+            )
+        
+        return health_status
+        
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Health check failed: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Health check failed"
+        )

features/rag_chatbot/document_handler.py

@@ -0,0 +1,37 @@
+from io import BytesIO
+from fastapi import UploadFile, HTTPException
+import PyPDF2
+import docx
+
+async def extract_text_from_file(file: UploadFile) -> str:
+    """Extracts text from various file types."""
+    content = await file.read()
+    file_stream = BytesIO(content)
+
+    if file.content_type == "application/pdf":
+        return extract_text_from_pdf(file_stream)
+    elif file.content_type == "application/vnd.openxmlformats-officedocument.wordprocessingml.document":
+        return extract_text_from_docx(file_stream)
+    elif file.content_type == "text/plain":
+        return file_stream.read().decode("utf-8")
+    else:
+        raise HTTPException(
+            status_code=415,
+            detail="Unsupported file type. Please upload a .pdf, .docx, or .txt file."
+        )
+
+def extract_text_from_pdf(file_stream: BytesIO) -> str:
+    """Extracts text from a PDF file."""
+    reader = PyPDF2.PdfReader(file_stream)
+    text = ""
+    for page in reader.pages:
+        text += page.extract_text() or ""
+    return text
+
+def extract_text_from_docx(file_stream: BytesIO) -> str:
+    """Extracts text from a DOCX file."""
+    doc = docx.Document(file_stream)
+    text = ""
+    for para in doc.paragraphs:
+        text += para.text + "\n"
+    return text

features/rag_chatbot/rag_pipeline.py

@@ -0,0 +1,327 @@
+import os
+import chromadb
+from dotenv import load_dotenv
+from langchain_core.documents import Document
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain_community.embeddings import HuggingFaceEmbeddings
+from langchain_community.llms import OpenAI
+from langchain.chains.question_answering import load_qa_chain
+from langchain_community.vectorstores import Chroma
+from langchain.chains import LLMChain
+from langchain.prompts import PromptTemplate
+from langchain.chat_models import ChatOpenAI
+
+
+load_dotenv()
+
+# ChromaDB configuration
+CHROMA_HOST = os.getenv("CHROMA_HOST", "localhost") # change in env in production when hosted
+COLLECTION_NAME = "company_docs_collection"
+
+# LLM Provider Configuration
+LLM_PROVIDER = os.getenv("LLM_PROVIDER", "openai").lower()  
+LLM_API_KEY = os.getenv("LLM_API_KEY")
+LLM_MODEL = os.getenv("LLM_MODEL", "gpt-3.5-turbo")
+LLM_TEMPERATURE = float(os.getenv("LLM_TEMPERATURE", "0"))
+LLM_MAX_TOKENS = int(os.getenv("LLM_MAX_TOKENS", "2048"))
+
+# Provider-specific configurations
+PROVIDER_CONFIGS = {
+    "openai": {
+        "api_base": "https://api.openai.com/v1",
+        "default_model": "gpt-3.5-turbo"
+    },
+    "groq": {
+        "api_base": "https://api.groq.com/openai/v1",
+        "default_model": "llama-3.3-70b-versatile"
+    },
+    "openrouter": {
+        "api_base": "https://openrouter.ai/api/v1",
+        "default_model": "mistralai/mistral-small-3.2-24b-instruct:free"
+    }
+}
+
+vector_store = None
+company_qa_chain = None
+query_router_chain = None
+cybersecurity_chain = None
+llm = None
+
+def get_llm_config():
+    """Get the appropriate LLM configuration based on the provider."""
+    if LLM_PROVIDER not in PROVIDER_CONFIGS:
+        raise ValueError(f"Unsupported LLM provider: {LLM_PROVIDER}. Supported: {list(PROVIDER_CONFIGS.keys())}")
+    
+    config = PROVIDER_CONFIGS[LLM_PROVIDER].copy()
+    
+    # Use provided model or fall back to default
+    model = LLM_MODEL if LLM_MODEL != "gpt-3.5-turbo" else config["default_model"]
+    
+    return {
+        "model": model,
+        "openai_api_key": LLM_API_KEY,
+        "openai_api_base": config["api_base"],
+        "temperature": LLM_TEMPERATURE,
+        "max_tokens": LLM_MAX_TOKENS,
+    }
+
+def initialize_llm():
+    """Initialize the LLM based on the configured provider."""
+    if not LLM_API_KEY:
+        raise ValueError(f"LLM_API_KEY environment variable is required for {LLM_PROVIDER}")
+    
+    config = get_llm_config()
+    
+    print(f"Initializing {LLM_PROVIDER.upper()} with model: {config['model']}")
+    
+    return ChatOpenAI(**config)
+
+def initialize_pipelines():
+    """Initializes all required models, chains, and the vector store."""
+    global vector_store, company_qa_chain, query_router_chain, cybersecurity_chain, llm
+    
+    try:
+        # Initialize LLM
+        llm = initialize_llm()
+
+        # Initialize embeddings
+        embeddings = HuggingFaceEmbeddings(
+            model_name="all-MiniLM-L6-v2",  
+            model_kwargs={'device': 'cpu'}, 
+            encode_kwargs={'normalize_embeddings': True}
+        )
+        
+        # Initialize ChromaDB client 
+        try:
+            chroma_client = chromadb.HttpClient(host=CHROMA_HOST, port=8000)
+            chroma_client.heartbeat()
+        except Exception as e:
+            raise ConnectionError("Failed to connect to ChromaDB.") from e
+        
+        # Initialize vector store
+        vector_store = Chroma(
+            client=chroma_client,
+            collection_name=COLLECTION_NAME,
+            embedding_function=embeddings,
+        )
+
+        # Query Router Chain
+        router_template = """You are a query classifier. Classify the following query into one of these categories:
+- COMPANY: Questions about our company, its products, services, or general information
+- CYBERSECURITY: Questions about cybersecurity, security threats, best practices, or vulnerabilities
+- OFF_TOPIC: Questions that don't fit the above categories
+
+Query: {query}
+
+Respond with only the category name (COMPANY, CYBERSECURITY, or OFF_TOPIC):"""
+        
+        router_prompt = PromptTemplate(
+            input_variables=["query"],
+            template=router_template
+        )
+        
+        query_router_chain = LLMChain(
+            llm=llm,
+            prompt=router_prompt
+        )
+        
+        # Custom Company QA Chain
+        company_qa_template = """You are a helpful assistant for CyberAlertNepal. Answer the following question about our company using the information provided and links if only available. Give a natural, direct and polite response.
+
+Question: {question}
+
+Information:
+{context}
+
+Answer:"""
+        
+        company_qa_prompt = PromptTemplate(
+            input_variables=["question", "context"],
+            template=company_qa_template
+        )
+        
+        company_qa_chain = LLMChain(
+            llm=llm,
+            prompt=company_qa_prompt
+        )
+        
+        # Cybersecurity Chain
+        cybersecurity_template = """You are a cybersecurity professional. Answer the following question truthfully and concisely. 
+If you are not 100% sure about the answer, simply respond with: "I am not sure about the answer." 
+Do not add extra explanations or assumptions. Do not provide false or speculative information.
+
+Question: {question}
+
+Provide a comprehensive and accurate answer about cybersecurity:"""
+        
+        cybersecurity_prompt = PromptTemplate(
+            input_variables=["question"],
+            template=cybersecurity_template
+        )
+        
+        cybersecurity_chain = LLMChain(
+            llm=llm,
+            prompt=cybersecurity_prompt
+        )
+        
+        print(f"Successfully initialized pipelines with {LLM_PROVIDER.upper()}")
+
+    except Exception as e:
+        print(f"Error initializing pipelines: {e}")
+        raise
+
+def add_document_to_rag(text: str, metadata: dict):
+    """Splits a document and adds it to the ChromaDB index."""
+    global vector_store
+    
+    if not vector_store:
+        initialize_pipelines()
+        
+    try:
+        text_splitter = RecursiveCharacterTextSplitter(
+            chunk_size=1000, 
+            chunk_overlap=200
+        )
+        docs = text_splitter.create_documents([text], metadatas=[metadata])
+        
+        if not docs:
+            print("Document was empty after splitting, not adding to ChromaDB.")
+            return False
+
+        vector_store.add_documents(docs)
+        print("Successfully added documents.")
+        return True
+        
+    except Exception as e:
+        print(f"Error adding document to RAG: {e}")
+        return False
+
+def route_and_process_query(query: str):
+    """Routes the query and processes it using the appropriate pipeline."""
+    global query_router_chain, vector_store, company_qa_chain, cybersecurity_chain
+    
+    if not all([query_router_chain, vector_store, company_qa_chain, cybersecurity_chain]):
+        initialize_pipelines()
+
+    try:
+        # 1. Classify the query
+        route_result = query_router_chain.run(query)
+        route = route_result.strip().upper()
+        
+
+        # 2. Route to appropriate logic
+        if "CYBERSECURITY" in route:
+            answer = cybersecurity_chain.run(question=query)
+            return {
+                "answer": answer, 
+                "source": "Cybersecurity Knowledge Base",
+                "route": "CYBERSECURITY",
+                "provider": LLM_PROVIDER.upper(),
+                "model": get_llm_config()["model"]
+            }
+            
+        elif "COMPANY" in route:
+            # Perform similarity search on ChromaDB
+            docs = vector_store.similarity_search(query, k=3)
+
+            if not docs:
+                return {
+                    "answer": "I could not find any relevant information to answer your question.",
+                    "source": "Company Documents",
+                    "route": "COMPANY",
+                    "provider": LLM_PROVIDER.upper(),
+                    "model": get_llm_config()["model"]
+                }
+            
+            # Combine document content for context
+            context = "\n\n".join([doc.page_content for doc in docs])
+            
+            # Run the custom QA chain
+            answer = company_qa_chain.run(question=query, context=context)
+            sources = list(set([doc.metadata.get("source", "Unknown") for doc in docs]))
+            
+            return {
+                "answer": answer,
+                "source": "Company Documents",
+                "documents": sources,
+                "route": "COMPANY",
+                "provider": LLM_PROVIDER.upper(),
+                "model": get_llm_config()["model"]
+            }
+
+        else:  # OFF_TOPIC
+            return {
+                "answer": "I am a specialized assistant of CyberAlertNepal. I cannot answer questions outside of cybersecurity topics.",
+                "source": "N/A",
+                "route": "OFF_TOPIC",
+                "provider": LLM_PROVIDER.upper(),
+                "model": get_llm_config()["model"]
+            }
+            
+    except Exception as e:
+        print(f"Error processing query: {e}")
+        return {
+            "answer": "I encountered an error while processing your query. Please try again.",
+            "source": "Error",
+            "route": None,
+            "documents": None,
+            "provider": LLM_PROVIDER.upper(),
+            "error": str(e)
+        }
+
+def check_system_health():
+    """Check if all components are properly initialized."""
+    try:
+        # Test ChromaDB connection
+        if vector_store:
+            vector_store._client.heartbeat()
+        
+        # Test if all chains are initialized
+        components = {
+            "vector_store": vector_store is not None,
+            "company_qa_chain": company_qa_chain is not None,
+            "query_router_chain": query_router_chain is not None,
+            "cybersecurity_chain": cybersecurity_chain is not None,
+            "llm": llm is not None
+        }
+        
+        return {
+            "status": "healthy" if all(components.values()) else "unhealthy",
+            "components": components,
+            "provider": LLM_PROVIDER.upper(),
+            "model": get_llm_config()["model"] if llm else "Not initialized"
+        }
+        
+    except Exception as e:
+        return {
+            "status": "unhealthy",
+            "error": str(e),
+            "provider": LLM_PROVIDER.upper()
+        }
+
+def test_llm_connection():
+    """Test the LLM API connection."""
+    try:
+        if not llm:
+            initialize_pipelines()
+        
+        # Simple test query
+        test_response = llm("Say 'Hello, LLM is working!'")
+        return {
+            "success": True,
+            "provider": LLM_PROVIDER.upper(),
+            "model": get_llm_config()["model"],
+            "response": str(test_response)
+        }
+    except Exception as e:
+        return {
+            "success": False,
+            "provider": LLM_PROVIDER.upper(),
+            "error": str(e)
+        }
+
+# Initialize pipelines on module import
+try:
+    initialize_pipelines()
+except Exception as e:
+    print(f"Failed to initialize pipelines on startup: {e}")

features/rag_chatbot/routes.py

@@ -0,0 +1,111 @@
+from fastapi import APIRouter, Depends, HTTPException, UploadFile, File, Request
+from fastapi.security import HTTPBearer
+from pydantic import BaseModel, Field
+from slowapi.util import get_remote_address
+from slowapi import Limiter
+from typing import Optional
+from config import ACCESS_RATE
+from .controller import (
+    handle_rag_query,
+    handle_document_upload,
+    handle_health_check,
+    verify_token,
+)
+
+limiter = Limiter(key_func=get_remote_address)
+router = APIRouter(prefix="/rag", tags=["RAG Chatbot"])
+security = HTTPBearer()
+
+class QueryInput(BaseModel):
+    query: str = Field(..., min_length=1, max_length=1000, description="The question to ask")
+
+class QueryResponse(BaseModel):
+    answer: str
+    source: str
+    route: Optional[str] = None
+    documents: Optional[list] = None
+    error: Optional[str] = None
+
+class UploadResponse(BaseModel):
+    message: str
+    filename: str
+    text_length: int
+    content_type: str
+
+class HealthResponse(BaseModel):
+    status: str
+    components: Optional[dict] = None
+    error: Optional[str] = None
+
+@router.post("/question", response_model=QueryResponse)
+@limiter.limit(ACCESS_RATE)
+async def ask_question(
+    request: Request, 
+    data: QueryInput, 
+    token: str = Depends(verify_token)
+) -> QueryResponse:
+    """
+    Ask a question to the RAG chatbot.
+    
+    The chatbot can answer:
+    - Company-related questions (based on uploaded documents)
+    - Cybersecurity questions (from knowledge base)
+    """
+    response = await handle_rag_query(data.query)
+    return QueryResponse(**response)
+
+@router.post("/upload", response_model=UploadResponse)
+@limiter.limit(ACCESS_RATE)
+async def upload_document(
+    request: Request, 
+    file: UploadFile = File(..., description="Document file (PDF, DOCX, or TXT)"),
+    token: str = Depends(verify_token)
+) -> UploadResponse:
+    """
+    Upload a document to the company knowledge base.
+    
+    Supported formats:
+    - PDF (.pdf)
+    - Word documents (.docx)
+    - Plain text (.txt)
+    
+    Maximum file size: 10MB
+    """
+    response = await handle_document_upload(file)
+    return UploadResponse(**response)
+
+@router.get("/health", response_model=HealthResponse)
+@limiter.limit(ACCESS_RATE)
+async def health_check(request: Request) -> HealthResponse:
+    """
+    Check the health status of the RAG system.
+    
+    Returns the status of all components:
+    - ChromaDB connection
+    - Vector store
+    - AI chains
+    """
+    response = await handle_health_check()
+    return HealthResponse(**response)
+
+@router.get("/info")
+@limiter.limit(ACCESS_RATE)
+async def get_system_info(request: Request):
+    """Get information about the RAG system capabilities."""
+    return {
+        "name": "RAG Chatbot",
+        "version": "1.0.0",
+        "description": "A specialized chatbot for cybersecurity and company-related questions",
+        "capabilities": [
+            "Company document Q&A (based on uploaded documents)",
+            "Cybersecurity knowledge and best practices",
+            "Document upload and processing (PDF, DOCX, TXT)"
+        ],
+        "supported_file_types": [
+            "application/pdf",
+            "application/vnd.openxmlformats-officedocument.wordprocessingml.document", 
+            "text/plain"
+        ],
+        "max_file_size_mb": 10,
+        "max_query_length": 1000
+    }

features/real_forged_classifier/controller.py

@@ -0,0 +1,36 @@
+from typing import IO
+from preprocessor import preprocessor
+from inferencer import interferencer
+
+class ClassificationController:
+    """
+    Controller to handle the image classification logic.
+    """
+    def classify_image(self, image_file: IO) -> dict:
+        """
+        Orchestrates the classification of a single image file.
+
+        Args:
+            image_file (IO): The image file to classify.
+
+        Returns:
+            dict: The classification result.
+        """
+        try:
+            # Step 1: Preprocess the image
+            image_tensor = preprocessor.process(image_file)
+
+            # Step 2: Perform inference
+            result = interferencer.predict(image_tensor)
+
+            return result
+        except ValueError as e:
+            # Handle specific errors like invalid images
+            return {"error": str(e)}
+        except Exception as e:
+            # Handle unexpected errors
+            print(f"An unexpected error occurred: {e}")
+            return {"error": "An internal error occurred during classification."}
+
+# Create a single instance of the controller
+controller = ClassificationController()

features/real_forged_classifier/inferencer.py

@@ -0,0 +1,52 @@
+import torch
+import torch.nn.functional as F
+import numpy as np
+
+# Import the globally loaded models instance
+from model_loader import models
+
+class Interferencer:
+    """
+    Performs inference using the FFT CNN model.
+    """
+    def __init__(self):
+        """
+        Initializes the interferencer with the loaded model.
+        """
+        self.fft_model = models.fft_model
+
+    @torch.no_grad()
+    def predict(self, image_tensor: torch.Tensor) -> dict:
+        """
+        Takes a preprocessed image tensor and returns the classification result.
+
+        Args:
+            image_tensor (torch.Tensor): The preprocessed image tensor.
+
+        Returns:
+            dict: A dictionary containing the classification label and confidence score.
+        """
+        # 1. Get model outputs (logits)
+        outputs = self.fft_model(image_tensor)
+        
+        # 2. Apply softmax to get probabilities
+        probabilities = F.softmax(outputs, dim=1)
+        
+        # 3. Get the confidence and the predicted class index
+        confidence, predicted_idx = torch.max(probabilities, 1)
+        
+        prediction = predicted_idx.item()
+        
+        # 4. Map the prediction to a human-readable label
+        # Ensure this mapping matches the labels used during training
+        # Typically: 0 -> fake, 1 -> real
+        label_map = {0: 'fake', 1: 'real'}
+        classification_label = label_map.get(prediction, "unknown")
+
+        return {
+            "classification": classification_label,
+            "confidence": confidence.item()
+        }
+
+# Create a single instance of the interferencer
+interferencer = Interferencer()

features/real_forged_classifier/main.py

@@ -0,0 +1,26 @@
+from fastapi import FastAPI
+from routes import router as api_router
+
+# Initialize the FastAPI app
+app = FastAPI(
+    title="Real vs. Fake Image Classification API",
+    description="An API to classify images as real or forged using FFT and cnn.",
+    version="1.0.0"
+)
+
+# Include the API router
+# All routes defined in routes.py will be available under the /api prefix
+app.include_router(api_router, prefix="/api", tags=["Classification"])
+
+@app.get("/", tags=["Root"])
+async def read_root():
+    """
+    A simple root endpoint to confirm the API is running.
+    """
+    return {"message": "Welcome to the Image Classification API. Go to /docs for the API documentation."}
+
+# To run this application:
+# 1. Make sure you have all dependencies from requirements.txt installed.
+# 2. Make sure the 'svm_model.joblib' file is in the same directory.
+# 3. Run the following command in your terminal:
+#    uvicorn main:app --reload

features/real_forged_classifier/model.py

@@ -0,0 +1,34 @@
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+class FFTCNN(nn.Module):
+    """
+    Defines the Convolutional Neural Network architecture.
+    This structure must match the model that was trained and saved.
+    """
+    def __init__(self):
+        super(FFTCNN, self).__init__()
+         # Ensure 'self.' is used here to define the layers as instance attributes
+        self.conv_layers = nn.Sequential(
+            nn.Conv2d(1, 16, kernel_size=3, padding=1),
+            nn.ReLU(),
+            nn.MaxPool2d(kernel_size=2, stride=2),
+            nn.Conv2d(16, 32, kernel_size=3, padding=1),
+            nn.ReLU(),
+            nn.MaxPool2d(kernel_size=2, stride=2)
+        )
+        
+        # Ensure 'self.' is used here as well
+        self.fc_layers = nn.Sequential(
+            nn.Linear(32 * 56 * 56, 128), # This size depends on your 224x224 input
+            nn.ReLU(),
+            nn.Linear(128, 2) # 2 output classes
+        )
+
+    def forward(self, x):
+        # Now, 'self.conv_layers' can be found because it was defined correctly
+        x = self.conv_layers(x)
+        x = x.view(x.size(0), -1) # Flatten the feature maps
+        x = self.fc_layers(x)
+        return x

features/real_forged_classifier/model_loader.py

@@ -0,0 +1,60 @@
+import torch
+from pathlib import Path
+from huggingface_hub import hf_hub_download
+from model import FFTCNN # Import the model architecture
+
+class ModelLoader:
+    """
+    A class to load and hold the PyTorch CNN model.
+    """
+    def __init__(self, model_repo_id: str, model_filename: str):
+        """
+        Initializes the ModelLoader and loads the model.
+
+        Args:
+            model_repo_id (str): The repository ID on Hugging Face.
+            model_filename (str): The name of the model file (.pth) in the repository.
+        """
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        print(f"Using device: {self.device}")
+
+        self.fft_model = self._load_fft_model(repo_id=model_repo_id, filename=model_filename)
+        print("FFT CNN model loaded successfully.")
+
+    def _load_fft_model(self, repo_id: str, filename: str):
+        """
+        Downloads and loads the FFT CNN model from a Hugging Face Hub repository.
+
+        Args:
+            repo_id (str): The repository ID on Hugging Face.
+            filename (str): The name of the model file (.pth) in the repository.
+
+        Returns:
+            The loaded PyTorch model object.
+        """
+        print(f"Downloading FFT CNN model from Hugging Face repo: {repo_id}")
+        try:
+            # Download the model file from the Hub. It returns the cached path.
+            model_path = hf_hub_download(repo_id=repo_id, filename=filename)
+            print(f"Model downloaded to: {model_path}")
+            
+            # Initialize the model architecture
+            model = FFTCNN()
+            
+            # Load the saved weights (state_dict) into the model
+            model.load_state_dict(torch.load(model_path, map_location=torch.device(self.device)))
+            
+            # Set the model to evaluation mode
+            model.to(self.device)
+            model.eval()
+            
+            return model
+        except Exception as e:
+            print(f"Error downloading or loading model from Hugging Face: {e}")
+            raise
+
+# --- Global Model Instance ---
+MODEL_REPO_ID = 'rhnsa/real_forged_classifier' 
+MODEL_FILENAME = 'fft_cnn_model_78.pth'
+models = ModelLoader(model_repo_id=MODEL_REPO_ID, model_filename=MODEL_FILENAME)
+

features/real_forged_classifier/preprocessor.py

@@ -0,0 +1,67 @@
+from PIL import Image
+import torch
+import numpy as np
+from typing import IO
+import cv2
+from torchvision import transforms
+
+# Import the globally loaded models instance
+from model_loader import models
+
+class ImagePreprocessor:
+    """
+    Handles preprocessing of images for the FFT CNN model.
+    """
+    def __init__(self):
+        """
+        Initializes the preprocessor.
+        """
+        self.device = models.device
+        # Define the image transformations, matching the training process
+        self.transform = transforms.Compose([
+            transforms.ToPILImage(),
+            transforms.Resize((224, 224)),
+            transforms.ToTensor(),
+        ])
+
+    def process(self, image_file: IO) -> torch.Tensor:
+        """
+        Opens an image file, applies FFT, preprocesses it, and returns a tensor.
+
+        Args:
+            image_file (IO): The image file object (e.g., from a file upload).
+
+        Returns:
+            torch.Tensor: The preprocessed image as a tensor, ready for the model.
+        """
+        try:
+            # Read the image file into a numpy array
+            image_np = np.frombuffer(image_file.read(), np.uint8)
+            # Decode the image as grayscale
+            img = cv2.imdecode(image_np, cv2.IMREAD_GRAYSCALE)
+        except Exception as e:
+            print(f"Error reading or decoding image: {e}")
+            raise ValueError("Invalid or corrupted image file.")
+
+        if img is None:
+            raise ValueError("Could not decode image. File may be empty or corrupted.")
+
+        # 1. Apply Fast Fourier Transform (FFT)
+        f = np.fft.fft2(img)
+        fshift = np.fft.fftshift(f)
+        magnitude_spectrum = 20 * np.log(np.abs(fshift) + 1) # Add 1 to avoid log(0)
+
+        # Normalize the magnitude spectrum to be in the range [0, 255]
+        magnitude_spectrum = cv2.normalize(magnitude_spectrum, None, 0, 255, cv2.NORM_MINMAX)
+        magnitude_spectrum = np.uint8(magnitude_spectrum)
+        
+        # 2. Apply torchvision transforms
+        image_tensor = self.transform(magnitude_spectrum)
+        
+        # Add a batch dimension and move to the correct device
+        image_tensor = image_tensor.unsqueeze(0).to(self.device)
+        
+        return image_tensor
+
+# Create a single instance of the preprocessor
+preprocessor = ImagePreprocessor()

features/real_forged_classifier/routes.py

@@ -0,0 +1,37 @@
+from fastapi import APIRouter, File, UploadFile, HTTPException, status
+from fastapi.responses import JSONResponse
+
+# Import the controller instance
+from controller import controller
+
+# Create an API router
+router = APIRouter()
+
+@router.post("/classify_forgery", summary="Classify an image as Real or Fake")
+async def classify_image_endpoint(image: UploadFile = File(...)):
+    """
+    Accepts an image file and classifies it as 'real' or 'fake'.
+
+    - **image**: The image file to be classified (e.g., JPEG, PNG).
+
+    Returns a JSON object with the classification and a confidence score.
+    """
+    # Check for a valid image content type
+    if not image.content_type.startswith("image/"):
+        raise HTTPException(
+            status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
+            detail="Unsupported file type. Please upload an image (e.g., JPEG, PNG)."
+        )
+
+    # The controller expects a file-like object, which `image.file` provides
+    result = controller.classify_image(image.file)
+
+    if "error" in result:
+        # If the controller returned an error, forward it as an HTTP exception
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=result["error"]
+        )
+    
+    return JSONResponse(content=result, status_code=status.HTTP_200_OK)
+

features/text_classifier/controller.py

@@ -60,12 +60,12 @@ async def handle_file_upload(file: UploadFile):
     try:
         file_contents = await extract_file_contents(file)
         if len(file_contents) > 10000:
-            raise HTTPException(status_code=413, detail="Text must be less than 10,000 characters")
+            return {"status_code": 413, "detail": "Text must be less than 10,000 characters"}
 
         cleaned_text = file_contents.replace("\n", " ").replace("\t", " ").strip()
         if not cleaned_text:
             raise HTTPException(status_code=404, detail="The file is empty or only contains whitespace.")
-      
+        # print(f"Cleaned text: '{cleaned_text}'")  # Debugging statement
         label, perplexity, ai_likelihood = await asyncio.to_thread(classify_text, cleaned_text)
         return {
             "content": file_contents,
@@ -102,12 +102,15 @@ async def handle_sentence_level_analysis(text: str):
             "ai_likelihood": ai_likelihood
         })
 
-    return {"analysis": results}# Analyze each sentence from uploaded file
+    return {"analysis": results}
+
+# Analyze each sentence from uploaded file
 async def handle_file_sentence(file: UploadFile):
     try:
         file_contents = await extract_file_contents(file)
         if len(file_contents) > 10000:
-            raise HTTPException(status_code=413, detail="Text must be less than 10,000 characters")
+            # raise HTTPException(status_code=413, detail="Text must be less than 10,000 characters")
+            return {"status_code": 413, "detail": "Text must be less than 10,000 characters"}
 
         cleaned_text = file_contents.replace("\n", " ").replace("\t", " ").strip()
         if not cleaned_text:

features/text_classifier/preprocess.py

@@ -1,4 +1,4 @@
-import fitz  # PyMuPDF
+from pypdf import PdfReader  
 import docx
 from io import BytesIO
 import logging
@@ -15,18 +15,16 @@ def parse_docx(file: BytesIO):
 
 def parse_pdf(file: BytesIO):
     try:
-        doc = fitz.open(stream=file, filetype="pdf")
+        doc = PdfReader(file)
         text = ""
-        for page_num in range(doc.page_count):
-            page = doc.load_page(page_num)
-            text += page.get_text()
-        return text
+        for page in doc.pages:
+            text += page.extract_text()
+        return text 
     except Exception as e:
         logging.error(f"Error while processing PDF: {str(e)}")
         raise HTTPException(
             status_code=500, detail="Error processing PDF file")
 
-
 def parse_txt(file: BytesIO):
     return file.read().decode("utf-8")
 

requirements.txt

@@ -15,6 +15,6 @@ tensorflow
 opencv-python
 pillow
 scipy
-fitz
+pypdf
 frontend
 tools

test.md

@@ -0,0 +1,31 @@
+
+**Update: Edited & AI-Generated Content Detection – Project Plan**
+
+### 🔍 Phase 1: Rule-Based Image Detection (In Progress)
+
+We're implementing three core techniques to individually flag edited or AI-generated images:
+
+* **ELA (Error Level Analysis):** Highlights inconsistencies via JPEG recompression.
+* **FFT (Frequency Analysis):** Uses 2D Fourier Transform to detect unnatural image frequency patterns.
+* **Metadata Analysis:** Parses EXIF data to catch clues like editing software tags.
+
+ These give us visual + interpretable results for each image, and currently offer \~60–70% accuracy on typical AI-edited content.
+
+---
+
+###  Phase 2: AI vs Human Detection System (Coming Soon)
+
+**Goal:** Build an AI model that classifies whether content is AI- or human-made — initially focusing on **images**, and later expanding to **text**.
+
+**Data Strategy:**
+
+* Scraping large volumes of recent AI-gen images (e.g. SDXL, Gibbli, MidJourney).
+* Balancing with high-quality human images.
+
+**Model Plan:**
+
+* Use ELA, FFT, and metadata as feature extractors.
+* Feed these into a CNN or ensemble model.
+* Later, unify into a full web-based platform (upload → get AI/human probability).
+
+