Data/profile_data.md

About Me

Hi! I am Mohit Gupta, a passionate AI engineer from Delhi, India. I had done my B.Tech in Artificial Intelligence and Data Science which is specialization of Computer Science from University School of Automation & Robotics, GGSIPU. My Btech completed in 2025. I am smart, playfull and deeply enthusiastic about building impactful AI systems. I enjoy working on multimodal AI, Retrieval-Augmented Generation (RAG) systems, and scalable deployments, and I’m constantly exploring advancements in generative AI and large language models. Outside of tech, I am an athletic person who enjoys playing basketball and listening to music. I am always eager to hustle, explore, and contribute to impactful AI-driven projects.

---

Profile Links

• GitHub: https://github.com/MohitGupta0123
• LinkedIn: https://www.linkedin.com/in/mohitgupta012
• Kaggle: https://www.kaggle.com/mohitgupta12
• LeetCode: https://leetcode.com/u/MohitGupta012/

Skills

I possess expertise across a broad range of technologies and tools:

• Programming Languages: Python, C++
• Developer Tools: HTML/CSS, Tailwind, Streamlit, FastAPI, GitHub, SQL, MongoDB, OOPS, DBMS
• Machine Learning & AI: Machine Learning and Deep Learning Algorithms, AutoML, Data Analysis, Power BI, Tableau, PyTorch, TensorFlow, Natural Language Processing (NLP), RAG, LangChain, LangGraph, ChromaDB, CrewAI, AutoGen, OpenCV, Time Series Forecasting, LLM Finetuning, Generative AI, Diffusion Models
• MLOps & Cloud: MLflow, Docker, Kubernetes, Prometheus, Grafana, Amazon AWS S3

---

Experience

Machine Learning Research Intern – Indian Institute of Technology Delhi (Feb 2025 – Present)

• Led development of an AI-powered MRI acceleration system using diffusion models and FFT, reducing scan times from 30 minutes to a few minutes.
• Engineered and optimized deep learning models in Python and PyTorch to reconstruct high-fidelity MRI images from highly limited data (4x acceleration).

Machine Learning Intern – ImagoAI (Jan 2025 – Feb 2025)

• Boosted high-value toxin prediction accuracy by 38% using regression-aware augmentation (SMOTE, MixUp) on imbalanced hyperspectral data.
• Enhanced model R² by 0.25 by developing ViT and branched CNN architectures for hyperspectral data, outperforming baseline Dense/CNN models.

---

Education

B.Tech in Artificial Intelligence & Data Science – University School Of Automation & Robotics, GGSIPU (Sep 2021 – Apr 2025)

• CGPA: 9.032

---

Achievements & Extracurriculars

• Led a team in the Amazon ML Challenge 2024, achieving a top 0.17% rank in a competitive hackathon by implementing a transformer-based model with an F1 score of 0.57341.
• Able to reach in top 17 teams out of 1000+ teams from All India in GSTN Analytics Hackathon Organized be GSTN and received 25000 Indian Rupees.
• Active on GitHub, Kaggle, and LeetCode, continuously contributing to open-source projects and competitive coding challenges.

---

Projects

1. 💊 Medical Agentic AI Assistant – RAG + Agentic AI

The Medical Assistant – RAG + Agentic AI is an end-to-end healthcare assistant that combines Retrieval-Augmented Generation (RAG) for medical Q&A, agentic workflows for handling patient operations, and a real-time admin dashboard for medical staff. The system integrates a Streamlit frontend, FastAPI backend, and SQLite/Supabase database, with embeddings powered by SentenceTransformers and a FAISS vector store for semantic search. Both frontend and backend are Dockerized and deployable on Hugging Face Spaces.

Live Demo (Frontend) - https://medical-agentic-aibot-mohit-gupta.streamlit.app/ Backend API - https://mohitg012-medical-bot-agentic-ai.hf.space

---

Project Overview

This project enables both patients and doctors to interact seamlessly in a digital healthcare workflow. Patients can register, book appointments, and check medicine availability, while doctors can view summaries, KPIs, and records on a centralized dashboard. The RAG chatbot provides answers with citations and page numbers from a medical book, making it a reliable knowledge assistant. The system is designed to be offline-friendly with bundled embeddings and can run in air-gapped environments.

---

Features

• RAG Chatbot – Ask medical queries and get answers with citations & page numbers from the medical book.
• Agentic Assistant – Automates patient registration, appointment confirmation, medicine checks, and case summarization.
• Admin Dashboard – Real-time KPIs, searchable tables, and CSV export for patients, doctors, and medicines.
• HF Token Pass-Through – Secure token handling via frontend without server storage.
• Offline-Friendly Embeddings – Bundled all-MiniLM-L6-v2 enables local inference without internet.
• Multi-Deployment Ready – Local (Streamlit + FastAPI), Docker Compose, or Hugging Face Spaces.

---

Tech Stack

• Frontend: Streamlit
• Backend: FastAPI (served via Uvicorn)
• Database: SQLite / Supabase
• Vector Store: FAISS
• Embeddings: SentenceTransformers (all-MiniLM-L6-v2)
• Containerization: Docker, Docker Compose
• Deployment: Hugging Face Spaces

---

Architecture & Workflow

1. Frontend (Streamlit): Collects Hugging Face token, displays chatbot UI, and shows results/dashboards.
2. Backend (FastAPI): Handles RAG queries, agent workflows, CRUD APIs for patients/doctors/medicines.
3. Database (SQLite/Supabase): Stores structured data for patients, doctors, and medicines.
4. RAG Pipeline:
   • PDF preprocessing → chunking → embeddings (SBERT) → FAISS index
   • Semantic retrieval + re-ranking → Answer generation with citations
5. Agent Orchestrator: Routes user intents to tools (registration, medicine checks, summarization) or fallback RAG.
6. Deployment: Docker Compose for local dev, Hugging Face Spaces for production-ready hosting.

---

Results & Impact

The assistant provides trusted, citation-backed answers and streamlines hospital operations. Doctors and administrators can view live KPIs, track patient flow, and manage medicine availability efficiently. By combining RAG with agentic workflows, the project goes beyond Q&A to become a workflow automation tool for digital healthcare.

---

Future Enhancements

• Support for multiple medical books/corpora with a corpus switcher.
• Doctor scheduling calendar with slot booking.
• User-specific chat history with consented analytics.
• Evaluation framework for retrieval accuracy and answer quality.
• Cloud-native deployments (AWS/GCP) with CI/CD for scalability.

Project's Detailed Readme

💊 Medical Assistant – RAG + Agentic AI (Streamlit + FastAPI + Supabase)

An end-to-end medical assistant that combines RAG Q&A, agentic workflows (registration, appointment, medicines, summaries), and an admin dashboard. Frontend in Streamlit, backend in FastAPI, persistence with SQLite/Supabase, and embeddings via SentenceTransformers (offline compatible). This Project is real time project in which real time registration of patients, appointment confirmation, medicine stock status and patient case summaries can be seen.

Deployed Link: https://medical-agentic-aibot-mohit-gupta.streamlit.app/

Deployed Backend Link: https://mohitg012-medical-bot-agentic-ai.hf.space

---

✨ Features

• RAG Chatbot: Ask medical questions with citations & page numbers from your medical book.
• Agentic Assistant: One prompt to register patients, confirm appointments, check medicines, or summarize cases.
• Admin Dashboard: Live KPIs + tables for patients, doctors, medicines, with CSV export.
• HF Token Pass-Through: Frontend collects your Hugging Face token and forwards it to backend per request.
• Offline-friendly Embeddings: Bundled all-MiniLM-L6-v2 supports air-gapped runs.

---

🏗️ Architecture

Streamlit (Frontend)
   ├─ collects HF token (never stored on server)
   ├─ hits FastAPI endpoints
   └─ renders chat, agent results, dashboard

FastAPI (Backend)
   ├─ /query (RAG) → retriever + reader over medical_book.pdf
   ├─ /orchestrator_query (Agent) → routes to tools
   ├─ /register_patient, /check_registration_status
   ├─ /medicine_availability, /release_stale_doctors
   ├─ /summarize_case/{id}
   └─ /admin/* read APIs for dashboard

Storage
   ├─ SQLite or Supabase (patients, doctors, medicines)
   ├─ FAISS index + chunk metadata in Artifacts/
   └─ PDF and page images in Artifacts/raw_pdf & page_images

Models
   └─ SentenceTransformer: models/all-MiniLM-L6-v2

---

📁 Repository Layout (key folders)

Medical-Assistant/
├── 📂 Artifacts/                       # Data artifacts for RAG
│   ├── 📂 embeddings/                  # FAISS index + metadata
│   │   ├── 📄 faiss_index.bin
│   │   └── 📄 metadata.pkl
│   ├── 🖼️ images/                      # Extracted diagrams/tables
│   │   └── 🖼️ medical_book_pageXXX_imgX.jpeg
│   ├── 🖼️ page_images/                 # Page-level snapshots for citations
│   │   └── 🖼️ medical_book_pageXXX_snapshot.png
│   ├── 📂 processed_text/              # Chunked text + metadata
│   │   └── 📄 chunks_metadata.json
│   └── 📂 raw_pdf/
│       └── 📄 medical_book.pdf

├── 📂 DataBase/
│   └── 🗄️ medical_assistant.db         # Local SQLite DB (if not using Supabase)

├── 🎨 Frontend/                        # Streamlit UI
│   ├── 🐍 Main.py                      # Home page (health + nav)
│   ├── ⚙️ config.py                    # BASE_URL settings
│   ├── 📂 pages/                       # Streamlit multipage structure
│   │   ├── 🐍 1_Medical_Chatbot.py
│   │   ├── 🐍 2_Registration_And_Operations.py
│   │   ├── 🐍 3_Agent_Bot.py
│   │   └── 🐍 4_Dashboard.py
│   ├── 📄 requirements.txt             # Frontend dependencies
│   ├── 🐳 Dockerfile                   # Frontend container
│   └── 📄 .dockerignore

├── ⚙️ Src/                             # Backend (FastAPI + services)
│   ├── 📂 api/
│   │   └── 🐍 fastapi_app.py           # FastAPI entrypoint
│   ├── 📂 rag/                         # Retrieval-Augmented Generation
│   │   ├── 🐍 pdf_utils.py
│   │   ├── 🐍 preprocess.py
│   │   ├── 🐍 retriever.py
│   │   ├── 🐍 rag_pipeline.py
│   │   └── 🐍 embed_store.py
│   ├── 📂 agent/                       # Orchestrator + tools
│   │   ├── 🐍 agent_executor.py
│   │   ├── 🐍 gemma_chat_llm.py
│   │   ├── 🐍 orchestrator.py
│   │   └── 🐍 tools.py
│   ├── 📂 services/                    # Business logic (DB, doctors, patients…)
│   │   ├── 🐍 db.py
│   │   ├── 🐍 doctor_assignment.py
│   │   ├── 🐍 doctor_service.py
│   │   ├── 🐍 medicine_service.py
│   │   ├── 🐍 patient_service.py
│   │   └── 🐍 summarizer.py
│   ├── 📄 requirements.txt             # Backend dependencies
│   ├── 🐳 Dockerfile                   # Backend container
│   └── 📄 .dockerignore

├── 🤖 models/                          # Local SentenceTransformer model
│   └── 📂 all-MiniLM-L6-v2/
│       ├── 📄 model.safetensors
│       ├── 📄 config.json
│       ├── 📄 tokenizer.json
│       └── 📄 vocab.txt
│       ... (other ST files)

├── 📓 Notebooks/                       # Dev & exploration
│   ├── 📓 01_data_preprocessing.ipynb
│   ├── 📓 02_embeddings_rag.ipynb
│   └── 📓 03_rag_pipeline.ipynb

├── ⚙️ .gitignore
├── ⚙️ .dvcignore
├── ⚙️ .dockerignore
├── 🐳 docker-compose.yml               # Compose for frontend + backend
├── 🐳 dockerfile                       # (Optional) project-level Docker
├── 📄 Data.dvc                         # DVC tracking file
├── 📝 Future_steps_for_this_project.txt
├── 📘 Readme.md                        # Project documentation
└── 📄 requirements.txt                 # Global deps (if needed)

---

🔌 API Endpoints (FastAPI)

• GET / – health/info
• GET /docs – Swagger UI
• GET /query?q=... – RAG answer with references
• GET /orchestrator_query?q=... – Agent router
• POST /register_patient – JSON: {name, age, reason}
• POST /check_registration_status – JSON: {name}
• GET /medicine_availability?name=...
• POST /release_stale_doctors
• GET /summarize_case/{patient_id}
• GET /admin/patients | /admin/doctors | /admin/medicines

Auth: Frontend forwards Authorization: Bearer <HF_TOKEN> to backend for any HF-model calls.

---

⚙️ Prerequisites

• Python 3.10+
• (Optional) Docker / docker-compose
• (Optional) Supabase credentials
• A Hugging Face token with access to models you call from the backend

---

🚀 Quickstart (Local, 2 terminals)

1) Backend (FastAPI)

cd Src
python -m venv .venv && source .venv/bin/activate    # (Windows: .venv\Scripts\activate)
pip install -r requirements.txt

# ENV (see “Environment Variables” below)
export SUPABASE_URL=...          # optional
export SUPABASE_KEY=...          # optional
export HF_API_TIMEOUT=60         # optional

uvicorn api.fastapi_app:app --host 0.0.0.0 --port 8000 --reload

2) Frontend (Streamlit)

cd Frontend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Point frontend to backend:
export BASE_URL="http://localhost:8000"

streamlit run Main.py

Open http://localhost:8501 → enter your HF token in the sidebar → use the Navigation buttons.

---

🌐 Deployment Options

A) Docker Compose (Frontend + Backend)

docker compose up --build

Browse http://localhost:8501.

B) Hugging Face Spaces (Docker)

• Backend Space → type: Docker; Src/Dockerfile as root.
• Frontend Space → type: Docker; Frontend/Dockerfile as root.
• Set CORS in FastAPI to allow the frontend origin.
• In the frontend UI, set Backend URL to your backend Space URL.
• Enter your HF token in the frontend sidebar.

---

🔑 Environment Variables

Backend (FastAPI):

• SUPABASE_URL (optional) – use Supabase instead of SQLite
• SUPABASE_KEY (optional) – service key/token
• HF_API_TIMEOUT (optional, default=60) – timeout for HF calls
• (Project-specific) any model name/endpoint your tools require

Frontend (Streamlit):

• BASE_URL – FastAPI URL (e.g., http://localhost:8000 or your HF Space URL)

HF Token is not stored as an env—users paste it in the sidebar; the app forwards it per request.

.env example:

# Backend
SUPABASE_URL=https://xyzcompany.supabase.co
SUPABASE_KEY=your-service-role-key
HF_API_TIMEOUT=60

# Frontend
BASE_URL=http://localhost:8000

---

📚 Data & Artifacts

• Artifacts/raw_pdf/medical_book.pdf – source
• Artifacts/processed_text/chunks_metadata.json – chunk map
• Artifacts/embeddings/faiss_index.bin – FAISS index
• Artifacts/page_images/*.png – page snapshots for citations
• Artifacts/images/* – extracted diagrams/tables

Use the included notebooks in Notebooks/ to (re)build chunks and embeddings:

• 01_data_preprocessing.ipynb
• 02_embeddings_rag.ipynb
• 03_rag_pipeline.ipynb

---

🧠 RAG Pipeline (high-level)

1. Preprocessing: split PDF into pages → OCR/parse → chunk text (windowing + overlap).
2. Embedding: all-MiniLM-L6-v2 (bundled) → FAISS index + metadata.
3. Retrieval: semantic search top-k + re-rank; attach page links.
4. Generation: LLM forms answer, citing source pages.

---

🤖 Agent Orchestrator

• Input → intent classification → tool routing:

  • register_patient → doctor assignment (Gemma-powered reasoning)
  • check_registration_status / confirm_appointment
  • medicine_availability
  • summarize_case
  • RAG fallback for open questions

• Returns typed payloads that the frontend renders nicely (cards, expanders, downloads).

---

🖥️ Frontend Pages

• Home: health checks (/, /docs, small auth ping), quick actions, navigation.
• Medical Chatbot: chat UI with optional References (page & link).
• Registration & Operations: forms for register, appointment, medicines, summary.
• Dashboard: KPIs, charts, searchable tables, CSV downloads, Release stale doctors.

---

🧪 Smoke Tests

After starting both services:

• GET {BASE_URL}/ → returns server info JSON
• GET {BASE_URL}/docs → OpenAPI UI
• RAG test from Home → “Run sample RAG”
• Agent tests from Agent Bot sidebar examples
• Dashboard loads counts and tables

---

🔐 Security Notes

• HF token is only kept in Streamlit session_state (per browser session).

---

🛠️ Troubleshooting

• Home → Backend Status

  • Root/Docs down → backend not reachable; check BASE_URL, ports, CORS.
  • Auth Test failed → invalid/missing HF token; backend rejecting Authorization header; or model not accessible.

• Dashboard empty → ensure /admin/* endpoints return arrays under items or raw arrays.

• Summarize case “No summary” → check patient id exists; inspect backend logs.

• Embeddings mismatch → rebuild FAISS with Notebooks/02_embeddings_rag.ipynb.

---

🛣️ Roadmap (Future_steps_for_this_project.txt hooks)

• Multi-book/corpus support with collection switcher
• Per-user chat history + consented analytics
• Doctor scheduling calendar with slots
• Evaluations: retrieval metrics & answer grading set

---

🙏 Acknowledgements

• SentenceTransformers all-MiniLM-L6-v2
• FastAPI, Streamlit, FAISS
• (Optional) Supabase

---

2. 📄 PDF Translate – Automated PDF Translation & Redaction

The PDF Translate Project is a complete pipeline for automated PDF translation, OCR correction, and selective redaction while preserving the original layout, fonts, and colors. It supports English ↔ Hindi translations out of the box and can be extended to other languages. The system integrates OCR (ocrmypdf + Tesseract), PyMuPDF text-layer analysis, Google Translate (googletrans), and custom overlay logic to rebuild documents with native-looking translated text.

Live Demo (Hugging Face Space): https://mohitg012-pdf-translation.hf.space/

---

Project Overview

This project enables users to translate PDFs while retaining formatting fidelity and apply secure redactions/masking in sensitive documents. It processes both scanned PDFs (via OCR) and digital PDFs (via text-layer extraction). Multiple placement modes (span, line, block, hybrid) allow for column-aware translation in multi-column or tabular layouts. Users can run the pipeline via CLI, Python API, or Docker, making it flexible for researchers, publishers, and enterprises dealing with multilingual PDF archives.

---

Features

• Bidirectional Translation: English ↔ Hindi with auto-detection, extendable to other scripts.
• OCR Support: Cleans up scanned/low-quality PDFs with ocrmypdf.
• Layout Preservation: Maintains font size, style, and colors in translated output.
• Hybrid Column Handling: Column/table-aware segmentation improves accuracy in structured PDFs.
• Redaction & Masking: Dynamic fill color ensures visual consistency (dark text → white fill, light text → black fill).
• Overlay Options: Supports both JSON-driven overlays and auto-overlay generation; can render text as searchable text or high-DPI images.
• Multi-Mode Processing: Run span/line/block/hybrid/overlay individually or batch them with all mode.
• CLI & Python API: Unified script for quick translation and modular API for custom pipelines.
• Dockerized Execution: Easily portable across systems with container support.

---

Tech Stack

• PDF Handling: PyMuPDF (fitz)
• OCR: ocrmypdf + Tesseract
• Translation: Google Translate (googletrans)
• Rendering: Pillow (PIL) for overlay text/image placement
• Containerization: Docker
• Programming Language: Python 3.12

---

Architecture & Workflow

1. OCR (optional): Clean scanned PDFs and add searchable text layers.
2. Text Extraction: Parse spans, lines, and blocks from PDF text layers.
3. Hybrid Segmentation: Detect table cells/multi-column text using X-gap heuristics.
4. Style Sampling: Map original fonts, sizes, and colors for faithful reproduction.
5. Translation: Translate spans/lines/blocks using googletrans.
6. Redaction: Erase sensitive text via mask or true redaction annotations.
7. Overlay: Insert translated text back into document with textboxes or image tiles.
8. Output: Generate multiple PDFs per mode + a bundled ZIP archive.

---

Results & Impact

The project enables high-quality, layout-preserving PDF translation for legal, educational, and medical use cases. It significantly reduces the manual effort required for translation while maintaining compliance through built-in redaction features. The ability to run locally (with Docker) ensures data security for sensitive documents.

---

Future Enhancements

• Support for official translation APIs (Google Cloud, DeepL, Azure) for production reliability.
• Advanced table detection & alignment heuristics for complex documents.
• Language packs & font auto-selection for broader multilingual coverage.
• A Streamlit drag-and-drop UI for user-friendly PDF uploads.
• Batch translation pipeline for large corporate archives.

Project's Detailed Readme

PDF Translate — Automated PDF Translation & Redaction (Python)

DEPLOYED LINK: https://mohitg012-pdf-translation.hf.space/

Automate high-quality translation and selective redaction of PDFs while preserving layout, font sizing, and colors. The project blends:

• OCR (via ocrmypdf/Tesseract) for scanned or low-quality PDFs
• Text-layer analysis (PyMuPDF/fitz) for precise boxes, spans, lines, and blocks
• AI translation (Google Translate via googletrans)
• Overlay & drawing logic to put translated text back exactly where it belongs
• Redaction/masking that adapts to background/foreground contrast

It supports English ↔︎ Hindi out of the box and can be extended to other scripts.

---

Key features

1. Language translation English ↔︎ Hindi supported; auto direction detection available. Extendable by swapping fonts & translation parameters.

2. Text layer analysis Extracts spans, lines, blocks, and a hybrid (column/table-aware) mode to keep text where it belongs even in multi-column pages and tables.

3. OCR for scanned PDFs Uses ocrmypdf to produce a clean, searchable PDF prior to analysis.

4. Style preservation Transfers font size & color from the original objects to the translated overlay so the result looks native.

5. Smart redaction / masking

   • redact (true PDF redactions) or mask (draw filled rectangles).
   • Fill color is chosen dynamically from surrounding luminance (dark text → white fill, light text → black fill) to maintain visual consistency.

6. Overlay options

   • Generate overlays automatically from the current document, or
   • Drive from JSON (page + bbox + translated_text) to paint exactly what you want.
   • Render as real text or high-DPI images (for bulletproof glyph coverage).

7. CLI & Python API A single unified script provides modes for span, line, block, hybrid, overlay, and all (batch all modes + zip).

8. Error correction helpers Normalizes whitespace and punctuation spacing; de-noises OCR artifacts where possible.

9. Multiple input formats Any format PyMuPDF can open (primarily PDF; images should be PDF-wrapped before processing—ocrmypdf handles this).

10. Security & compliance Use local OCR and redaction; redact before writing translated text to prevent data leaks in sensitive areas.

---

How it works

1. OCR pass (optional but recommended) ocrmypdf runs with language packs (e.g., hin+eng) and deskew/rotate to create a clean text layer.

2. Text extraction & structure building PyMuPDF extracts raw dicts of blocks/lines/spans; the code constructs:

   • basic spans, lines, blocks
   • hybrid blocks that split each raw line into segments by significant X-gaps (detects table cells / columns)

3. Style sampling A lightweight index of original color & font size is built and transferred to translated objects using IoU/nearest heuristics.

4. Translation Uses googletrans (Google Translate) with direction:

   • hi->en, en->hi, or auto (detect from dominant script).

5. Erasure / Redaction Depending on mode:

   • mask: draw filled rectangles (per-box adaptive fill)
   • redact: actual redaction annotations applied page-wide

6. Overlay The translated text is written back using either:

   • Text boxes (insert_textbox with font fallback), or
   • High-DPI image tiles rendered via PIL for maximum glyph fidelity.

7. All-mode Runs span, line, block, hybrid, and optionally overlay, writing separate PDFs and a combined ZIP.

---

Project structure

PDF-TRANSLATOR
├── app.py                      # (Optional) app entry (e.g., Streamlit)
├── PDF_Translate/              # Modular library
│   ├── __init__.py
│   ├── cli.py
│   ├── constants.py
│   ├── hybrid.py
│   ├── ocr.py
│   ├── overlay.py
│   ├── pipeline.py
│   ├── textlayer.py
│   └── utils.py
├── pdf_translate_unified.py    # Unified CLI/API (span/line/block/hybrid/overlay/all)
├── assets/
│   └── fonts/                  # Pre-bundled font files (English & Devanagari)
│       ├── NotoSans-Regular.ttf
│       ├── NotoSans-Bold.ttf
│       ├── NotoSansDevanagari-Regular.ttf
│       ├── NotoSansDevanagari-Bold.ttf
│       ├── TiroDevanagariHindi-Regular.ttf
│       ├── Hind-Regular.ttf
│       ├── Karma-Regular.ttf
│       └── Mukta-Regular.ttf
├── samples/
│   ├── Test1.pdf
│   ├── Test1_translated.pdf
│   ├── Test2.pdf
│   ├── Test2_translated.pdf
│   ├── Test3.pdf
│   └── Test3_translated.pdf
├── output_pdfs/                # Generated outputs land here
├── temp/                       # OCR/rasterization scratch (e.g., ocr_fixed.pdf)
├── requirements.txt
├── Dockerfile
└── Readme.md                   # (this document)

---

Installation

1) System prerequisites

• Python: 3.12 recommended
• Tesseract & ocrmypdf: required for OCR
• Ghostscript + qpdf: required by ocrmypdf

Ubuntu/Debian

sudo apt update
sudo apt install -y tesseract-ocr tesseract-ocr-hin ocrmypdf ghostscript qpdf

macOS (Homebrew)

brew install tesseract ocrmypdf ghostscript qpdf

Windows

• Install Tesseract (UB Mannheim build recommended) and make sure tesseract.exe is on PATH.
• Install Ghostscript and qpdf; add to PATH.
• Install ocrmypdf via pip (will use the system binaries above).

2) Python packages

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt

---

Quick start

Translate a PDF (English → Hindi) using all modes:

python pdf_translate_unified.py \
  --input samples/Test3.pdf \
  --output output_pdfs/result.pdf \
  --mode all \
  --translate en->hi

What you get:

• result.span.pdf
• result.line.pdf
• result.block.pdf
• result.hybrid.pdf
• result.overlay.pdf
• result_all_methods.zip bundling the above

---

Command-line usage

python pdf_translate_unified.py --help

Required

• --input / -i: path to your source PDF
• --output / -o: output path (for --mode all, this is the base name)

Modes

• --mode {span,line,block,hybrid,overlay,all} (default: all)

When to use which

• span – ultra-fine placement, best for mixed inline styles; can look busy
• line – per line; balances fidelity & readability
• block – per paragraph/block; often the cleanest look
• hybrid – column/table-aware; great for multi-column layouts and tabular data
• overlay – paint from a JSON (see below) or from --auto-overlay
• all – run several modes and zip them for comparison

OCR options

• --lang (default: hin+eng) – languages passed to ocrmypdf
• --dpi (default: 1000) – --image-dpi/--oversample for ocrmypdf
• --optimize (default: 3) – ocrmypdf --optimize level
• --skip-ocr – use the input PDF as-is (not recommended for scanned PDFs)

Translation direction

• --translate {hi->en,en->hi,auto} (default: hi->en)

Redaction / masking

• --erase {redact,mask,none} (default: redact)
• --redact-color r,g,b – only used when a fixed color is required; otherwise the tool automatically picks black or white from context.

Fonts

• --font-en-name (logical name; default NotoSans)
• --font-en-path (path to TTF; default bundled Noto Sans)
• --font-hi-name (default NotoSansDevanagari)
• --font-hi-path (path to Devanagari TTF; defaults to Base14 helv if missing)

Overlay-specific knobs

• --overlay-json /path/to/text_data.json
• --auto-overlay – build overlay items from the doc and chosen --translate
• --overlay-render {image,textbox} (default image)
• --overlay-align {0,1,2,3} – left/center/right/justify (justify only for textbox)
• --overlay-line-spacing (default 1.10)
• --overlay-margin-px (default 0.1)
• --overlay-target-dpi (default 600)
• --overlay-scale-x|y, --overlay-off-x|y – fix geometry if the JSON was created on a near-duplicate PDF

Example commands

1) English → Hindi (hybrid mode)

python pdf_translate_unified.py -i samples/Test1.pdf -o output_pdfs/t1.hybrid.pdf \
  --mode hybrid --translate en->hi

2) Hindi → English (block mode, masking)

python pdf_translate_unified.py -i samples/Test2.pdf -o output_pdfs/t2.block.pdf \
  --mode block --translate hi->en --erase mask

3) Overlay from JSON with real text (keep searchable layer)

python pdf_translate_unified.py -i samples/Test3.pdf -o output_pdfs/t3.overlay.pdf \
  --mode overlay --overlay-json text_data.json --overlay-render textbox \
  --overlay-align 0 --overlay-line-spacing 1.15

4) Auto-overlay (no JSON; build from doc)

python pdf_translate_unified.py -i samples/Test3.pdf -o output_pdfs/t3.overlay.pdf \
  --mode overlay --auto-overlay --translate en->hi

---

Fonts

For Devanagari, the bundled fonts work well:

• NotoSansDevanagari-Regular.ttf
• TiroDevanagariHindi-Regular.ttf
• Others: Hind, Mukta, Karma

Specify alternatives via --font-hi-path. For English, NotoSans is the default.

---

Overlay JSON

You can drive the overlay precisely with a JSON file:

[
  {
    "page": 0,
    "bbox": [72.0, 144.0, 270.0, 180.0],
    "translated_text": "Hello world",
    "fontsize": 11.5
  }
]

• Required: page, bbox ([x0,y0,x1,y1] in PDF points), translated_text
• Optional: fontsize (used as a base; the renderer will fit it)

Run:

python pdf_translate_unified.py -i in.pdf -o out.pdf \
  --mode overlay --overlay-json text_data.json

Geometry mismatch? If your JSON came from a slightly different source PDF:

• --overlay-scale-x|y to scale all boxes
• --overlay-off-x|y to shift them

---

Python API (modular usage)

You can call the building blocks directly from Python for custom pipelines.

from pdf_translate_unified import (
    extract_original_page_objects, ocr_fix_pdf, build_base,
    resolve_font, run_mode, build_overlay_items_from_doc
)

input_pdf = "samples/Test3.pdf"
output_pdf = "output_pdfs/demo_all.pdf"
translate_direction = "en->hi"

# 1) Style index from original (pre-OCR) for accurate color/size
orig_index = extract_original_page_objects(input_pdf)

# 2) OCR pass
src_fixed = ocr_fix_pdf(input_pdf, lang="hin+eng", dpi="1000", optimize="3")

# 3) Create source/output documents with background preserved
src, out = build_base(src_fixed)

# 4) Configure fonts
en_name, en_file = resolve_font("NotoSans", "assets/fonts/NotoSans-Regular.ttf")
hi_name, hi_file = resolve_font("NotoSansDevanagari", "assets/fonts/TiroDevanagariHindi-Regular.ttf")

# 5) Optional: auto-build overlay items
overlay_items = build_overlay_items_from_doc(src, translate_direction)

# 6) Run any mode (or "all")
run_mode(
    mode="all",
    src=src, out=out,
    orig_index=orig_index,
    translate_dir=translate_direction,
    erase_mode="redact",
    redact_color=(1,1,1),
    font_en_name=en_name, font_en_file=en_file,
    font_hi_name=hi_name, font_hi_file=hi_file,
    output_pdf=output_pdf,
    overlay_items=overlay_items,
    overlay_render="image",
    overlay_target_dpi=600
)

---

Docker

Build:

docker build -t pdf-translate .

Run (mount your PDFs):

docker run --rm -v "$PWD:/work" pdf-translate \
  python pdf_translate_unified.py -i /work/samples/Test3.pdf \
  -o /work/output_pdfs/result.pdf --mode all --translate en->hi

---

Samples & outputs

See samples/ for input PDFs and _translated.pdf examples. Recent runs create files under output_pdfs/, including individual mode outputs and a zipped bundle like:

result_YYYYMMDD-HHMMSS.all.block.pdf
result_YYYYMMDD-HHMMSS.all.hybrid.pdf
result_YYYYMMDD-HHMMSS.all.line.pdf
result_YYYYMMDD-HHMMSS.all.overlay.pdf
result_YYYYMMDD-HHMMSS.all.span.pdf
result_YYYYMMDD-HHMMSS_all_methods.zip

---

Limitations & notes

• googletrans relies on unofficial endpoints; for production, consider swapping in an official translation API (e.g., Google Cloud Translate, Azure, DeepL).
• OCR quality determines downstream accuracy; garbage in → garbage out.
• Complex vector art or text on curves isn’t reflowed; overlay is rectangular.
• True layout editing (re-wrapping across pages) is out of scope by design.

---

Contributing

Issues and PRs are welcome!:

• New language/font packs & font auto-selection rules
• Pluggable translator backends
• Better table detection & alignment heuristics
• Streamlit UX in app.py for drag-and-drop PDFs

Please run ruff/black (if configured) and include before/after sample PDFs for visual changes.

---

Acknowledgements

• PyMuPDF (fitz) for robust PDF parsing/rendering
• ocrmypdf + Tesseract for OCR
• Pillow (PIL) for high-DPI text rendering in image overlays
• Google Translate (via googletrans) for quick translation prototyping

---

3. Fraud Detection MLOps Pipeline

The Fraud Detection MLOps Pipeline is an end-to-end system designed to identify potentially fraudulent financial transactions with high accuracy and scalability, integrating machine learning with robust MLOps practices. The primary objective of this project is to ensure seamless experimentation, deployment, and real-time monitoring of fraud detection models, while maintaining modularity and reproducibility across the entire ML lifecycle.

Live Demo (Streamlit): https://frauddetectionmlops.streamlit.app/

Project Overview

This pipeline leverages a custom-built FraudPipeline that handles feature engineering, preprocessing, class imbalance management via SMOTE, and threshold tuning to optimize fraud detection metrics. Experiments are tracked using MLflow, enabling parameter logging, artifact storage, and comparative analysis between multiple runs. Deployment is achieved through FastAPI for REST APIs and Streamlit for an interactive prediction dashboard, both containerized using Docker and orchestrated with Kubernetes (Minikube) for scalability. Continuous monitoring of system health and API performance is managed via Prometheus and Grafana dashboards, ensuring reliability in production. The system is designed to deliver high recall for fraudulent transactions while minimizing false positives, a critical balance in financial systems.

Tech Stack

The project uses Python (3.12+) as the core language and leverages libraries such as Scikit-learn for model building, Imbalanced-learn for SMOTE, and Pandas/Numpy for data manipulation. Deployment and MLOps tools include MLflow, FastAPI, Streamlit, Docker, Kubernetes, Prometheus, and Grafana. This combination ensures an efficient pipeline for experimentation, deployment, and monitoring.

Architecture

The architecture follows a modular design: data ingestion and preprocessing lead into feature engineering (interaction terms, ratio features, and time-of-day bins), followed by imputation, encoding, scaling, and resampling. Models like logistic regression, RandomForest, or XGBoost are trained and optimized via precision-recall thresholds. The pipeline is fully containerized and orchestrated through Kubernetes, with real-time metrics exposed for Prometheus and visualized via Grafana dashboards.

Features

• Real-time fraud prediction via FastAPI REST APIs and Streamlit web UI.
• Experiment tracking with MLflow, logging hyperparameters, metrics, confusion matrices, and PR curves.
• Scalable deployment using Dockerized microservices and Kubernetes orchestration.
• Robust monitoring with Prometheus scraping API metrics and Grafana visualizing system health and request patterns.
• Automatic preprocessing and SMOTE resampling for highly imbalanced datasets, coupled with dynamic threshold optimization to achieve optimal precision-recall trade-off.

Results & Metrics

The pipeline demonstrated strong performance on hold-out datasets: achieving up to 99% recall and 98% precision on balanced subsets, and effectively minimizing false negatives, which is crucial in fraud detection. Precision-recall curves and confusion matrices highlight the system’s effectiveness, and these artifacts are stored for review within the MLflow experiment tracking framework.

Future Enhancements

Planned improvements include integrating CI/CD pipelines using GitHub Actions or Jenkins, adding a model registry (MLflow Registry or Seldon Core), deploying cloud-native solutions on AWS/GCP/Azure, incorporating real-time streaming predictions with Kafka, and enhancing explainability with SHAP or LIME.

Project's Detailed Readme

Fraud Detection MLOps Pipeline

The Fraud Detection MLOps Pipeline is an end-to-end system designed to identify potentially fraudulent financial transactions with high accuracy and scalability. This project integrates Machine Learning (ML) with MLOps principles to ensure robust experimentation, deployment, and real-time monitoring of fraud detection models.

DEMO LINK: https://frauddetectionmlops.streamlit.app/

---

1. Project Overview

Objectives

• This project implements a complete MLOps pipeline for fraud detection using transactional data. It covers the entire ML lifecycle
• Build a modular FraudPipeline capable of feature engineering, preprocessing, resampling (SMOTE), and threshold tuning.
• Track experiments using MLflow for reproducibility and comparative analysis.
• Deploy the model using FastAPI for REST API services and Streamlit for an interactive UI.
• Containerize and orchestrate services using Docker and Kubernetes (Minikube).
• Monitor system health and metrics using Prometheus and Grafana dashboards.

Goal

Detect fraudulent transactions in real-time with high recall while minimizing false positives.

---

2. Tech Stack

Languages

• Python 3.12+

Core ML & Data Libraries

• Scikit-learn: Model building, preprocessing, metrics.
• Imbalanced-learn: SMOTE for class imbalance handling.
• Pandas / NumPy: Data manipulation and numerical operations.

MLOps & Deployment Tools

• MLflow: Experiment tracking, logging metrics, model registry.
• FastAPI: Serving the fraud detection model via REST API.
• Streamlit: Interactive web UI for predictions and model insights.
• Docker: Containerization of the FastAPI and Streamlit apps.
• Kubernetes (Minikube): Local orchestration and scaling of microservices.

Monitoring Tools

• Prometheus: Metrics scraping for FastAPI endpoints.
• Grafana: Visualization dashboards for system and API monitoring.

---

3. Architecture Diagrams

MLOps Pipeline

The complete pipeline involves:

1. Data Ingestion & Preprocessing
2. Model Training & Threshold Optimization
3. Experiment Tracking with MLflow
4. Model Deployment via FastAPI & Streamlit
5. Containerization with Docker
6. Orchestration using Kubernetes (Minikube)
7. Monitoring using Prometheus + Grafana

---

Model Pipeline

1. Feature Engineering: Interaction, ratio, binning, time-of-day categorization.
2. Preprocessing: Imputation, encoding, log transform, scaling.
3. Resampling: SMOTE to address class imbalance.
4. Model Training: Logistic Regression (configurable to RandomForest/XGBoost).
5. Threshold Tuning: Optimize precision-recall trade-off for fraud detection.

---

4. Features

• Real-Time Fraud Prediction:

  • Streamlit UI for quick predictions.
  • FastAPI endpoint for programmatic integration.

• Experiment Tracking:

  • MLflow logs parameters, metrics, artifacts (confusion matrix, PR curve).

• Scalable Deployment:

  • Dockerized microservices deployed on Kubernetes (Minikube).

• Robust Monitoring:

  • Prometheus scrapes real-time metrics from FastAPI.
  • Grafana dashboards visualize system health and request patterns.

• Data Handling:

  • Automatic preprocessing (missing values, scaling, encoding).
  • SMOTE resampling for highly imbalanced fraud datasets.

• Threshold Optimization:

  • Dynamically finds the best threshold balancing recall and precision.

5. Directory Structure

The project follows a modular structure separating API, model, monitoring, and visualization components:

FRAUD_MLOPS_PROJECT/
│
├── API/                         # FastAPI microservice
│   ├── main.py                   # API entry point
│   ├── schemas.py                # Pydantic models for request/response
│   ├── services.py               # Core service logic
│   └── mlruns/                   # MLflow experiment tracking logs
│
├── Data/                         # Datasets
│   ├── payment_fraud.csv
│   └── combined_holdout.csv
│
├── Images/                       # Project diagrams & screenshots
│   ├── Docker/
│   ├── FastAPI/
│   ├── Grafana/
│   ├── MLFlow/
│   ├── MLOps_Architecture/
│   ├── Model_Architecture/
│   └── Prometheus/
│
├── K8s/                          # Kubernetes manifests
│   ├── fraud-api-deployment.yaml
│   ├── fraud-api-service.yaml
│   ├── grafana-deployment.yaml
│   └── prometheus-deployment.yaml
│
├── Notebooks/                    # Jupyter Notebooks
│   ├── EDA.ipynb
│   ├── training_model.ipynb
│   ├── test_files.ipynb
│   └── artifacts/                # Trained model artifacts
│       ├── confusion_matrix.png
│       ├── pr_curve.png
│       └── fraud_pipeline_deployed.pkl
│
├── Pages/                        # Streamlit multi-page app
│   ├── home.py
│   ├── about_model.py
│   ├── metrics_page.py
│   └── about_me.py
│
├── Src/                          # Core ML pipeline code
│   ├── model.py                   # FraudPipeline, FeatureEngineering, Preprocessing
│   ├── utils.py                   # Helper functions
│   ├── config.py                  # Configurations
│   └── artifacts/                 # MLflow model logs
│
├── app.py                         # Streamlit entry point
├── Dockerfile                      # Docker setup for Streamlit/FastAPI
├── requirements.txt                # Dependencies
├── .gitignore
└── README.md

---

6. Setup Instructions

Prerequisites

• Python 3.10 or higher
• Docker Desktop
• Minikube (for Kubernetes)
• kubectl CLI
• Prometheus & Grafana (installed via Helm or K8s manifests)

---

Local Development Setup

1. Clone the repository

git clone https://github.com/MohitGupta0123/Fraud_Detection_MLOps.git
cd Fraud_Detection_MLOps

2. Create virtual environment & install dependencies

python -m venv .venv
source .venv/bin/activate    # Linux/Mac
.venv\Scripts\activate       # Windows
pip install -r requirements.txt

3. Run Streamlit app locally

streamlit run app.py

4. Run FastAPI service locally

cd API
uvicorn main:app --reload --host 0.0.0.0 --port 8000

---

Docker Setup

1. Build Docker images

docker build -t fraud-streamlit -f Dockerfile .
docker build -t fraud-fastapi -f Dockerfile ./API

2. Run containers

docker run -p 8501:8501 fraud-streamlit
docker run -p 8000:8000 fraud-fastapi

---

Kubernetes Deployment (Minikube)

1. Start Minikube

minikube start --driver=docker

2. Apply Kubernetes manifests

kubectl apply -f K8s/fraud-api-deployment.yaml
kubectl apply -f K8s/fraud-api-service.yaml
kubectl apply -f K8s/prometheus-deployment.yaml
kubectl apply -f K8s/grafana-deployment.yaml

3. Access services

minikube service fraud-api-service
minikube service prometheus -n monitoring
minikube service grafana -n monitoring

7. Running the Streamlit App

The Streamlit app provides an interactive interface to test fraud detection predictions and visualize model metrics.

Local Run

streamlit run app.py

• Access at: http://localhost:8501

Features

• Input transaction details (Category, Payment Method, Account Age, etc.)
• Auto-fill examples for Legitimate and Fraudulent transactions
• Real-time prediction with threshold-based confidence
• Navigation to About Model, Metrics, and About Me pages

---

8. Running the FastAPI Service

FastAPI serves the fraud prediction model as a REST API, useful for production-grade deployment and integration with external systems.

Local Run

cd API
uvicorn main:app --reload --host 0.0.0.0 --port 8000

• Access API docs at: http://localhost:8000/docs

Key Endpoints

• POST /predict – Accepts JSON payload and returns prediction
• GET /health – Health check endpoint

Docker Run

docker build -t fraud-fastapi -f Dockerfile ./API
docker run -p 8000:8000 fraud-fastapi

---

9. Experiment Tracking with MLflow

MLflow is integrated to log experiments, parameters, metrics, and artifacts (PR curve, confusion matrix, models).

Usage

• Automatically tracks during training via FraudPipeline

• Logs include:

  • Parameters: Steps applied, resampling method, model type
  • Metrics: Accuracy, Precision, Recall, F1-score, PR-AUC
  • Artifacts: PR Curve, Confusion Matrix, Serialized Model

Access MLflow UI

mlflow ui

• Opens at http://127.0.0.1:5000
• Explore experiment runs and compare metrics visually

---

10. Monitoring with Prometheus & Grafana

The deployed FastAPI service exposes metrics for Prometheus, visualized via Grafana dashboards.

Prometheus

• Scrapes FastAPI metrics (request counts, response latency, error rates)
• Runs on port 9090 in monitoring namespace

Grafana

• Visualizes Prometheus data using pre-built dashboards
• Runs on port 3000 in monitoring namespace
• Import your saved JSON dashboard via Grafana UI

Steps to Access

minikube service prometheus -n monitoring
minikube service grafana -n monitoring

11. Model Details

The fraud detection model is built using a custom pipeline with multiple stages:

Pipeline Steps

1. Feature Engineering

   • Interaction: Category x PaymentMethod
   • Ratio: paymentMethodAgeDays / accountAgeDays
   • Binning: accountAgeDays into new/medium/old
   • Time Feature: Categorize localTime into time-of-day bins

2. Preprocessing

   • Imputation for missing values (median/mode)
   • One-hot encoding for categorical variables
   • Log transformation for skewed features
   • Scaling: StandardScaler (skewed) + MinMaxScaler (symmetric)

3. Resampling

   • SMOTE to handle extreme class imbalance

4. Model Training

   • Logistic Regression (default)
   • Supports other models like RandomForest, XGBoost

5. Threshold Tuning

   • Optimal threshold found via precision-recall curve
   • Current best threshold: 0.8370 (Precision = 0.955, Recall = 0.991)

---

12. Results & Metrics

Hold-out Set Performance

• Hold-out A: Accuracy 97%, Recall 100%, Precision 25% (imbalanced case)
• Hold-out B: Accuracy 99%, Recall 100%, Precision 50% (imbalanced case)
• Hold-out C: Accuracy 98%, Recall 98%, Precision 98%

PR Curve & Confusion Matrix

• Stored in Notebooks/artifacts/
• PR Curve demonstrates strong precision-recall balance
• Confusion Matrix confirms minimal false negatives (critical for fraud detection)

---

14. Future Work

• Integrate CI/CD pipelines with GitHub Actions or Jenkins
• Add model registry using MLflow’s registry or Seldon Core
• Deploy cloud-native on AWS/GCP/Azure (EKS/GKE/AKS)
• Implement real-time streaming predictions with Kafka
• Add explainability (SHAP/LIME) for fraud predictions

---

4. Study Tracker Dashboard

The Study Tracker Dashboard is a personalized and visually rich web application built using Streamlit to help students track their study schedules, daily goals, and overall progress in an interactive way. It is designed to support exam preparation by providing real-time insights into session-wise performance, subject completion rates, and backlog tracking, while ensuring secure authentication and per-user data storage.

Live Demo (Render): https://study-tracker-2zhd.onrender.com/

Project Overview

This dashboard enables students to plan and monitor their studies with a dynamic timetable system powered by CSV data, JSON state management, and SQLite-based authentication. Users can log in, view subject-wise schedules, mark completed study sessions, and track their progress via interactive charts and grids. The system automatically refreshes before each session begins, ensuring students stay on track and manage their time effectively.

Features

• Secure Login & Registration using SQLite and streamlit_authenticator, storing hashed passwords and user-specific data.
• Dynamic Study Plans auto-generated from real-time calendars with per-day session allocations.
• Progress Tracking with donut charts, bar graphs, and checkbox grids to visualize subject completion.
• Backlog Detection for missed sessions and dynamic rerun alerts to maintain consistency.
• Time Analytics including usage distribution and comparison charts for weekly and overall summaries.
• Multi-User Support with JSON-based state management, allowing each user to resume progress seamlessly.

Tech Stack

• Frontend: Streamlit for UI, Matplotlib for charts
• Backend & Auth: SQLite with streamlit_authenticator
• Data Handling: Pandas for CSV and JSON operations
• Storage: CSV for study plans, JSON for user progress persistence

Results & Impact

This tool simplifies exam preparation by consolidating study plans, completion rates, and time management into one dashboard. Its clean UI, auto-rerun reminders, and subject backlog insights ensure students can maintain consistency, track milestones, and reduce cognitive load during preparation.

Future Enhancements

Planned upgrades include integrating cloud storage for multi-device access, adding goal-setting and streak tracking, mobile responsiveness, and AI-driven study recommendations based on progress history.

Project's Detailed Readme

📚 Study Tracker Dashboard

A beautiful and personalized 📈 Study Progress Dashboard built using Streamlit with secure authentication. Designed for students preparing for any exam — to track daily goals, visualize subject-wise progress, and maintain consistency using real data, session-wise scheduling, and performance metrics.

---

🚀 Live Demo

👉 LIVE APP! : https://study-tracker-application-mohit.streamlit.app/

---

🔑 Features

• 🔐 Login & Registration using SQLite (users.db) with streamlit_authenticator
• 📅 Auto-Traced Study Sessions with date-driven scheduling
• ✅ Checkbox Grid to mark each session as done
• 📊 Subject-Wise Progress with donut & bar charts
• ⏳ Backlog Tracker for missed subjects
• 🔁 Auto-Rerun Logic before session starts
• 🧠 Per-user JSON Save State for progress
• 🧾 Daily Completion Table with tick marks
• 📌 Reset Button to start over anytime

---

🗂️ Directory Structure

📁 study-tracker/

    ├── study\_tracker\_final.py             # Streamlit main app

    ├── auth\_db.py                           # SQLite-based authentication

    ├── users.db                             # Registered users stored here

    ├── \*.json                               # User progress files (per user)

    ├── credentials.yaml                     # (Optional) legacy credentials

    ├── Study\_Plan\_Schedule.csv              # Timetable (days × session slots)

    ├── Subject\_Study\_Time\_Table.csv         # Subjects with video hours

    ├── requirements.txt                     # Python dependencies

    └── README.md                            # You're reading it 🙂

---

🛠️ Installation

1️⃣ Clone the Repo

git clone https://github.com/MohitGupta0123/Study_Tracker.git
cd Study_Tracker

2️⃣ Install Requirements

pip install -r requirements.txt

3️⃣ Run the App

streamlit run study_tracker_final.py --server.port 8501

---

🧠 Tech Stack

• Frontend: Streamlit
• Auth: streamlit_authenticator + SQLite (users.db)
• Charts: Matplotlib
• Data Processing: Pandas, JSON
• Storage: CSV files for study data, JSON for user progress

---

📁 Data Files

File	Description
Study_Plan_Schedule.csv	Weekly plan (Day-wise × Sessions)
Subject_Study_Time_Table.csv	Subjects with total video hours
<username>_progress.json	Per-user saved checkbox state

---

📈 Dashboard Features in Action

📅 Today's Work

• Auto-detects what you need to study today based on the real calendar

🔁 Auto Rerun

• Reruns app 5 mins before session start to alert you

🧾 Progress Tracker Grid

• Tick off what you’ve studied; persists even after reload

🍩 Donut Charts

• See your completion % for each subject

🟥 Backlogs

• Automatically shows what you've missed in previous days

📊 Time Slot Analytics

• Frequency of time slots used across your plan

---

📸 Visual Elements

• ✅ Checkbox table for sessions per day
• 📊 Bar chart for session frequency
• 🟡 Today's plan
• 🔴 Backlogs
• 📅 Dynamic live clock
• 📈 Donut & pie charts showing progress distribution

---

🔐 Authentication System

• Uses SQLite-based DB (users.db) for login/signup
• Passwords are hashed using streamlit_authenticator
• Each user has their own progress file (<username>_progress.json)

---

🙌 Acknowledgements

• Thanks to streamlit_authenticator for easy auth integration
• Inspired by real preparation needs of students
• Created with 💙 using Python and Streamlit

---

5. GraphRAG: eBay User Agreement Chatbot

The GraphRAG: eBay User Agreement Chatbot is a Knowledge Graph-powered conversational AI system designed to answer legal and policy-related questions from the eBay User Agreement. Built with Neo4j, Meta LLaMA 3B (instruction-tuned), and FAISS memory, it integrates natural language understanding, graph-based reasoning, and contextual memory to provide accurate and grounded responses to user queries. The chatbot is deployed via Streamlit with real-time LLM response streaming using Hugging Face endpoints.

Live Demo (Streamlit): https://graphrag-ebay-user-aggrement-chatbot.streamlit.app/

Project Overview

This chatbot addresses the challenge of understanding lengthy user agreements by converting them into structured knowledge graphs. User queries are processed through Named Entity Recognition (NER) and Relation Extraction (RE) pipelines (using SpaCy and custom rules), mapped to triples stored in Neo4j, and enhanced with conversation history stored in FAISS. The retrieved triples and memory context are dynamically injected into prompts sent to the Meta LLaMA 3B model, ensuring concise and fact-grounded answers without hallucination.

Features

• Knowledge Graph-based Reasoning: Extracts triples from legal documents and queries them in real-time using Cypher.
• Memory-Augmented Retrieval: Past Q&A context stored in FAISS ensures conversational continuity.
• Grounded Legal Q&A: Responses are fact-checked against structured data from the agreement.
• Interactive UI: Streamlit app with chat history, save/load sessions, and Neo4j visualizations.
• Streaming Responses: Real-time LLM output streamed to users for smooth interactions.

Tech Stack

• Frontend: Streamlit
• Backend & Reasoning: Meta LLaMA-3B-Instruct via HuggingFace API, FAISS for memory, Neo4j for graph storage
• Triplet Extraction & NER: SpaCy and custom RE pipelines
• Embeddings: SentenceTransformers for synonym and entity similarity expansion

Architecture & Workflow

1. User submits a query via the Streamlit interface.
2. Entities are extracted using SpaCy-based NER + RE pipelines.
3. Relevant triples are fetched from the Neo4j knowledge graph using Cypher queries.
4. Memory module retrieves related past Q&A from FAISS.
5. Combined context (triples + memory) is injected into prompts and sent to LLaMA-3B.
6. Response is streamed to the user and optionally saved for later sessions.

Results & Impact

The chatbot successfully transforms dense legal documents into queryable knowledge structures and provides precise answers with contextual memory, reducing information overload for users. It demonstrates how GraphRAG techniques can be applied for legal, compliance, and policy-oriented conversational systems.

Future Enhancements

Planned improvements include multi-document support, automated graph building pipelines, integration with LangChain memory chains, and deployment to cloud environments (e.g., Hugging Face Spaces or AWS).

Project's Detailed Readme

🧠 GraphRAG: eBay User Agreement Chatbot

A Knowledge Graph-Powered Conversational AI built to answer legal and policy-related queries from the eBay User Agreement using Neo4j, LLMs (Meta LLaMA 3B), and Memory via FAISS.

🌐 Website - https://graphrag-ebay-user-aggrement-chatbot.streamlit.app/

💡 Project Motivation

Reading long user agreements is painful. This project creates an intelligent chatbot that:

• Understands natural language queries
• Retrieves facts from a Neo4j-based Knowledge Graph
• Enhances responses using memory of past conversations
• Uses open-source LLMs to generate grounded, concise, and transparent answers

---

🔁 End-to-End Architecture

1. 🧱 Steps:

1. User submits a query via Streamlit UI.
2. Named Entities are extracted using SpaCy + RE.
3. Matching triples are fetched from Neo4j KG.
4. Memory module (FAISS) adds past Q&A context.
5. Prompt is dynamically injected and sent to LLaMA-3B.
6. Response is streamed and displayed to the user.

---

2. 🧠 Knowledge Graph Construction

• Text source: eBay User Agreement PDF
• Preprocessing: cleaned and tokenized using SpaCy
• NER & RE: Custom rules + pre-trained SpaCy models
• Triplets: Extracted using pattern matching and OpenIE-style RE
• Storage: JSON + CSV → Loaded into Neo4j (local or Aura Free)
• Tools: graph_builder.py, KG_creation.ipynb

---

3. 🔍 Query-to-KG Translation

• Input query is processed for Named Entities.
• Synonyms are expanded using Sentence Transformers.
• KG is queried using Cypher to retrieve matching triplets.
• Top-k results ranked based on entity similarity & relevance.
• Implemented in retriever.py using match (s)-[r]->(o) pattern.

---

4. 💬 Prompting Strategy

• Format: [Triples] + [Memory] → Context Window
• Model: Meta LLaMA-3B (Instruct-tuned)
• Sent via HF endpoint with streaming

📌 Example

System: You are a legal assistant for eBay User Agreement.
Context:

* \[User] may terminate the agreement with 30 days notice.
* \[eBay] may restrict access for violation.
  Memory:
* Q: What if I break the policy? A: Your access may be restricted.
  Question: Can I end the agreement anytime?

Answer:

eBay allows termination with 30 days’ notice. However, immediate termination may depend on specific conditions outlined in Section X.

---

5. ▶️ Running the Chatbot

• Install dependencies: pip install -r requirements.txt

• Add your Hugging Face token in the UI sidebar.

• Add Neo4j credentials in .streamlit/secrets.toml

• Run: streamlit run app.py

---

6. 🧠 Model Details & Streaming

• Model: Meta LLaMA-3B-Instruct (via HuggingFace)
• Endpoint: HuggingFace Inference Endpoint (stream=True)
• Temperature: 0 - 0.2 for factual output
• Streaming: Enabled to simulate real-time response using requests with stream

---

🚀 Features

✅ Knowledge Graph-based reasoning

✅ Memory-augmented retrieval (FAISS)

✅ Legal/Policy Q&A grounded in real documents

✅ Streamlit-powered UI with chat history and controls

✅ Chat save/load functionality

✅ Real-time LLM responses using HuggingFace inference endpoint

---

🧱 Project Structure

.
├── app.py                          # Main Streamlit app
├── requirements.txt               # Dependencies
├── create_code.py                 # Code generation helper
├── chat_history.json              # Sample chat history
│
├── Src/                           # Core logic modules
│   ├── memory.py                  # Persistent memory using Chroma
│   ├── retriever.py               # Entity extractor & KG triple retriever
│   ├── prompt_injector.py         # Prompt builder & LLM streaming query
│   └── graph_builder.py           # For KG construction
│
├── Triples/                       # Triplets extracted from the source doc
│   ├── graphrag_triplets.csv/json
│   ├── triples_raw.json
│   ├── triples_structured.json
│   └── knowledge_graph_triplets.json
│
├── KG/                            # Visuals & summaries
│   ├── knowledge_graph_image.png
│   └── summary.json
│
├── NER/                           # Extracted named entities
│   └── ner_entities.json
│
├── Data/
│   ├── Ebay_user_agreement.pdf
│   └── cleaned_ebay_user_agreement.txt
│
├── Notebooks/                     # Jupyter notebooks for exploration
│   ├── KG_creation.ipynb
│   ├── preprocessing.ipynb
│   └── graphrag-quering-kg-and-llm-prompting.ipynb
│
├── .streamlit/
│   └── secrets.toml               # API keys & credentials
├── .gitignore
└── README.md

---

⚙️ Setup & Installation

1. Clone the repository

git clone https://github.com/MohitGupta0123/GraphRAG-Ebay-User-Aggrement-Chatbot.git
cd GraphRAG-Ebay-User-Aggrement-Chatbot

2. Install dependencies

pip install -r requirements.txt

3. Token Configuration

This app prompts for your Hugging Face token (HF_TOKEN) securely at runtime in the sidebar.
You no longer need to store the token in secrets.toml.

However, Neo4j credentials are still required in .streamlit/secrets.toml:

NEO4J_URI = "bolt://your_neo4j_uri"
NEO4J_USERNAME = "neo4j"
NEO4J_PASSWORD = "your_password"
NEO4J_DATABASE = "neo4j"

4. Launch the app

streamlit run app.py

---

🧠 How It Works

1. User Input

You ask a question like:

"Can I terminate the agreement anytime?"

2. Entity Extraction

Entities like terminate, agreement are extracted.

3. Knowledge Graph Retrieval

Relevant triples from Neo4j are retrieved.

4. Memory Recall

Past similar Q&A are pulled from persistent memory (Faiss).

5. Prompt Generation

Triples + memory form a context which is sent to LLaMA-3B via Hugging Face API.

6. Answer Generation

The LLM answers based only on retrieved facts — no hallucination.

---

💾 Save & Load Chat

You can download your chat as a .json file and re-upload it to continue your session. All retrieved triples and memory are retained across sessions!

---

📌 Tech Stack

• Frontend: Streamlit
• LLM: Meta LLaMA-3B-Instruct via HuggingFace
• Graph: Neo4j (Aura Free or Local)
• Embeddings: SentenceTransformers
• Memory Store: FAISS
• Triplet Extraction: SpaCy / RE Pipelines
• NER: Custom + pre-trained models

---

🛡 Limitations

• Currently optimized for the eBay User Agreement
• Requires manual graph building from text
• Needs HuggingFace token (streaming)

---

🧠 Acknowledgement

• GraphRAG research from Meta AI
• Neo4j Knowledge Graphs
• LangChain Memory Chains

---

6. Fashion Sense AI (Fashion Visual Search & Personalized Styling Assistant)

The Fashion Sense AI project is a modern AI-powered web application that enables users to search for visually similar fashion products, receive personalized outfit recommendations, and simulate style suggestions based on browsing history and trending fashion insights. Built using Streamlit, CLIP embeddings, FAISS indexing, and Gemma-3B LLM via Hugging Face, it serves as a foundation for AI-driven e-commerce integrations and personal styling tools.

Live Demo (Streamlit): https://fashion-sense-ai.streamlit.app/

Project Overview

Unlike traditional search engines that rely on manual tags or filters, Fashion Sense AI enhances user experience with multimodal search capabilities. Users can upload images to retrieve visually similar products, input descriptive text queries (e.g., “oversized hoodie” or “floral print dress”), and receive history-based recommendations or AI-generated outfit completions powered by large language models. The pipeline combines vector-based retrieval using FAISS with trend analysis and recommendation logic for a seamless, personalized shopping experience.

Features

• Visual Search: Upload an image or input a textual style description to find similar products instantly.
• LLM-Powered Outfit Generation: Use Gemma-3B to suggest outfit completions including shoes, accessories, or layering items.
• Browsing History Simulation: Generate fake user histories to mimic personalized suggestions.
• Trend Analysis: Extract trending fashion keywords from inventory and scraped web data.
• Efficient Vector Search: FAISS-based nearest-neighbor search using 1152-dimensional embeddings from CLIP and SentenceTransformers.
• Modular Pipeline: Each module (dataloader, search, outfit suggestions, trend inference) is reusable and extensible for future e-commerce integrations.

Tech Stack

• Frontend: Streamlit
• Image Embeddings: CLIP (ViT-L/14)
• Text Embeddings: SentenceTransformers
• LLM Suggestion Engine: Gemma-3B via Hugging Face Inference API
• Nearest-Neighbor Search: FAISS
• Trend Analysis: TF-IDF and keyword extraction from product metadata

Architecture & Workflow

1. User uploads an image or enters a style query.
2. The system generates embeddings using CLIP/SentenceTransformers and performs a FAISS search across indexed inventory.
3. Results are displayed visually with associated metadata (price, style tags).
4. Browsing history (real or simulated) feeds into personalized suggestions.
5. The Gemma LLM generates complementary outfit recommendations based on style context and recent fashion trends.

Results & Impact

Fashion Sense AI demonstrates how multimodal AI can transform fashion discovery, providing more relevant, visually accurate, and trend-driven recommendations compared to traditional keyword-based search. It serves as a blueprint for integrating visual search, personalization, and LLM-driven fashion styling into e-commerce platforms.

Future Enhancements

Planned improvements include user authentication with persistent history, add-to-cart integration for online stores, voice-based outfit search, and fashion Q&A chatbots powered by generative AI.

Project's Detailed Readme

👗 Fashion Sence AI

(Fashion Visual Search & Personalized Styling Assistant)

A modern AI-powered web application that helps users search for visually similar fashion products, receive outfit recommendations based on their style preferences and simulate personalized experiences using recent trends and LLMs — all in one elegant interface built with Streamlit, CLIP, and Gemma LLM.

🌐 Website : https://fashion-sense-ai.streamlit.app/

---

🧠 Project Overview

Traditional fashion search engines rely on tags or manual filters. Our system enhances user experience by allowing:

• Image-based Search: Upload any fashion item image to retrieve visually similar products.
• Text-based Search: Enter queries like "oversized hoodie, floral print dress" and retrieve matching products.
• Personalized Suggestions: Based on your browsing or simulated history.
• Intelligent Outfit Completion: Generate LLM-powered suggestions to complete your outfit using trending insights.

This app can serve as a virtual fashion assistant, a personal stylist, or a foundation for an e-commerce AI integration.

---

🚀 Features at a Glance

Module	Description
🔍 Visual Search	Upload an image or type a query to find similar fashion products.
🧠 LLM-based Outfit Generator	Uses Gemma 3B via Hugging Face API to suggest outfit completions.
👤 History Simulation	Fake user history generation + product-based similarity suggestions.
📈 Fashion Trend Inference	Extracts trending keywords from inventory and online data.
📊 FAISS Indexing	Efficient nearest-neighbor search using pretrained embeddings.
🌐 Hugging Face Integration	Seamless API token handling and LLM inference.
✅ Modular Pipeline	Each part of the pipeline is reusable, testable, and extensible.

---

🗂️ Project Structure

Fashion AI/
├── Assets/                  # FAISS Index, product IDs, and image embeddings
├── Data/                    # Cleaned CSV files for jeans and dresses
├── Modules/                 # Modular Python scripts for each major functionality
│   ├── dataloader.py
│   ├── faiss_index.py
│   ├── outfit_suggester.py
│   ├── preprocessing.py
│   ├── search.py
│   ├── trends.py
│   └── user_profile.py
├── Src/                     # Static assets like animation GIFs or logos
├── Notebooks/               # Python Jupyter Notebooks
├── Test_Images/             # Sample test images
├── app.py                   # Main Streamlit app file
├── requirements.txt         # Required Python libraries
└── README.md                # You're here!

---

🛠️ Tech Stack

• Frontend: Streamlit
• Image Embeddings: CLIP (ViT-L/14)
• Text Embeddings: SentenceTransformers
• LLM Suggestion Engine: Gemma-3B (via HF Inference API)
• Nearest Neighbors Search: FAISS
• Trends Analysis: Web scraping + TF-IDF on product metadata
• Deployment-ready: Works on Streamlit Cloud

---

📦 Installation & Setup

Recommended Python version: >=3.11

1. Clone the repository

git clone https://github.com/MohitGupta0123/Fashion-Sense-AI.git
cd Fashion-Sense-AI

2. Install the dependencies

pip install -r requirements.txt

3. Run the application

streamlit run app.py

4. Enter Hugging Face Token

• Get your free token from: https://huggingface.co/settings/tokens
• Paste it in the sidebar to access outfit suggestions using the Gemma model.

---

🧭 How to Use

Once the app is running:

👤 Step-by-Step

1. Upload an image or enter a style description like "striped top, loose fit".
2. Adjust the number of desired results using the slider.
3. View Top Matching Products — displayed with image and price.
4. Click 🧪 Simulate Fake History to create browsing history (or use your own).
5. See Suggestions Based on History.
6. Tap 🧠 Generate Outfit to let the LLM suggest fashion complements (shoes, accessories, layering items, etc.).

---

🧪 Sample Use Cases

• 🔍 A user uploads a black denim jacket → finds 10 similar styles instantly.
• 🛍️ A shopper queries for "Off Shoulder dresses" → retrieves relevant dresses.
• 🤖 Fake user browsing history is generated → recommendations are shown.
• 🎨 LLM completes the outfit with accessories and layering items based on the uploaded look.

---

🔐 Hugging Face Token Handling

• The HF_TOKEN is required to query Gemma for outfit recommendations.
• It is securely saved in st.session_state for each user session.
• It is never stored persistently or sent elsewhere.

---

🎯 Future Improvements

• 🔒 Authenticated user sessions with persistent history
• 🛒 Add-to-cart integration for e-commerce use
• 🗣️ Voice-based outfit search (via Whisper or Speech-to-Text APIs)
• 🧵 Tailor chatbot integration for fashion Q&A

---

7. MRI Image Reconstruction using K-Space

The MRI Image Reconstruction using K-Space project is an interactive Streamlit application that demonstrates step-by-step reconstruction of MRI images from frequency domain (k-space) data using the 2D Fourier Transform. This educational tool visualizes how low and high-frequency components contribute to the final reconstructed image, enabling a better understanding of MRI physics and k-space representation.

Live Demo (Streamlit): https://mri-image-reconstruction-using-kspace.streamlit.app/

Project Overview

The application allows users to upload their own DICOM (.dcm) MRI images or use a default sample to observe live reconstruction. The app converts the input image into its frequency domain using Fast Fourier Transform (FFT) and progressively reconstructs the spatial image as more frequency components are added. This progressive visualization highlights the role of low-frequency (contrast) and high-frequency (detail) components in MRI imaging.

Features

• DICOM Upload Support: Accepts .dcm MRI files or uses a default image if none is uploaded.
• K-Space Visualization: Displays the magnitude spectrum of the Fourier Transform.
• Progressive Reconstruction: Step-by-step visualization of image formation from frequency components.
• Interactive UI: Real-time updates with Streamlit session_state for smooth user experience.
• Error Handling: Fallback to default image in case of upload or processing errors.

Tech Stack

• Frontend: Streamlit
• Numerical Processing: NumPy (FFT for k-space transformations)
• Visualization: Matplotlib (optional for additional plots)
• DICOM Handling: pydicom or similar libraries

Architecture & Workflow

1. Load .dcm MRI image from user upload or default sample.
2. Convert the image into frequency domain using FFT to obtain k-space.
3. Progressively reconstruct the image by adding low- and high-frequency components step by step.
4. Display both the k-space magnitude and the live reconstructed image in the Streamlit interface.

Results & Impact

This project provides a visual learning tool for students, researchers, and enthusiasts exploring MRI imaging concepts. By showing how k-space frequencies contribute to final image quality, it bridges the gap between theoretical physics and practical medical imaging understanding.

Future Enhancements

Planned improvements include multi-mode reconstruction (e.g., radial or spiral filling), support for raw scanner data, and integration of additional image quality metrics (e.g., SSIM, PSNR) for comparative visualization.

Project's Detailed Readme

⚕️ MRI Image Reconstruction using K-Space ⚕️

🧩 With Fourier Transform Demo

A Streamlit application that demonstrates step-by-step reconstruction of MRI images from their frequency domain (k-space) data using the 2D Fourier Transform.

This interactive tool helps users understand how low and high-frequency components contribute to the final image, showing live reconstruction of the image as more of the frequency space is progressively filled.

👉 Live App:
https://mri-image-reconstruction-using-kspace.streamlit.app/

---

🖼️ Features

• 📂 Upload your own .dcm (DICOM) MRI image, or use the provided default sample.
• 🌀 Visualize the magnitude spectrum of the Fourier Transform (k-space).
• 🔍 Step-by-step live image reconstruction from k-space data.
• ⚙️ Session management using Streamlit's session_state.
• 🚫 Robust error handling — fallback to default image if upload fails.

---

🚀 How It Works

1. Image Loading:
   The app accepts .dcm files (commonly used for medical imaging). If none is provided, it loads a default MRI image.

2. Fourier Transform:
   The image is converted into its frequency domain using Fast Fourier Transform (FFT). The k-space represents spatial frequencies present in the image.

3. Coordinate Processing:
   The app focuses first on reconstructing the left half of the frequency spectrum (progressive loading), prioritizing pixels closer to the center (lower frequencies).

4. Progressive Reconstruction:
   Using live_fourier_reconstruction(), the image is reconstructed in real-time as frequency components are progressively added back.

5. Display:
   View both the k-space magnitude and the live-updating reconstructed image.

---

📦 Installation & Local Development

To run the project locally:

git clone https://github.com/MohitGupta0123/MRI-Image-reconstruction-using-kspace.git
cd MRI-Image-reconstruction-using-kspace
pip install -r requirements.txt
streamlit run kspace.py

Note: Replace kspace.py with your actual Streamlit script filename.

---

🖥️ Usage

• Upload a .dcm image file.
• Watch the k-space magnitude spectrum.
• Observe real-time image reconstruction.
• Understand how different frequency components affect image quality.

---

📂 Project Structure

📦 Your Project Name
├── .streamlit/                # Streamlit configuration
│   └── config.toml
├── docs/                      # assets
│   ├── demo.gif
│   └── fourier_reconstruction.gif
├── images/                    # Images and icons
│   ├── default.dcm
│   └── icon.ico
├── kspace.py                  # Main Python script
├── readme.md                  # Project README
├── requirements.txt           # Python dependencies

---

🧠 Learn More

• MRI Physics and k-space Explained - https://mriquestions.com/index.html

---

🌐 Try it Online

No need to set up locally!
Access the live version here:
👉 Launch App: https://mri-image-reconstruction-using-kspace.streamlit.app/

---

🛠️ Tech Stack

• Python 🐍
• Streamlit 🎈
• NumPy ⚙️
• Matplotlib (optional for visualization)
• DICOM image handling (assumed via pydicom or similar)

---

🤝 Contributing

Contributions are welcome! If you have ideas to improve the visualization, add more reconstruction modes, or extend this tool, feel free to open an issue or PR.

---

✨ Acknowledgements

• Inspiration from medical imaging techniques and MRI physics.
• Powered by Streamlit for fast web app development.

---

Disclaimer and limitations

This software is not intended for medical use. Even if a scanner original DICOM file is used, the resulting k-space is not equivalent to the scanner raw data as it contains all post-acquisition modifications applied by the scanner software.

---

8. Corn Vomitoxin Prediction (DeepCorn Project)

The Corn Vomitoxin Prediction (DeepCorn Project) is a complete pipeline for predicting vomitoxin_ppb concentration in corn samples using both Machine Learning (ML) and Deep Learning (DL) techniques. The project leverages spectral reflectance data across 0–447 bands to build robust predictive models and includes a Dockerized inference API for streamlined deployment.

GitHub Repo: https://github.com/MohitGupta0123/Corn-Vomitoxin-PPB-Prediction

Project Overview

The repository contains two parallel pipelines — a traditional ML approach and a deep learning approach — with dedicated notebooks for exploratory data analysis, feature scaling, and model training. The deep learning models are built with PyTorch, achieving superior performance compared to traditional methods. A Dockerized version of the inference pipeline is provided for easy containerized deployment, making the project production-ready and portable.

Features

• Spectral Data Analysis: Utilizes 447-band reflectance data for vomitoxin prediction.
• Dual Modeling Approaches: Traditional ML pipeline (EDA + model building) and DL pipeline (optimized PyTorch models).
• Optimized Deep Models: Multiple optimized deep learning models (DeepCorn_Best_Model_optimized.pth, etc.) tested for performance on scaled datasets.
• Dockerized Inference API: Containerized Flask API for easy deployment and testing.
• End-to-End Pipeline: Includes EDA, preprocessing, scaling, model training, and inference.

Tech Stack

• Programming Language: Python
• Machine Learning Framework: Scikit-learn for ML; PyTorch for DL
• Containerization: Docker (Flask-based API for inference)
• Data Handling: Pandas, NumPy
• Visualization: Matplotlib, Seaborn for EDA

Architecture & Workflow

1. Perform EDA and build baseline ML models using the EDA & ML Approach Notebook.
2. Train deep learning models with scaled inputs using the Deep Learning Notebook.
3. Select best-performing models based on metrics and save weights (.pth files).
4. Deploy the optimized deep learning model via the Dockerized Flask API for inference.

Results & Impact

The deep learning approach achieved the best predictive performance on vomitoxin_ppb concentration, outperforming baseline ML models. Scaling input data significantly improved model accuracy, enabling better generalization on unseen samples.

Future Enhancements

Planned improvements include enhancing model robustness, deploying cloud-native APIs (AWS/GCP), and incorporating real-time spectral data streaming for field-based predictions.

Project's Detailed Readme

🌽 Corn Vomitoxin Prediction (DeepCorn Project)

📝 Project Overview

This repository contains a complete pipeline for predicting vomitoxin_ppb concentration in corn samples using both Machine Learning and Deep Learning approaches.

---

🏗️ Project Structure

Corn_Vomitoxin_Prediction/
│
├── DeepCorn_Project_docker/   # Dockerized Inference API
│   └── README.md              # Docker-specific documentation
│
├── EDA & ML Approach Notebook.ipynb   # Traditional ML Pipeline
├── Deep Learning Notebook.ipynb      # Deep Learning Pipeline
│
├── complete_df.csv                   # Original dataset
├── final_scaled_df.csv               # Scaled input dataset
│
├── best_deepcorn_model.pth           # PyTorch Model (Best Model)
├── DeepCorn_Best_Model_optimized.pth    
├── DeepCorn_Best_Model_optimized_better_with_less_features.pth
├── DeepCorn_Best_Model_optimized_Pytorch_with_best_results.pth
│
├── requirements.txt                  # Project Dependencies
├── .gitignore                        # Files to Ignore
└── README.md                         # This Documentation

---

🎯 Objective

Predict the concentration of vomitoxin (ppb) in corn samples using spectral reflectance data from 0 to 447 bands.

---

🚀 Quick Start

Step 1: Run EDA and Traditional Machine Learning Approach

jupyter notebook "EDA & ML Approach Notebook.ipynb"

Step 2: Run Deep Learning Model

jupyter notebook "Deep Learning Notebook.ipynb"

---

🐋 Dockerized Deep Learning Inference (Optional)

Navigate to the DeepCorn_Project_docker folder and follow the instructions in its README.md to run the dockerized model.

---

📂 Files Description

File/Folder	Description
EDA & ML Approach Notebook.ipynb	Exploratory Data Analysis and Traditional ML Model
Deep Learning Notebook.ipynb	Deep Learning Model Training and Inference
complete_df.csv	Original Dataset
final_scaled_df.csv	Scaled Dataset (Input for Deep Model)
DeepCorn_Best_Model_optimized.pth	Best Deep Learning Model Weights
requirements.txt	Python Dependencies
DeepCorn_Project_docker/	Dockerized Inference API

---

🛠️ Dependencies

Install all required packages:

pip install -r requirements.txt

---

🛑 Results and Metrics

• Achieved the best performance with the deep learning model: DeepCorn_Best_Model_optimized.pth
• Improved performance on scaled input data (final_scaled_df.csv)

---

✅ How to Run the DeepCorn Docker API

cd DeepCorn_Project_docker

Follow the Docker README instructions to:

• Pull Docker Image
• Run Flask API
• Test the API for Inference

---

🌟 Future Work

• Improve Model Robustness
• Deploy on Cloud Platform (AWS/GCP)

---

9. Image Processing Project

The Image Processing Project is an interactive Streamlit application that allows users to perform various image transformations and visual enhancements, including brightness and contrast adjustments, color space conversions, binary thresholding, and image annotation. Built with OpenCV and NumPy, this application serves as a hands-on tool for learning basic image processing techniques and provides features for real-time adjustments and downloadable outputs.

Live Demo (Streamlit): https://image-processing-project-843relnehappnf9adb6mnqk.streamlit.app/

Youtube Link(complete app walkthrough): https://www.youtube.com/watch?v=O0zejFyF05Y

Project Overview

This project focuses on providing a clean, user-friendly interface for experimenting with fundamental image processing operations. Users can upload their own images (JPG, JPEG, PNG) and apply transformations such as RGB conversion, grayscale conversion, binary thresholding, and fine-tuned brightness/contrast controls. Annotations like lines, rectangles, circles, and custom text can be added to images with configurable colors, sizes, and positions. The app also includes direct download functionality to save the processed images locally.

Features

• Image Upload and Display: Supports JPG, JPEG, and PNG formats with automatic dimension display (height and width).
• Color Transformations: Convert images to RGB, grayscale, or binary threshold modes.
• Brightness and Contrast Adjustment: Real-time sliders for controlling brightness and contrast levels.
• Annotations: Add lines, rectangles, circles, or text to images with customizable coordinates, thickness, font styles, and colors.
• Interactive UI: Built with Streamlit’s widgets (sliders, dropdowns, color pickers) for real-time updates.
• Download Processed Images: Download any modified image (RGB, grayscale, binary, annotated) in JPG format directly from the app.

Tech Stack

• Frontend & App Framework: Streamlit
• Image Processing: OpenCV (cv2)
• Data Handling: NumPy for array manipulation
• UI Enhancements: Streamlit components (sliders, color picker, text inputs) and custom CSS for styling

Architecture & Workflow

1. Image Input: User uploads an image via the sidebar uploader.
2. Processing Options: User selects desired processing mode (Original, RGB, Grayscale, Binary, Brightness, Contrast, Annotation).
3. Transformation: Corresponding OpenCV operations (color conversion, thresholding, scaling) are applied.
4. Annotation Handling: Users provide coordinates, thickness, and colors to draw shapes or add text to the image.
5. Preview & Download: Processed images are displayed in real-time and can be downloaded using the built-in download button.

Results & Impact

This project provides an intuitive environment to explore core image processing concepts interactively. It is especially useful for beginners learning OpenCV, educators demonstrating transformations in real-time, or anyone needing a lightweight image editing tool for quick adjustments and annotations.

Future Enhancements

Planned improvements include support for additional filters (Gaussian blur, edge detection), layered annotations with undo/redo functionality, batch image processing, and mobile-friendly UI for broader accessibility.

10. M.A.R.S.H.A.L (Mohit’s AgenticAI Representation System for Humanized Assistance and Legacy)

This is this project with which you are interacting.

A voice-first personal AI that lets users talk to “Mohit’s AI voice twin.” The system pairs a React + Web Speech API frontend with a FastAPI backend grounded on your profile markdown. It supports speech-to-text, text-to-speech, LLM provider/model switching (Gemini / Hugging Face), link extraction, and secure key handling—all wrapped in a clean chat UI.

Live Frontend: https://m-a-r-s-h-a-l.vercel.app/ Backend (Vercel): https://m-a-r-s-h-a-l-backend.vercel.app/ Backend (HF Spaces): https://mohitg012-voice-bot-portfolio-backend.hf.space/

---

Project Overview

MARSHAL delivers a polished, browser-based assistant that speaks and listens in real time. Users ask questions via mic (or text), the frontend relays queries to a FastAPI backend, and the backend generates strictly grounded answers from a curated profile markdown. The UI provides provider/model controls, API key inputs, TTS tuning, and a floating card of links extracted from the most recent answer. Designed for demos, portfolios, and hands-free Q&A, it keeps responses concise, first-person, and date-aware.

---

Features

• Voice I/O

  • 🎤 In-browser STT (Web Speech API) with language picker (en-US, en-GB, hi-IN)
  • 🔊 TTS with voice, rate, pitch, and volume controls; URLs auto-stripped for natural speech

• Smart Chat UX

  • 💬 Clean user/assistant bubbles; auto-scroll
  • 🔗 Automatic link extraction + floating “Links in last answer” card
  • 🫧 Animated morphing blob that reacts while speaking

• LLM Controls

  • 🔌 Backend switcher with a safe allow-list
  • 🧠 Provider/model selection: Gemini or Hugging Face (optional manual model override)
  • 🔑 API keys entered in UI (stored in localStorage for dev convenience)

• Grounded Answers

  • Backend prompts models to answer only from profile_data.md
  • Injects today’s date for time/duration awareness

• Prod-Ready Touches

  • CORS controls, HTTPS-friendly design, concise error messages

---

Tech Stack

Frontend & App Framework: React (Vite), Web Speech API, JavaScript (ES2022) Backend & APIs: FastAPI, httpx, Uvicorn, Google Gemini, Hugging Face Inference API Tooling & CI/CD: ESLint, Prettier, GitHub Actions, Vercel deploy (frontend & backend) Data/Context: Markdown knowledge base (Data/profile_data.md) for strict grounding

---

Architecture & Workflow

1. User Input (Mic/Text): Browser captures speech via Web Speech API (STT) → question text.

2. Frontend Request: React app posts { question, provider, model? } to POST /api/chat on the selected backend. Optional keys sent via headers or request body per user choice.

3. Backend Orchestration:

   • Loads Data/profile_data.md at startup.
   • Builds a first-person, grounded prompt with today’s date.
   • Calls Gemini (SDK/REST) or Hugging Face Inference API.

4. Response Processing (Frontend):

   • Receives { answer }, extracts links, strips URLs for TTS clarity.
   • Speaks the final answer (TTS) and shows links in a floating card.

5. UI Feedback: Chat bubbles update; blob animation reflects speaking state.

---

Results & Impact

MARSHAL gives portfolio visitors and recruiters a hands-free, humanized way to explore Mohit’s work. The system proves out a production-lean agentic pattern—voice capture → grounded LLM → friendly speech output—while keeping security conscious (no hardcoded keys) and deployment-ready (static SPA + serverless API).

---

Future Enhancements

• Streaming responses (SSE/WebSockets) for token-by-token UI updates
• Persistent chat sessions with history + “Clear chat”
• Extended provider support (e.g., OpenAI, local models via proxy)
• Advanced mic viz & PTT (push-to-talk)
• Theming & A11y: Light/dark mode, keyboard shortcuts, ARIA labels
• Stronger key hygiene: Session storage, “clear keys” button, optional server-side keys
• Prompt tools: Inline citations back to profile sections

---

Project's Detailed Readme

M.A.R.S.H.A.L (Mohit’s AgenticAI Representation System for Humanized Assistance and Legacy)

MARSHAL Frontend Readme

A sleek, voice-driven React frontend that connects to your Voice Twin Backend (FastAPI) and lets users talk to “Mohit’s AI voice twin”. It supports speech-to-text, text-to-speech, and link extraction, and it can talk to either Gemini or Hugging Face models via your backend.

VERCEL LINK: https://m-a-r-s-h-a-l.vercel.app/

---

1) Features

• 🎤 In-browser STT (Speech-to-Text) with language picker (en-US, en-GB, hi-IN)
• 🔊 TTS (Text-to-Speech) with voice, rate, pitch, and volume controls
• 🔗 Link extraction from the model’s answer + a floating “Links in last answer” card
• 🔌 Backend switcher (drop-down) with a safe allow-list
• 🧠 Provider/model selection (Gemini or Hugging Face; optional manual model override)
• 🔑 API key inputs stored in localStorage for dev convenience
• 💬 Clean chat UI (user/assistant bubbles)
• 🫧 Animated morphing blob that speeds up when speaking

---

2) Architecture & Data Flow

User mic → Web Speech API (STT) → question text
      ↓
Frontend → POST /api/chat (FastAPI backend)
      ↓
Backend → LLM (Gemini or Hugging Face) grounded on profile_data.md
      ↓
Frontend ← { answer }
      ↓
- Extract links, strip URLs for TTS clarity
- Speak final answer with Web Speech API (TTS)
- Show links in floating card

---

3) Directory Structure

Frontend
├── .gitignore
├── README.md
├── eslint.config.js
├── index.html
├── package.json
├── package-lock.json
├── public/
│   └── vite.svg
├── src/
│   ├── App.css                # Main styles (layout, blob, chat, controls)
│   ├── App.jsx                # Top-level app / UI
│   ├── api/
│   │   └── client.js          # askBackend(), extractLinks(), stripLinksFromText()
│   ├── assets/
│   │   └── react.svg
│   ├── components/
│   │   └── blob.jsx           # Morphing blob visualization
│   ├── hooks/
│   │   └── useSpeechToText.js # STT hook (Web Speech API)
│   ├── index.css              # Global CSS resets
│   ├── lib/
│   │   └── useVoices.js       # TTS voices hook (Web Speech API)
│   ├── main.jsx               # React root
│   └── utils/
│       └── extractLinks.js    # (Optional utility – not currently imported)
└── vite.config.js

---

4) Requirements

• Node.js 18+ (recommended)
• A running backend (e.g., your FastAPI service from the Backend README) reachable via HTTPS in production
• Modern browser (Chrome/Edge recommended for STT)

Note: Safari and some mobile browsers have limited/quirky STT support.

---

5) Quick Start (Local)

# 1) Install deps
npm install

# 2) (Optional) Create a .env file in the project root with your defaults
# See the Environment Variables section below

# 3) Start dev server (Vite)
npm run dev

# 4) Open the app
# Vite will print a local URL like: http://localhost:5173

Make sure your backend is running (e.g., http://localhost:8000) and CORS is open to your frontend origin for local development.

---

6) Environment Variables

You can set defaults via Vite env variables (e.g., .env in project root). All are optional—users can override in the UI.

Variable	Meaning	Example
VITE_API_BASE	Default backend base URL	https://m-a-r-s-h-a-l-backend.vercel.app or http://localhost:8000
VITE_PROVIDER	Default provider (gemini or huggingface)	gemini
VITE_MODEL	Optional default model name	gemini-1.5-flash or google/gemma-3-27b-it
VITE_GEMINI_KEY	Default Gemini key (dev only)	AIza...
VITE_HF_KEY	Default Hugging Face key (dev only)	hf_...

Sample .env

VITE_API_BASE=https://m-a-r-s-h-a-l-backend.vercel.app
VITE_PROVIDER=gemini
VITE_MODEL=gemini-1.5-flash
# For dev only; don't commit real keys:
# VITE_GEMINI_KEY=AIza...
# VITE_HF_KEY=hf_...

Keys entered in the UI are saved to localStorage for convenience.

---

7) Production Build & Deployment

# Build
npm run build

# Preview locally
npm run preview

You can host the static dist/ output on:

• Vercel / Netlify / Cloudflare Pages / Firebase Hosting
• Any static hosting (Nginx, S3 + CloudFront, etc.)

---

8) How It Works (Under the Hood)

a) Asking the Backend (src/api/client.js)

askBackend({ baseUrl, question, provider, model, geminiKey, hfKey, sendKeyInBody })

• Builds a POST to POST {baseUrl}/api/chat with JSON { question, provider, model? }.

• Keys are sent in headers by default:

  • Gemini: X-Gemini-Api-Key: <geminiKey>
  • HuggingFace: Authorization: Bearer <hfKey> (or X-Hf-Api-Key)

• If sendKeyInBody: true, it will send { gemini_api_key, hf_api_key } in the body instead.

• Throws on non-2xx with the backend’s error text.

b) STT (Speech-to-Text): src/hooks/useSpeechToText.js

• Uses window.SpeechRecognition / webkitSpeechRecognition.
• Supports continuous mode and live interim results.
• Exposes { supported, recording, finalText, interimText, start, stop }.

c) TTS (Text-to-Speech): src/lib/useVoices.js + window.speechSynthesis

• Enumerates system voices (onvoiceschanged).
• App.jsx lets you pick voice + control rate, pitch, and volume.
• Before speaking, we strip URLs from the answer so TTS doesn’t read raw links.

d) Links: extractLinks() & stripLinksFromText()

• extractLinks(answer) captures both Markdown links and bare URLs.
• stripLinksFromText(answer) converts [label](url) → label and removes bare URLs.
• The “Links in last answer” card shows all found links.

e) Backend allow-list

• The backend dropdown in App.jsx only accepts a whitelisted set (BACKEND_OPTIONS).
• If a saved URL in localStorage isn’t in the allow-list, it falls back to a safe default.

---

9) UI & UX Notes

• Layout: Split view: left = config + STT; right = visual + chat + links card.
• Morphing blob: src/components/blob.jsx + animations in App.css. Blob speeds up while TTS is speaking (.is-speaking class).
• Chat: Simple message bubbles; auto-scroll on new messages.
• Language picker: Switches STT language (e.g., hi-IN for Hindi). TTS voice is independent—pick a voice that matches the language for best pronunciation.

---

10) Security Notes

• Do not commit real API keys to the repo or .env.

• Keys entered in the UI are kept in localStorage (dev convenience). For production:

  • Prefer server-side keys and send requests without exposing keys to the browser.
  • If you must accept user-supplied keys, consider session-only storage and a “clear keys” button.

• Ensure your backend has:

  • Rate-limiting / abuse protection
  • Allowed origins (CORS) locked down
  • Input validation (question length caps, etc.)

---

11) File-by-File Guide

• index.html: Vite entry. Mounts <div id="root" />.

• src/main.jsx: React root; imports App.

• src/App.jsx: Main UI.

  • Backend/provider/model selection
  • API key fields (persisted to localStorage)
  • STT controls (Start, Stop & Ask)
  • Chat area and links card
  • TTS controls (voice/rate/pitch/volume)
  • Calls askBackend() and handles speaking

• src/App.css: Layout, responsive rules, blob animation, chat styles, controls grid, links card.

• src/api/client.js:

  • askBackend() – fetch wrapper
  • extractLinks() – parse Markdown and bare URLs
  • stripLinksFromText() – remove links before TTS

• src/components/blob.jsx: Animated visual that reacts to speaking state.

• src/hooks/useSpeechToText.js: Web Speech API STT hook.

• src/lib/useVoices.js: Loads and returns available TTS voices.

• src/utils/extractLinks.js: (Optional utility; not used in the current imports)

---

12) Troubleshooting

• “Backend error 4xx/5xx” Check that:

  • The backend URL is correct and in the allow-list
  • CORS allows your frontend origin
  • The provider’s API key is present and valid
  • Model name is correct (or leave it blank to use backend defaults)

• STT doesn’t start / “Your browser doesn’t support STT” Some browsers (or insecure origins) block STT. Try Chrome/Edge on desktop, and ensure the page is served over https.

• No voices in TTS selector speechSynthesis.getVoices() can be async. The hook listens to onvoiceschanged. If still empty, try a different browser/OS.

• HTTPS page + HTTP backend (mixed content) Use an HTTPS backend endpoint in production.

• No links detected Ensure the model answer includes proper Markdown links or bare URLs. The links card only shows what’s detected.

---

13) Customization Ideas / Roadmap

• Streaming answers: Show tokens as they arrive (SSE/WebSocket)
• Better chat history: Persist messages and session IDs; add “Clear chat”
• Mic states: Visualizer for input volume; PTT (push-to-talk)
• Key handling: “Clear all keys” button; session storage instead of local
• Theme: Light/Dark toggle; Tailwind migration if preferred
• Accessibility: Keyboard shortcuts for start/stop/speak; ARIA labels
• Provider expansion: Add OpenAI / Gemini streaming / local models via proxy
• Analytics: Gather anonymized UX metrics (with consent)

---

Scripts

• npm run dev – Start Vite dev server
• npm run build – Production build to dist/
• npm run preview – Preview built app locally

Voice Twin Backend — README (FastAPI + Gemini/HuggingFace)

Purpose: A lightweight API that turns your Markdown résumé/profile into a voice-ready Q&A agent. It prompts an LLM to answer only from your provided context, with first-person tone (“I…”) and today’s date awareness.

VERCEL BACKEND LINK : https://m-a-r-s-h-a-l-backend.vercel.app/

HUGGING FACE SPACES LINK : https://mohitg012-voice-bot-portfolio-backend.hf.space/

---

Overview

• Tech stack: FastAPI, httpx

• Providers:

  • Gemini (via google-generativeai SDK, auto-fallback to REST)
  • Hugging Face Inference API (e.g., google/gemma-3-27b-it)

• Context source: Data/profile_data.md (loaded at server start)

• Prompting: Single-message system prompt with strict grounding—the model must answer only from your Markdown.

• Date awareness: Injects today’s date so the model can compute durations/ages.

---

Directory Layout

Backend
├── .gitattributes
├── .gitignore
├── Constants.py                # CONTEXT token/length budget
├── Data/
│   ├── loaded_text.txt         # (optional: staging/diagnostics)
│   ├── profile_data.md         # <<< primary knowledge source
│   ├── profile_data.pdf        # (optional: artifact)
│   └── profile_data.txt        # (optional: artifact)
├── README.md                   # (this file)
├── app.py                      # FastAPI application
├── requirements.txt
└── vercel.json                 # (optional) Vercel config for deploy

---

How It Works

1. Load Context: On boot, the server reads Data/profile_data.md into memory.

2. Build Prompt: For each request, a structured prompt is constructed:

   • First-person voice (“I have worked on…”)
   • Strictly grounded answers (no invention; politely decline if unknown)
   • Optionally insert links found in the context if relevant
   • Include Today’s date for time calculations

3. Call Provider:

   • Gemini: Prefer SDK → falls back to REST if SDK missing
   • Hugging Face: Direct call to Inference API with your model

4. Return Answer: A concise, professional response tailored to the question.

---

Endpoints

a) GET /

Health ping (lightweight).

Response

{ "ok": true, "message": "Voice Agent API (Gemini / Hugging Face)" }

b) GET /api/health

Server and context status (no external calls).

Response

{
  "ok": true,
  "profile_loaded": true,
  "default_context_chars": 12345,
  "providers": { "gemini": "supported", "huggingface": "supported" }
}

c) POST /api/chat

Ask the agent a question. Supports Gemini or Hugging Face.

Request body

{
  "question": "Summarize my projects briefly.",
  "session_id": "demo-1",
  "provider": "gemini",                 // "gemini" | "huggingface"
  "model": "gemini-1.5-flash",          // optional; defaults set per provider
  "gemini_api_key": "YOUR_GEMINI_KEY",  // optional if passed via header/env
  "hf_api_key": "YOUR_HF_KEY"           // optional if passed via header/env
}

Response

{ "answer": "..." }

Headers (optional alternative to body keys)

• X-Gemini-Api-Key: <key>
• X-Hf-Api-Key: <key>
• Authorization: Bearer <hf_key> (HF only)

Provider defaults

• Gemini model default: gemini-1.5-flash
• HF model default: google/gemma-3-27b-it

d) POST /api/debug/prompt

Returns the exact prompt (length + preview) used for your question—useful for debugging.

Request body

{ "question": "Give me a one-line intro about yourself." }

Response

{ "length": 2048, "preview": "You are Mohit Gupta's AI voice twin..." }

---

Request Examples

a) cURL — Gemini

curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "X-Gemini-Api-Key: $GEMINI_API_KEY" \
  -d '{
    "question": "Give a crisp overview of my top 3 projects.",
    "provider": "gemini",
    "model": "gemini-1.5-flash"
  }'

b) cURL — Hugging Face

curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HF_API_KEY" \
  -d '{
    "question": "What are my strengths based on my profile?",
    "provider": "huggingface",
    "model": "google/gemma-3-27b-it"
  }'

c) Axios (JS)

const res = await fetch("http://localhost:8000/api/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-Gemini-Api-Key": process.env.GEMINI_API_KEY
  },
  body: JSON.stringify({
    question: "Explain my MRI project like I'm in an interview.",
    provider: "gemini"
  })
});
const data = await res.json();
console.log(data.answer);

---

Environment & Configuration

a) Constants.py

CONTEXT = 10000  # token/length budget for model output

Used to set max_output_tokens for Gemini and max_new_tokens for HF.

b) Supported env vars

• GEMINI_API_KEY — default Gemini key (fallback if not provided per request)
• DEFAULT_GEMINI_MODEL — override default (e.g., gemini-1.5-flash)
• HF_API_KEY — default HF key (fallback)
• DEFAULT_HF_MODEL — override default (e.g., google/gemma-3-27b-it)

Create a .env (if you use one) and export in your shell or set secrets in your host:

export GEMINI_API_KEY=...
export HF_API_KEY=...
export DEFAULT_GEMINI_MODEL=gemini-1.5-flash
export DEFAULT_HF_MODEL=google/gemma-3-27b-it

---

Local Setup

1. Python

   • Python 3.10+ recommended

2. Install deps

   pip install -r requirements.txt
   

   Minimal expected packages:

   • fastapi
   • uvicorn
   • httpx
   • pydantic
   • google-generativeai (optional but preferred for Gemini SDK)

3. Prepare context

   • Put your content into Data/profile_data.md (see Profile Markdown Guide).

4. Run

   # IMPORTANT: the app instance is named "app" inside app.py
   uvicorn app:app --reload --port 8000
   

   Note: In app.py, the if __name__ == "__main__": block calls uvicorn.run("server:app", reload=True). For local runs, prefer the CLI above (uvicorn app:app --reload). If you do run the file directly, change "server:app" to "app:app" or keep a separate server.py.

5. Docs

   • OpenAPI/Swagger UI: http://localhost:8000/docs
   • ReDoc: http://localhost:8000/redoc

---

Deployment

a) Vercel (Serverless)

• Ensure vercel.json routes Python to ASGI via Vercel’s Python runtime or via a container.

• Set the following Environment Variables in Vercel:

  • GEMINI_API_KEY, HF_API_KEY, DEFAULT_GEMINI_MODEL, DEFAULT_HF_MODEL

• If using serverless functions:

  • Keep cold start in mind (HF may return 503 “model loading”).

• If using a containerized approach:

  • Expose port 8000, run uvicorn app:app --host 0.0.0.0 --port 8000.

b) Other hosts

• Render/Fly.io/EC2/Dokku/Docker: run uvicorn app:app --host 0.0.0.0 --port 8000
• Reverse proxy with Nginx/Caddy and enable TLS.

---

Profile Markdown Guide

Place your content in Data/profile_data.md. Use clear sections:

##### About Me
- I am Mohit Gupta, final-year B.Tech (AI & DS) ...
- Interests: Generative AI, Diffusion Models, RAG, Healthcare AI.

###### Skills
- Python, PyTorch, TensorFlow, FastAPI, LangChain, ...
- MLOps: Docker, MLflow, ...

###### Projects
a) Accelerated MRI Reconstruction (Diffusion Models)
- Goal: Speed up acquisition; current PSNR ~30, SSIM ~0.90
- Tech: PyTorch, FFT, k-space undersampling
- Highlights: ...

b) Medical Workflow Assistant (Agentic + RAG)
- Multi-modal, voice-enabled, Streamlit UI
- Features: patient registration, medicine check, ...
- Links: https://github.com/...  (include any link you want surfaced)

###### Experience
1. SAMEER + IIT Delhi (AI Intern)
.
.
etc and soon you can add about yourself in this project to tell more about yourself to Agent.
- ...

###### Achievements
- Kaggle: ...
- Hackathons: ...

###### Contact
- Email: ...
- LinkedIn: ...
- GitHub: ...

Tips

• Keep it truthful and concise.
• Add links you want the model to include when relevant.
• Update the file whenever your profile changes; restart the server to reload.

---

Error Handling & Responses

• 400 — Missing inputs (e.g., no question, no API key for chosen provider)
• 502 — Provider returned empty/invalid output or unexpected format
• 503 — HF model warming/loading (try again)
• 200 — { "answer": "..." }

Examples

• Gemini empty candidates → HTTPException(502, "Gemini returned no candidates: ...")
• HF warmup → HTTPException(503, "Hugging Face model is loading. Please retry.")

---

Security Notes

• Never hardcode keys in the repo. Pass via headers or environment variables.

• Prefer headers from frontend to avoid storing keys server-side:

  • X-Gemini-Api-Key (Gemini)
  • Authorization: Bearer <hf_key> or X-Hf-Api-Key (HF)

• Rate limit and auth (JWT/API key) if exposing publicly.

• Validate question length if needed; consider basic abuse/flood protection.

---

Customization & Extensibility

• Change default models: set DEFAULT_GEMINI_MODEL / DEFAULT_HF_MODEL.

• Adjust output length/creativity:

  • Constants.py → CONTEXT
  • Gemini generation_config: {"temperature": 0.2, "max_output_tokens": CONTEXT}
  • HF parameters: {"max_new_tokens": CONTEXT, "temperature": 0.2, "repetition_penalty": 1.1}

• Add providers: Create a new call_<provider>() method and branch in /api/chat.

• Tighten grounding: Expand prompt rules (e.g., “cite section headers”).

• Add streaming: Implement server-sent events (SSE) for token streaming if desired.

---

Troubleshooting

• profile_loaded: false: Ensure Data/profile_data.md exists and is readable.
• Gemini key error (400): Provide gemini_api_key or header X-Gemini-Api-Key or set GEMINI_API_KEY.
• HF 503 (loading): Try again; cold starts are common on first call.
• Running uvicorn fails with server:app: Use uvicorn app:app --reload.
• CORS errors in browser: Set allow_origins to your frontend’s origin (e.g., ["https://your-frontend.app"]).

---

Quick Start (Copy/Paste)

# 1) Install
pip install -r requirements.txt

# 2) Env
export GEMINI_API_KEY=...
export HF_API_KEY=...

# 3) Put your content
# edit Data/profile_data.md

# 4) Run
uvicorn app:app --reload --port 8000

# 5) Test
curl -s http://localhost:8000/api/health | jq
curl -s -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "X-Gemini-Api-Key: $GEMINI_API_KEY" \
  -d '{"question":"Give me a one-line intro about yourself.","provider":"gemini"}' | jq

---

M.A.R.S.H.A.L. Automated Pipeline

A full-stack, voice‑ready personal agent that answers strictly from your Markdown profile. This repo includes:

• Backend: FastAPI service that grounds LLM answers on Data/profile_data.md and supports Gemini & Hugging Face providers.
• Frontend: Sleek React/Vite app with Speech‑to‑Text (STT), Text‑to‑Speech (TTS), link extraction, and provider/model selection.
• CI/CD: Jenkins pipeline that auto‑builds, tests, and deploys on each GitHub push.

---

🔗 Live Links

• Frontend (Vercel): https://m-a-r-s-h-a-l.vercel.app/
• Backend (Vercel): https://m-a-r-s-h-a-l-backend.vercel.app/
• Backend (Hugging Face Space): https://mohitg012-voice-bot-portfolio-backend.hf.space/
• Jenkins (HF Space): https://mohitg012-jenkins-setup.hf.space/

Frontend can switch between allowed backends at runtime.

---

🧭 Repository Layout

.
├── .gitignore
├── Backend
│   ├── .gitattributes
│   ├── .gitignore
│   ├── Constants.py
│   ├── Data
│   │   ├── loaded_text.txt
│   │   ├── profile_data.md     # Primary knowledge source (grounding)
│   │   ├── profile_data.pdf    # Optional artifact
│   │   └── profile_data.txt    # Optional artifact
│   ├── README.md               # Backend‑only readme (subset of this)
│   ├── app.py                  # FastAPI app (ASGI: "app")
│   ├── requirements.txt
│   └── vercel.json             # Optional Vercel config
│
├── HF_Backend
│   └── Voice_Bot_Portfolio_Backend
│       ├── .gitattributes
│       ├── .gitignore
│       ├── Constants.py
│       ├── Data
│       │   ├── loaded_text.txt
│       │   ├── profile_data.md
│       │   ├── profile_data.pdf
│       │   └── profile_data.txt
│       ├── Dockerfile          # Containerized backend for HF Spaces
│       ├── README.md
│       ├── app.py
│       └── requirements.txt
│
├── Jenkinsfile                 # CI/CD pipeline for mono‑repo
├── Readme.md                   # (You are here)
└── speech-lab                  # Frontend
    ├── .gitignore
    ├── README.md
    ├── eslint.config.js
    ├── index.html
    ├── package-lock.json
    ├── package.json
    ├── public
    │   └── vite.svg
    ├── src
    │   ├── App.css
    │   ├── App.jsx
    │   ├── api
    │   │   └── client.js
    │   ├── assets
    │   │   └── react.svg
    │   ├── components
    │   │   └── blob.jsx
    │   ├── hooks
    │   │   └── useSpeechToText.js
    │   ├── index.css
    │   ├── lib
    │   │   └── useVoices.js
    │   ├── main.jsx
    │   └── utils
    │       └── extractLinks.js
    └── vite.config.js

---

🧱 Tech Stack

Backend

• FastAPI (ASGI via Uvicorn)
• httpx (HTTP client)
• pydantic v2 (request/response models)
• google‑generativeai SDK (Gemini) with REST fallback
• Hugging Face Inference API
• CORS middleware

Frontend

• React 18 + Vite
• Web Speech API: STT (SpeechRecognition) + TTS (speechSynthesis)
• Clean chat UI, provider/model switcher, link extraction

CI/CD & Deploy

• Jenkins (pipeline on each GitHub push)
• Vercel (frontend + Python serverless or container proxy)
• Hugging Face Spaces (containerized backend)

---

🗺️ System Overview

User mic → Web Speech API (STT) → text question
        ↓
Frontend (React/Vite) ─── POST /api/chat ───▶ Backend (FastAPI)
        │                                          │
        │                                      Build prompt:
        │                                      - strict grounding from profile_data.md
        │                                      - first‑person voice ("I…")
        │                                      - inject Today’s date
        │                                          │
        │                               ┌──────────┴──────────┐
        │                               ▼                     ▼
        │                         Gemini provider      HF Inference API
        │                               │                     │
        ◀────────── { answer } ◀───────┴─────────────────────┘
                    - extract links                      
                    - strip URLs for TTS                 
                    - speak via TTS                      
                    - show links card                    

---

✨ Features

• Voice Twin answers in first‑person (“I…”) from your Markdown.
• Strict grounding: If info isn’t in context, it politely declines.
• Date‑aware: Today’s date injected for durations/ages.
• Switchable providers: Gemini or HF (e.g., google/gemma-3-27b-it).
• STT/TTS: Speak to it; it speaks back (voices, rate, pitch, volume).
• Link extraction: Floating card shows links detected in last answer.
• CORS‑ready: Configurable allow‑origins.

---

🔒 Security Notes (Read First)

• Never commit real API keys. Use env vars or pass keys via headers from the frontend.

• Supported headers:

  • X-Gemini-Api-Key: <key>
  • Authorization: Bearer <hf_key> (or X-Hf-Api-Key: <key>) for HF

• Optionally send in body: { gemini_api_key, hf_api_key } (dev‑only convenience).

• Set CORS to your frontend origin in production.

• Add basic rate limiting and input length checks if exposing publicly.

---

🧩 Backend (FastAPI)

a) Environment

Backend/Constants.py

CONTEXT = 10000  # token/length budget for model output

Supported env vars:

• GEMINI_API_KEY (fallback if header/body missing)
• DEFAULT_GEMINI_MODEL (default: gemini-1.5-flash)
• HF_API_KEY (fallback)
• DEFAULT_HF_MODEL (default: google/gemma-3-27b-it)

b) Run Locally

cd Backend
python -m venv .venv && source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt
# Put your content into Data/profile_data.md
uvicorn app:app --reload --port 8000

Docs:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

c) Endpoints

GET / – health ping

{ "ok": true, "message": "Voice Agent API (Gemini / Hugging Face)" }

GET /api/health – server + context status

{
  "ok": true,
  "profile_loaded": true,
  "default_context_chars": 12345,
  "providers": { "gemini": "supported", "huggingface": "supported" }
}

POST /api/chat – ask a question

{
  "question": "Summarize my projects briefly.",
  "session_id": "demo-1",
  "provider": "gemini",           // "gemini" | "huggingface"
  "model": "gemini-1.5-flash",    // optional; defaults set per provider
  "gemini_api_key": "...",        // optional if provided via header/env
  "hf_api_key": "..."             // optional if provided via header/env
}

Response:

{ "answer": "..." }

POST /api/debug/prompt – returns the exact prompt preview/length

{ "question": "Give me a one-line intro about yourself." }

d) Curl Examples

Gemini:

curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "X-Gemini-Api-Key: $GEMINI_API_KEY" \
  -d '{
    "question": "Give a crisp overview of my top 3 projects.",
    "provider": "gemini",
    "model": "gemini-1.5-flash"
  }'

Hugging Face:

curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HF_API_KEY" \
  -d '{
    "question": "What are my strengths based on my profile?",
    "provider": "huggingface",
    "model": "google/gemma-3-27b-it"
  }'

e) Deployment

Vercel (serverless / ASGI)

• Expose app:app (ASGI) and ensure Python runtime.
• Set env vars: GEMINI_API_KEY, HF_API_KEY, DEFAULT_*.
• Cold starts may affect first response for HF (503 warmup); handle retries.

Hugging Face Spaces (Docker)

• See HF_Backend/Voice_Bot_Portfolio_Backend/Dockerfile.
• Build args: Python 3.10+, install requirements, run uvicorn app:app --host 0.0.0.0 --port 7860.
• Space shows the app at / and proxies to 7860.

---

🎛️ Frontend (speech‑lab)

a) Dev Setup

cd speech-lab
npm install
npm run dev  # http://localhost:5173

b) Env (Vite)

All optional – users can override in the UI.

Var	Meaning	Example
VITE_API_BASE	Default backend base URL	https://m-a-r-s-h-a-l-backend.vercel.app
VITE_PROVIDER	gemini or huggingface	gemini
VITE_MODEL	Default model	gemini-1.5-flash
VITE_GEMINI_KEY	Dev default key (don’t commit)	AIza...
VITE_HF_KEY	Dev default key (don’t commit)	hf_...

Sample .env

VITE_API_BASE=https://m-a-r-s-h-a-l-backend.vercel.app
VITE_PROVIDER=gemini
VITE_MODEL=gemini-1.5-flash

c) Production Build

npm run build
npm run preview

Deploy static dist/ to Vercel/Netlify/CF Pages/etc. Ensure backend allows the frontend origin in CORS.

d) UX Details

• STT via useSpeechToText.js (continuous mode + interim results).
• TTS via useVoices.js (voice/rate/pitch/volume controls). URLs are stripped before speech.
• Link extraction shown in a floating card.
• Backend allow‑list prevents arbitrary URLs.

---

🔁 CI/CD with Jenkins

a) What Triggers a Build?

• GitHub webhook on push/PR → Jenkins job → Pipeline defined in Jenkinsfile.

b) Recommended Stages

1. Checkout – fetch repo.

2. Backend: Lint/Test – pip install -r Backend/requirements.txt, run basic tests.

3. Frontend: Lint/Build – npm ci && npm run build in speech-lab.

4. Containerize HF Backend – build/push Space image from HF_Backend/Voice_Bot_Portfolio_Backend (optional if Space is git‑pushed).

5. Deploy –

   • Backend (HF Space +/or Vercel)
   • Frontend (Vercel)

6. Smoke Tests – hit GET /api/health, run a sample POST /api/chat.

7. Notify – Slack/email on result.

c) Example Jenkinsfile (annotated)

pipeline {
  agent any
  environment {
    NODE_OPTIONS = '--max_old_space_size=4096'
    PYTHON = 'python3'
    VERCEL_TOKEN = credentials('VERCEL_TOKEN')
    GEMINI_API_KEY = credentials('GEMINI_API_KEY')
    HF_API_KEY = credentials('HF_API_KEY')
  }
  stages {
    stage('Checkout') {
      steps { checkout scm }
    }

    stage('Backend: Install & Lint/Test') {
      steps {
        dir('Backend') {
          sh 'python -m venv .venv && . .venv/bin/activate && pip install -r requirements.txt'
          // TODO: add pytest or simple import checks
        }
      }
    }

    stage('Frontend: Install & Build') {
      steps {
        dir('speech-lab') {
          sh 'npm ci'
          sh 'npm run build'
          archiveArtifacts artifacts: 'dist/**/*', fingerprint: true
        }
      }
    }

    stage('HF Space Backend: Build & Deploy (Docker)') {
      when { expression { return fileExists('HF_Backend/Voice_Bot_Portfolio_Backend/Dockerfile') } }
      steps {
        dir('HF_Backend/Voice_Bot_Portfolio_Backend') {
          // Example: git push to the HF Space repo OR use huggingface-cli
          // sh 'huggingface-cli login --token ${HF_API_KEY}'
          // sh 'huggingface-cli space runtime restart <owner>/<space-name>'
        }
      }
    }

    stage('Deploy Frontend (Vercel)') {
      steps {
        dir('speech-lab') {
          sh 'npx vercel --token ${VERCEL_TOKEN} --prod --confirm'
        }
      }
    }

    stage('Smoke Tests') {
      steps {
        sh 'curl -s https://m-a-r-s-h-a-l-backend.vercel.app/api/health'
      }
    }
  }
  post {
    always { emailext to: 'you@example.com', subject: "${currentBuild.currentResult}: ${env.JOB_NAME} #${env.BUILD_NUMBER}", body: 'See Jenkins for details.' }
  }
}

Adapt deploy steps to your exact secrets/targets. If your HF Space mirrors this repo, a simple git push to the Space is often enough to trigger a rebuild.

d) Caching Tips

• Use Jenkins node/npm and pip caches (eg. ~/.cache/pip, ~/.npm).
• Archive speech-lab/dist as a build artifact for debugging rollbacks.

---

⚙️ Configuration Reference

a) Backend Provider Defaults

• Gemini model default: gemini-1.5-flash
• HF model default: google/gemma-3-27b-it

b) Headers

• X-Gemini-Api-Key: <key>
• Authorization: Bearer <hf_key> or X-Hf-Api-Key: <key>

c) Prompting Rules (server‑side)

• First person (“I…”) persona from your profile.
• Answer only from profile_data.md.
• Refuse/redirect politely if unknown.
• Optionally include relevant links present in the context.
• Inject Today’s date in prompt.

---

🧪 Quick Smoke Tests

Backend health

curl -s https://m-a-r-s-h-a-l-backend.vercel.app/api/health | jq

Ask a question (Gemini)

curl -s -X POST https://m-a-r-s-h-a-l-backend.vercel.app/api/chat \
  -H "Content-Type: application/json" \
  -H "X-Gemini-Api-Key: $GEMINI_API_KEY" \
  -d '{"question":"Explain my MRI project like in an interview.","provider":"gemini"}' | jq

Ask a question (HF)

curl -s -X POST https://m-a-r-s-h-a-l-backend.vercel.app/api/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $HF_API_KEY" \
  -d '{"question":"What are my strengths based on my profile?","provider":"huggingface"}' | jq

---

🧯 Troubleshooting

• profile_loaded: false – Ensure Data/profile_data.md exists and is readable; restart server.
• Gemini 400 – Provide valid X-Gemini-Api-Key or set GEMINI_API_KEY env.
• HF 503 (loading) – First call can warm the model; retry.
• CORS errors – Lock allow_origins to your frontend domain in FastAPI.
• Mixed content (HTTPS/HTTP) – Serve both frontend and backend over HTTPS in prod.
• STT unsupported – Some browsers block STT; use Chrome/Edge desktop, secure origin.

---

🗺️ Roadmap

• Token streaming (SSE/WebSocket) to the frontend.
• Session persistence + chat history.
• Stronger prompt‑level citations (section/page labels).
• Rate limiting and API keys for public access.
• Prometheus + Grafana dashboards; alerting on deploy failures.
• Optional local model support via gateway.

---

ADDITIONAL INFORMATION

LIFE STORY / BACKGROUND

I grew up with a deep curiosity about technology and problem-solving, which naturally led me toward artificial intelligence. During my undergraduate studies in AI and Data Science at the University School of Automation & Robotics, GGSIPU, I discovered my passion for building end-to-end AI systems — from data pipelines to deploying real-world solutions. Over the past few years, I’ve worked on projects ranging from fraud detection and medical imaging to fashion recommendation systems, blending creativity with technical rigor. What excites me most is transforming complex problems into scalable AI-driven products that can create real-world impact.

---

SUPERPOWER / STRENGTHS

My biggest strength is my ability to learn quickly and build complete AI solutions from scratch. I thrive in situations where I need to integrate multiple technologies — whether it’s combining deep learning models with MLOps pipelines or creating multimodal systems like my Fashion Sense AI project. I’m also known for being hardworking and adaptable; I can switch between research-heavy work, like diffusion models for MRI, and production-focused tasks, like deploying scalable APIs with Kubernetes.

---

AREAS OF GROWTH

I am actively working on improving three key areas: leadership skills to effectively manage AI teams and mentor juniors, advanced research in reinforcement learning and agentic AI systems, and scaling distributed AI solutions for production environments. These areas align with my long-term vision of contributing to high-impact AI products at scale.

---

WORK STYLE AND COLLABORATION

I am a proactive and organized team member who values clear communication and collaboration. I enjoy brainstorming ideas openly with teammates and believe in constructive feedback to drive innovation. When challenges or tight deadlines arise, I prioritize breaking down problems into smaller actionable steps and stay calm under pressure, ensuring high-quality results without compromising timelines.

---

MISCONCEPTIONS / UNIQUE TRAITS

A common misconception about me is that I am very serious because I focus deeply on work, especially when solving complex AI problems. In reality, I’m playful and enjoy creative brainstorming sessions, whether it’s during hackathons, casual team discussions, or even while playing basketball with friends.

---

PUSHING BOUNDARIES

I consistently push my boundaries by participating in hackathons, Kaggle competitions, and side projects. For instance, I developed a knowledge graph-powered chatbot from scratch in just a few weeks, and I’ve challenged myself to build multimodal systems like Fashion Sense AI that combine vision, language, and retrieval models. These projects help me stay at the cutting edge of AI while continuously expanding my technical and creative skill set.

---

HOBBIES AND PERSONALITY

Outside of work, I am passionate about basketball, traveling to new places, and enjoying music, especially during long coding sessions. These hobbies help me maintain balance and creativity, often inspiring fresh ideas when I return to solving technical problems.

---

VISION / FUTURE GOALS

In the next three to five years, I aim to lead AI product development teams that build impactful and scalable solutions in fields like healthcare, finance, or fashion technology. I want to bridge the gap between cutting-edge AI research and user-friendly products, ensuring that innovations are both ethical and accessible.

---

CORE VALUES

I am driven by innovation, continuous learning, and the desire to create meaningful impact through technology. Ethical AI development and scalability are central to my approach. I believe technology should solve real problems responsibly while inspiring trust and accessibility for everyone.