Spaces:

Vishu2006
/

SmartHire-AI

Running

App Files Files Community

SmartHire-AI / README.md

Vishu2006

Fix: resolve merge conflict, add HF Spaces YAML metadata

bb780c0 14 days ago

preview code

Raw

History Blame Contribute Delete

12.3 kB

metadata

title: SmartHire AI
emoji: 🎯
colorFrom: indigo
colorTo: purple
sdk: docker
pinned: false
license: mit
short_description: Transformer-based resume and job description matching API

🤖 SmartHire AI: Transformer-Based Resume & Job Matching System

ATS-inspired AI recruitment system that matches candidate resumes with job descriptions using fine-tuned Sentence Transformer embeddings and cosine semantic similarity — going far beyond simple keyword matching.

📌 Project Overview

SmartHire AI is a production-style HRTech application that demonstrates:

Transformer-based NLP using all-MiniLM-L6-v2 (fine-tuned on 127 resume–JD pairs across 41 job roles)
Semantic understanding via mean-pooled sentence embeddings
Cosine similarity scoring — context-aware, not keyword-matching
Candidate ranking from multiple simultaneous resume uploads
Skill gap analysis with critical missing skill detection (300+ skill vocabulary)
Persistent vector index (ChromaDB/NumPy) for instant sub-100ms resume search
REST API (FastAPI) for frontend integration
Interactive recruiter dashboard built with Streamlit + Plotly

📊 Fine-Tuning Results

Metric	Value
Pearson r	0.9733
Spearman ρ	0.9604
Strong Match Accuracy	98%
Partial Match Accuracy	47%
Mismatch Accuracy	100%
Overall 3-tier Accuracy	81.25%
Fine-tuning Gain	+9.4%

Fine-tuned on 127 pairs across 41 job roles: 43 Strong (34%), 40 Partial (31%), 44 Mismatch (35%)

🏗️ Project Structure

SmartHireAI/
│
├── app/
│   └── streamlit_app.py       # Full Streamlit dashboard (dark mode, port 8501)
│
├── api/
│   ├── __init__.py
│   ├── main.py                # FastAPI REST API server (port 8000)
│   └── README.md              # Full API endpoint documentation
│
├── src/
│   ├── __init__.py
│   ├── parser.py              # PDF, DOCX, TXT resume parser
│   ├── preprocess.py          # Text cleaning & normalization pipeline
│   ├── model.py               # Sentence Transformer embedding model
│   ├── similarity.py          # Cosine similarity & calibrated scoring
│   ├── skills.py              # Skill extraction & gap analysis (300+ skills)
│   ├── ranking.py             # Candidate ranking & export
│   └── vector_store.py        # ChromaDB/NumPy persistent vector index
│
├── train/
│   └── training_data.json     # 127 labeled resume–JD pairs (41 job roles)
│
├── datasets/
│   ├── sample_jd.txt
│   ├── candidate_alice.txt
│   ├── candidate_bob.txt
│   ├── candidate_carol.txt
│   └── candidate_david.txt
│
├── finetune.py                # Fine-tuning script (CosineSimilarityLoss, 6 epochs)
├── evaluate.py                # Evaluation (Pearson r, Spearman ρ, 3-tier accuracy)
├── diagnose.py                # Calibration diagnostics
├── requirements.txt
├── RUN_APP.bat                # Windows: launch Streamlit UI
├── RUN_API.bat                # Windows: launch FastAPI server
├── SETUP_AND_RUN.bat          # Windows: first-time setup
└── main.py                    # CLI entry point

⚡ Quick Start

1. Clone the Repository

git clone https://github.com/Vishu200672/SmartHire-AI.git
cd SmartHire-AI

2. Create a Virtual Environment

python -m venv venv
source venv/bin/activate        # Linux / macOS
# .\venv\Scripts\activate       # Windows

3. Install Dependencies

pip install -r requirements.txt

⚠️ First run downloads the embedding model (~90 MB). Subsequent runs use the HuggingFace cache.

4. Run the Streamlit Dashboard

streamlit run app/streamlit_app.py
# → http://localhost:8501

5. Run the REST API

uvicorn api.main:app --host 0.0.0.0 --port 8000 --reload
# → http://localhost:8000
# → http://localhost:8000/docs  (interactive Swagger UI)

Both servers can run simultaneously — they share the same src/ model.

6. Run the CLI Demo

python main.py --demo
# Or with your own files:
python main.py --resume resume.pdf --jd job_description.txt

🌐 REST API

SmartHire AI includes a full FastAPI REST API for integrating the matching engine into any frontend (React, Next.js, Vue, Node.js, etc.).

Base URL

http://localhost:8000

Interactive Docs

http://localhost:8000/docs      ← Swagger UI (try all endpoints in browser)
http://localhost:8000/redoc     ← Redoc

Key Endpoints

Method	Endpoint	Description
GET	`/health`	Health check
GET	`/model/info`	Loaded model metadata
POST	`/match`	Match resumes vs JD — main endpoint
POST	`/skills`	Skills-only analysis
POST	`/index/build`	Build persistent vector index
POST	`/index/search`	Instant search against index (<100ms)
GET	`/index/info`	Index stats
GET	`/index/candidates`	List indexed resumes
POST	`/index/add`	Add single resume to index
DELETE	`/index/clear`	Clear index
POST	`/parse`	Parse file → raw text
POST	`/embed`	Get embedding vector for any text

Example — Match Resumes (JavaScript)

const form = new FormData();
form.append("resumes", resumeFile1);
form.append("resumes", resumeFile2);
form.append("jd_text", "Looking for Python ML Engineer with PyTorch...");
form.append("similarity_weight", "0.7");

const res  = await fetch("http://localhost:8000/match", {
  method: "POST",
  body: form,
});
const data = await res.json();
// data.candidates → ranked list with scores, skills, recommendations

Example Response

{
  "status": "success",
  "duration_sec": 1.23,
  "total_candidates": 2,
  "summary": {
    "average_score": 72.5,
    "highest_score": 85.0,
    "highly_recommended": 1,
    "recommended": 1
  },
  "candidates": [
    {
      "rank": 1,
      "name": "John_Doe",
      "score_pct": 85.0,
      "semantic_similarity": 91.2,
      "skill_coverage_pct": 75.0,
      "recommendation": "Highly Recommended",
      "matching_skills": ["python", "pytorch", "docker"],
      "missing_skills": ["kubernetes"],
      "critical_missing": [],
      "ai_insight": "Strong contextual alignment with the JD..."
    }
  ]
}

See api/README.md for full endpoint documentation.

🖥️ Streamlit Dashboard Features

Tab	Features
Upload & Analyze	Upload PDF/DOCX/TXT resumes, paste or upload JD, run pipeline
Match Results	Score distribution bar chart, scatter plot, per-candidate cards
Skill Gap Analysis	Matching/missing/critical skill chips, skill matrix chart
Candidate Ranking	Leaderboard table, gauge chart for top candidate, CSV export
Vector Index	Build/search persistent resume index, instant JD search

🗄️ Vector Index

SmartHire AI includes a persistent vector index that pre-encodes resumes so JD search is instant:

Normal flow:   Upload resumes → encode each (~0.06s each) → compare → results
Vector index:  Index resumes once → search any JD → results in <100ms

Backends supported:

ChromaDB (recommended) — pip install chromadb
NumPy flat-file (automatic fallback) — no extra install needed

Usage via API:

# Index resumes once
curl -X POST http://localhost:8000/index/build \
  -F "resumes=@resume1.pdf" -F "resumes=@resume2.docx"

# Search instantly for any JD
curl -X POST http://localhost:8000/index/search \
  -F "jd_text=Python ML Engineer with PyTorch experience" \
  -F "top_k=5"

🧠 How It Works

Architecture Pipeline

Resume (PDF/DOCX/TXT)
        │
        ▼
[parser.py] Extract raw text
        │
        ▼
[preprocess.py] Normalize → clean → chunk (400 tokens, 50 overlap)
        │
        ▼
[model.py] Tokenize → forward pass → mean pooling → L2 normalize → embedding
        │
        ├──────────────────────────────────┐
        ▼                                  ▼
[similarity.py]                    [skills.py]
Cosine similarity vs JD            Skill extraction (300+ vocab)
Calibrated score 0–100%           Gap analysis (matching/missing/critical)
        │                                  │
        └──────────────┬───────────────────┘
                       ▼
              [ranking.py]
              Composite score = 70% semantic + 30% skill
              Sort → Recommendation tier → AI insight

Composite Ranking Score

Final Score = (Semantic Similarity × 0.70) + (Skill Coverage × 0.30)

Weights are configurable via API parameter or Streamlit sidebar slider.

🎯 Recommendation Tiers

Score	Recommendation	Action
≥ 60%	🟢 Highly Recommended	Fast-track to interview
38–60%	🔵 Recommended	Schedule screening call
18–38%	🟠 Consider	Review manually
< 18%	🔴 Not Recommended	Archive

🛠️ Tech Stack

Component	Technology
Core Model	Fine-tuned `all-MiniLM-L6-v2` (Sentence Transformers)
DL Framework	PyTorch 2.0+
NLP Library	Hugging Face Transformers + Sentence-Transformers
REST API	FastAPI + Uvicorn
Vector Store	ChromaDB / NumPy
Web App	Streamlit
Charts	Plotly
PDF Parsing	pdfplumber + PyPDF2
DOCX Parsing	python-docx
Data	Pandas, NumPy

🚀 Performance Benchmarks

Operation	Time (CPU)
Model load (first time)	~5–10s
Encode 1 resume	~0.06s
Encode 60 resumes	~4–5s
Vector index search	<100ms
Skill gap analysis	<0.01s per candidate

📝 Module Documentation

Each module is fully documented with:

Google-style docstrings
Python type hints throughout
logging at every pipeline step
Meaningful error messages

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/your-feature)
Commit changes (git commit -m "Add: your feature")
Push to branch (git push origin feature/your-feature)
Open a Pull Request

📄 License

This project is licensed under the MIT License — see LICENSE for details.

🙏 Acknowledgements

Hugging Face for Transformers and all-MiniLM-L6-v2
Sentence Transformers for the fine-tuning framework
FastAPI for the API framework
Streamlit for the dashboard framework
Plotly for interactive charts
ChromaDB for the vector store

📬 Contact

Built as a portfolio project demonstrating Transformer-based NLP, semantic search, fine-tuning, REST API design, and production ML engineering practices.

GitHub: github.com/Vishu200672/SmartHire-AI HF Space: huggingface.co/spaces/Vishu2006/SmartHire-AI