sutd-bge-large-ft67

Finetuned BAAI/bge-large-en-v1.5 for job-description-to-module retrieval in the SUTD Course Recommendation Chatbot (MLOps Group 9).

Given a job description, this model retrieves relevant SUTD elective modules. It is used as the dense retrieval backbone in the RAG and Hybrid pipelines.

Model Details

Property	Value
Base model	BAAI/bge-large-en-v1.5
Embedding dimension	1024
Max sequence length	512
Similarity function	Cosine
Loss	MultipleNegativesRankingLoss

Training Data

Trained on 67 hand-annotated (job description, relevant SUTD module) pairs spanning four pillars: ASD, EPD, ESD, ISTD/CSD. Each job description was matched to one or more relevant modules. After train/validation splitting and hard-negative expansion by the Sentence Transformers trainer, this produces 601 training samples and 66 validation samples.

A version trained on the augmented 98-pair dataset is available at henreads/sutd-bge-large-aug98.

Training Setup

• Hardware: Modal A10G (24 GB VRAM)
• Epochs: up to 10 with early stopping (patience 4); converged at epoch 3
• Effective batch size: 16 (per-device batch 4, gradient accumulation 4)
• Learning rate: 2e-5
• Tracking: Weights & Biases (sutd-mlops-bge-finetune)

Evaluation

Evaluated on a 10-job held-out retrieval set (completely separate from training). NDCG@10 improves from 0.679 (base BGE-large) to 0.747 with ft67 finetuning.

Model	NDCG@10
BGE-large-en-v1.5 (base)	0.679
sutd-bge-large-ft67 (this model)	0.747
sutd-bge-large-aug98	0.770

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("henreads/sutd-bge-large-ft67")

job_description = "Data Scientist at GovTech. Build ML models with Python..."
module_passage = "50.007 Machine Learning — Topics: supervised learning, neural networks..."

embeddings = model.encode([job_description, module_passage], normalize_embeddings=True)
similarity = embeddings[0] @ embeddings[1]
print(similarity)

Project

Part of the SUTD Course Recommendation Chatbot — MLOps Group 9.
Code: github.com/henreads/sutd-mlops-group9