aryanx16/mahabert-v2-daft

🏷️ MahaBERT v2 — DAFT Fine-Tuned Model

Authors: Tanvi Somani , Aryan Babare , Samyak Bora
Model Type: Masked Language Model (MLM)
Base Model: MahaBERT v2
Training Method: DAFT (Domain-Adaptive Fine-Tuning)
Language: Marathi 🇮🇳
Framework: HuggingFace Transformers

---

📌 Overview

mahabert-v2-daft is a Domain-Adaptive Fine-Tuned (DAFT) version of the original MahaBERT v2 model.
The model has been further trained on a large collection of unlabeled Marathi text to adapt it to the specific domain used in downstream Marathi NLP tasks such as:

• Sentiment analysis
• Text classification
• Emotion detection
• Topic analysis
• General Marathi language understanding tasks

DAFT helps the model learn domain-specific vocabulary, patterns, and semantics, improving downstream task performance without requiring labeled data.

---

🧠 What is DAFT?

DAFT (Domain Adaptive Fine-Tuning) is a continued pre-training method where:

• You take a pretrained language model
• Feed it large amounts of unlabeled domain text
• Train it again using the Masked Language Modeling (MLM) objective

This process improves the model’s understanding of domain-specific words, idioms, and sentence structures.

---

🚀 Training Details

Item	Details
Base Model	MahaBERT v2
Method	DAFT (continued pre-training)
Objective	Masked Language Modeling (MLM)
Dataset	Unlabeled Marathi text (domain-specific)
Batch Size	As per Colab training setup
Epochs	Several thousand steps (as seen in training logs)
Hardware	Google Colab (T4 GPU)
Optimizer	AdamW
Precision	FP32

---

📁 Files Included

The repository contains:

• model.safetensors — model weights
• config.json — model architecture
• tokenizer.json, tokenizer_config.json — tokenizer settings
• vocab.txt — BERT vocabulary
• special_tokens_map.json — CLS, SEP, PAD, MASK tokens
• training_args.bin — training configuration

---

🧩 Usage

🔹 Load the Model

from transformers import AutoTokenizer, AutoModelForMaskedLM

model_id = "aryanx16/mahabert-v2-daft"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)