meet5568/bhash_finetune

BhashaEmbed Hindi Songs — TSDAE Domain Adapted

Domain-adapted version of AkshitaS/bhasha-embed-v0 for Hindi film song lyrics.

Training

• Method: TSDAE (Transformer-based Sequential Denoising Auto-Encoder), unsupervised
• Quantization: 8-bit (load_in_8bit=True) + LoRA adapters (r=16, alpha=32)
• Corpus: 8,464 Hindi film songs, 28,713 verse-level chunks
• Scripts: Devanagari, Romanized Hindi (Hinglish), English — single unified model
• Script balancing: weighted sampling to 44/44/12 (Devanagari/Hinglish/English)
• Epochs: 3
• Batch size: 4
• Learning rate: 0.0001
• Max sequence length: 400 subword tokens

Intended Use

Sentence embeddings for Hindi film song lyrics across all three script forms. Designed for thematic classification and generational/temporal analysis of Hindi music.

Limitations

• Trained with LoRA on a quantized backbone — domain adaptation is weaker than full fine-tuning (approximately 70–85% of full-FT gains)
• No supervised thematic fine-tuning applied (Phase A only)
• Not evaluated on tasks outside Hindi song lyrics

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("meet5568/bhash_finetune")

lyrics = [
    "तेरे बिना जिंदगी से कोई शिकवा नहीं",
    "tere bina zindagi se koi shikwa nahi",
    "without you life has no complaint",
]
embeddings = model.encode(lyrics)
print(embeddings.shape)  # (3, 768)