File size: 925 Bytes
2403e67 cc86ef3 2403e67 cc86ef3 2403e67 cc86ef3 2403e67 cc86ef3 2403e67 cc86ef3 2403e67 cc86ef3 2403e67 cc86ef3 2403e67 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
library_name: transformers
tags: []
---
# HinDiffusionLM: Diffusion Language Model for Hindi Language
Turning BERT-based model into an instruct-tuned LLADA-style Diffusion LLM on Hindi instruction data using a masked language modeling approach with diffusion-style generation. The model learns to iteratively denoise masked tokens to generate coherent responses in Hindi (trained on Kaggle GPU T4*2).
## Experiments
### Models Evaluated
| Model | Performance |
|-------|-------------|
| `google/muril-base-cased` | **Best** |
| `google/muril-large-cased` | Poor |
| `ai4bharat/indic-bert` | Moderate |
### Datasets Tested
| Dataset | Subset | Status | Notes |
|---------|--------|--------|-------|
| `ai4bharat/indic-instruct-data-v0.1` | `anudesh` | **Used** | Primary dataset for demonstration |
| `ai4bharat/indic-instruct-data-v0.1` | `lm_sys` | Skipped | Too time-intensive for training & GPU constraints|
|