KobitaLM - Bengali Poetry Language Model

KobitaLM (কবিতা LM, meaning "Poetry LM" in Bengali) is a fine-tuned language model specialized in generating Bengali poetry. Built on top of Google's Gemma 2 9B model using LoRA adapters, it captures the distinctive styles of classical and modern Bengali poets.

Model Details

Model Description

KobitaLM is a LoRA adapter fine-tuned on a comprehensive corpus of Bengali poetry from renowned poets including Rabindranath Tagore, Jibanananda Das, Kazi Nazrul Islam, and many others. The model can generate poetry in various styles and follows instructions in Bengali.

• Developed by: Community Project
• Model type: Causal Language Model with LoRA Adapters
• Language: Bengali (বাংলা)
• License: Gemma License
• Base Model: unsloth/gemma-2-9b-bnb-4bit
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Primary Use: Bengali poetry generation

Model Sources

• Base Model: Gemma 2 9B
• Training Framework: Unsloth
• Dataset: Bengali Poems Collection (Classical and Modern)

Uses

Direct Use

KobitaLM is designed for:

• Generating Bengali poetry in various classical and modern styles
• Educational purposes to learn about Bengali poetic traditions
• Creative writing assistance for Bengali content creators
• Research in Bengali NLP and computational creativity

Example Usage

from unsloth import FastLanguageModel

# Load the model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="your-username/KobitaLM",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Prepare for inference
FastLanguageModel.for_inference(model)

# Generate poetry
prompt = "জীবনানন্দ দাশের স্টাইল-এ একটি কবিতা লিখুন।"
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.9,
    repetition_penalty=1.2,
)

poem = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(poem)

Out-of-Scope Use

This model is NOT suitable for:

• General Bengali text generation outside of poetry
• Translation tasks
• Factual question answering
• Any use requiring factual accuracy or real-time information

Bias, Risks, and Limitations

• The model may reflect biases present in classical Bengali poetry
• Generated content may occasionally contain repetitive patterns
• Performance is optimized for poetry generation, not general text
• Requires Bengali language understanding to evaluate output quality
• May not accurately represent all regional Bengali dialects

Recommendations

• Review generated poetry for appropriateness before publication
• Use as a creative assistant rather than autonomous content generator
• Consider cultural context when using generated poetry
• Respect copyright and attribution norms for the training data poets

Training Details

Training Data

The model was fine-tuned on a curated corpus of Bengali poetry including works from:

• রবীন্দ্রনাথ ঠাকুর (Rabindranath Tagore)
• জীবনানন্দ দাশ (Jibanananda Das)
• কাজী নজরুল ইসলাম (Kazi Nazrul Islam)
• সুকান্ত ভট্টাচার্য (Sukanta Bhattacharya)
• কামিনী রায় (Kamini Roy)
• মাইকেল মধুসূদন দত্ত (Michael Madhusudan Dutt)
• লালন ফকির (Lalon Fakir)
• রামপ্রসাদ সেন (Ramprasad Sen)
• And many other classical and modern Bengali poets

Additionally, Bengali instruction-following data was included for better controllability.

Training Procedure

• Fine-tuning Framework: Unsloth + PEFT
• Training Method: Supervised Fine-Tuning (SFT) with LoRA
• LoRA Rank: 8
• LoRA Alpha: 8
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Quantization: 4-bit (bitsandbytes)
• Max Sequence Length: 2048 tokens
• Base Model: unsloth/gemma-2-9b-bnb-4bit

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Framework versions

• PEFT 0.18.0