--- library_name: transformers license: gemma base_model: unsloth/gemma-4-E2B-it pipeline_tag: image-text-to-text language: - bn tags: - headline-generation - bangla - bengali - news - vlm - lora - gemma4 --- # BartaLens-E2B

Paper Coming Soon

[![Paper](https://img.shields.io/badge/arXiv-coming--soon-red)](https://arxiv.org/abs/0000.00000) [![Dataset](https://img.shields.io/badge/HuggingFace-Dataset-yellow)](https://huggingface.co/datasets/dipta007/shironam-pro-max) [![Models](https://img.shields.io/badge/HuggingFace-Models-orange)](https://huggingface.co/dipta007/BartaLens-E2B) **BartaLens-E2B** is a Bangla multimodal headline generation model fine-tuned from `gemma-4-E2B-it`. Given a Bengali news article (and optionally an accompanying image), it generates a concise, accurate newspaper-style headline. Trained with **LoRA** on the `dipta007/shironam-pro-max` dataset with 50% image supervision — the model is robust to both image-present and text-only inputs. ## Highlights - **ROUGE-1: 0.3851 | ROUGE-L: 0.3551 | BLEU-4: 10.43 | BERTScore: 0.8969** on own test split (with images) - **ROUGE-1: 0.3840 | ROUGE-L: 0.3553 | BLEU-4: 11.02 | BERTScore: 0.8970** on Shironam test split (text-only) - Outperforms zero-shot Gemma4-E2B, Qwen3.5-4B, Ministral-3 3B, and prior Bangla headline systems - Robust to missing images: trained with 50% text-only supervision so performance doesn't degrade without images ## Model Overview | Property | Value | |----------|-------| | **Model Type** | Vision-Language Model (Causal LM + Vision Encoder) | | **Base Model** | unsloth/gemma-4-E2B-it | | **Training** | SFT + LoRA (r=32, alpha=32) | | **LoRA Targets** | all-linear (vision + language + attention + MLP) | | **Max Sequence Length** | 4,096 tokens | | **Language** | Bengali (বাংলা) | | **Image Supervision** | 50% (model sees image for half of training samples) | | **Effective Batch Size** | 64 (8 per device x 8 gradient accumulation) | ## Quickstart ```python from unsloth import FastVisionModel, get_chat_template import torch from PIL import Image model_name = "dipta007/BartaLens-E2B" model, processor = FastVisionModel.from_pretrained( model_name, max_seq_length=4096, load_in_4bit=False, dtype=torch.bfloat16, ) processor = get_chat_template(processor, "gemma-4") FastVisionModel.for_inference(model) INSTRUCTION = ( "আপনি একজন অভিজ্ঞ বাংলা সংবাদ সম্পাদক। নিচের সংবাদ নিবন্ধটির জন্য একটি উপযুক্ত শিরোনাম তৈরি করুন।\n" "\n" "নিয়মাবলী:\n" "- নিবন্ধের মূল ঘটনা ও তথ্য সঠিকভাবে প্রকাশ করুন; কাল্পনিক তথ্য যোগ করবেন না।\n" "- সংবাদপত্রের সাধারণ শিরোনামের শৈলীতে, সংক্ষিপ্ত ও আকর্ষণীয়ভাবে লিখুন।\n" "- উদ্ধৃতি চিহ্ন, মার্কডাউন, ইমোজি, তালিকা চিহ্ন (*, -), অথবা নম্বর (১., 1.) ব্যবহার করবেন না।\n" "- কোনো ভূমিকা, ব্যাখ্যা, একাধিক বিকল্প বা অতিরিক্ত মন্তব্য যোগ করবেন না।\n" "- শুধু শিরোনামটি একটি লাইনে লিখুন, অন্য কিছু নয়।\n" ) def generate_headline(article: str, image: Image.Image | None = None): """Generate a Bengali headline for a news article.""" user_text = f"{INSTRUCTION}\nনিবন্ধ:\n{article}\n\nশিরোনাম:" if image is None: image = Image.new("RGB", (224, 224), color="black") messages = [ { "role": "user", "content": [ {"type": "image"}, {"type": "text", "text": user_text}, ], } ] input_text = processor.apply_chat_template( messages, add_generation_prompt=True ) inputs = processor( images=[[image]], text=[input_text], add_special_tokens=False, return_tensors="pt", ).to("cuda") with torch.no_grad(): out = model.generate( **inputs, max_new_tokens=64, use_cache=True, do_sample=False, ) headline = processor.tokenizer.decode( out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True, ).strip() return headline # Usage: text-only article = """বাংলাদেশ জাতীয় ক্রিকেট দলের অধিনায়ক নাজমুল হোসেন শান্ত আজ সংবাদ সম্মেলনে জানিয়েছেন, দল আগামী টেস্ট সিরিজের জন্য পুরোপুরি প্রস্তুত। তিনি বলেন, তরুণ খেলোয়াড়দের পারফরম্যান্স দলের শক্তি বাড়িয়েছে।""" headline = generate_headline(article) print(headline) # Usage: with image # image = Image.open("news_image.jpg") # headline = generate_headline(article, image=image) ``` ## Performance ### Comparison with zero-shot baselines (Own eval, n=15,000) | Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-4 | BERTScore | METEOR | |-------|---------|---------|---------|--------|-----------|--------| | Qwen3.5-0.8B | 0.1546 | 0.0541 | 0.1429 | 2.14 | 0.8373 | 0.1002 | | Qwen3.5-2B | 0.2029 | 0.0613 | 0.1821 | 1.83 | 0.8498 | 0.1191 | | Ministral-3 3B | 0.2892 | 0.0903 | 0.2445 | 2.48 | 0.8725 | 0.1625 | | Qwen3.5-4B | 0.2984 | 0.1065 | 0.2618 | 3.98 | 0.8729 | 0.1924 | | Gemma4-E2B (zero-shot) | 0.3484 | 0.1507 | 0.3127 | 6.37 | 0.8874 | 0.2531 | | **BartaLens-E2B (ours)** | **0.3851** | **0.1807** | **0.3551** | **10.43** | **0.8969** | **0.2644** | ### Cross-dataset generalization (Shironam eval, text-only, n=15,012) | Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-4 | BERTScore | METEOR | |-------|---------|---------|---------|--------|-----------|--------| | Gemma4-E2B (zero-shot) | 0.3535 | 0.1526 | 0.3190 | 6.36 | 0.8883 | 0.2620 | | **BartaLens-E2B (ours)** | **0.3840** | **0.1809** | **0.3553** | **11.02** | **0.8970** | **0.2590** | ## Training Details - **Dataset**: [dipta007/shironam-pro-max](https://huggingface.co/datasets/dipta007/shironam-pro-max) (train split) - **Supervision**: 50% of training samples include the news image; 50% use a black placeholder (text-only). This trains the model to be robust when no image is available. - **Early stopping**: patience=3 on validation loss, eval every 200 steps - **Optimizer**: AdamW, LR=5e-4, cosine schedule, warmup 5% - **Hardware**: NVIDIA L40S (46 GB) - **Metrics**: csebuetnlp multilingual ROUGE (Bengali stemmer), HF BLEU, BanglaBERT BERTScore, METEOR ## Intended Use - **In-scope**: generating concise Bengali news headlines from article text (optionally with an image), headline suggestion for editors, summarization benchmarks. - **Out-of-scope**: generating headlines in other languages, creative/clickbait headline generation, summarization of non-news content. ## Citation ```bibtex ``` ## License Released under the Gemma License.