karimraouf/mbart-arabic-summarizer

mbart_summarizer_xsum

This model is a fine-tuned version of facebook/mbart-large-50 on asas-ai/Arabic-article-summarization dataset.

Model description

• Base Model: facebook/mbart-large-50
• Task: Abstractive summarization
• Dataset: Arabic-article-summarization
• Evaluation Metric: ROUGE (1/2/L/Lsum)
• Mixed Precision: Yes (fp16)

Intended uses & limitations

Only Fine Tuned for 3 epochs , and the data is not in very good quality, better to take and fine tune more on better data quality depending on the use case.

Training and evaluation data

Evaluation Metric: ROUGE (1/2/L/Lsum) trained and evaluated on the [asas-ai/Arabic-article-summarization] dataset which i splitted into training and testing splts

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

After 3 Epochs of Training:

• ROUGE1: 41.2430
• ROUGE2: 24.7636
• ROUGEL: 40.6967
• ROUGELSUM: 40.9913

Usage

First Approach

```python
from transformers import MBartForConditionalGeneration, MBartTokenizer

# Load model and tokenizer from Hugging Face Hub
model_name = "karimraouf/mbart-arabic-summarizer"
tokenizer = MBartTokenizer.from_pretrained(model_name)
model = MBartForConditionalGeneration.from_pretrained(model_name)

# Input text (replace with your own)
input_text = "أي نص تريده"

# Tokenize the input
inputs = tokenizer(
    input_text,
    return_tensors="pt",
    max_length=1024,
    truncation=True,
    padding=True
)

# Generate the summary
summary_ids = model.generate(
    **inputs,
    max_length=200,
    num_beams=3,
    early_stopping=True
)

# Decode and print the summary
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print("Summary:", summary)

Second Approach

```python
from transformers import pipeline

# Load the summarization pipeline with the pre-trained Arabic MBart model
summarizer = pipeline(
    "summarization",
    model="karimraouf/mbart-arabic-summarizer",
    tokenizer="karimraouf/mbart-arabic-summarizer"
)

# Replace this with your own Arabic text
input_text = "أي نص تريده"

# Generate the summary
summary = summarizer(
    input_text,
    max_length=200,
    min_length=30,
    do_sample=False
)

# Print the result
print(summary[0]['summary_text'])

Framework versions

• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1