| # T5-Small Transformer Model for News Text Summarization | |
| This repository hosts a fine-tuned version of the T5-small Transformer model for abstractive text summarization. Trained on the CNN-DailyMail News dataset, this model generates concise and meaningful summaries from long-form news articles. It is well-suited for applications like news digest creation, content summarization engines, and information extraction systems. | |
| ## Model Details | |
| - **Model Architecture:** T5-small Transformer | |
| - **Task:** Abstractive Text Summarization | |
| - **Dataset:** CNN-DailyMail News Text Summarization Dataset | |
| - **Fine-tuning Framework:** Hugging Face Transformers | |
| ## Usage | |
| ### Installation | |
| ```sh | |
| pip install transformers torch | |
| ``` | |
| ### Loading the Model | |
| ```python | |
| from transformers import T5ForConditionalGeneration, T5Tokenizer | |
| import torch | |
| # Load model and tokenizer | |
| model_name = "AventIQ-AI/t5-small-news-text-summarization" | |
| model = T5ForConditionalGeneration.from_pretrained(model_name) | |
| tokenizer = T5Tokenizer.from_pretrained("t5-small") | |
| # Set model to evaluation mode | |
| model.eval() | |
| # Example input | |
| article_text = """ | |
| NASA’s Perseverance rover has successfully collected samples from Mars that may contain signs of ancient microbial life. | |
| These samples will eventually be returned to Earth as part of an ambitious mission involving NASA and the European Space Agency. | |
| """ | |
| # Preprocess input | |
| input_text = "summarize: " + article_text.strip().replace("\n", " ") | |
| inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True) | |
| # Generate summary | |
| with torch.no_grad(): | |
| summary_ids = model.generate( | |
| inputs["input_ids"], | |
| num_beams=4, | |
| length_penalty=2.0, | |
| max_length=150, | |
| early_stopping=True | |
| ) | |
| # Decode and print summary | |
| summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True) | |
| print(f"Summary:\n{summary}") | |
| ``` | |
| ## Performance Metrics | |
| - **ROUGE-L Score:** 0.35 (on CNN-DailyMail validation set) | |
| - **BLEU Score:** 0.27 | |
| ## Fine-Tuning Details | |
| ### Dataset | |
| The model was fine-tuned on the [CNN-DailyMail News dataset](https://huggingface.co/datasets/cnn_dailymail), which contains pairs of news articles and human-written summaries. | |
| ### Training | |
| - Number of epochs: 4 | |
| - Batch size: 16 | |
| - Evaluation strategy: epoch | |
| - Learning rate: 3e-4 | |
| - Optimizer: AdamW | |
| ## Repository Structure | |
| ``` | |
| . | |
| ├── model/ # Fine-tuned model files | |
| ├── tokenizer_config/ # Tokenizer configuration and vocab files | |
| ├── model.safensors/ # Model checkpoint (optional) | |
| ├── README.md # Model documentation | |
| ``` | |
| ## Limitations | |
| - The model may struggle with extremely technical or domain-specific texts outside the news genre. | |
| - Summaries may occasionally lose factual accuracy in favor of fluency and brevity. | |
| ## Contributing | |
| Contributions are welcome! Feel free to open an issue or submit a pull request with suggestions, improvements, or bug fixes. | |