File size: 3,599 Bytes
0cd3fe7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | # Text-to-Text Transfer Transformer Quantized Model for News Summarization
This repository hosts a quantized version of the T5 model, fine-tuned specifically for text summarization of news. The model extracts concise summaries from semi-structured or unstructured news texts, making it ideal for POS systems, kitchen displays, and chat-based food order logging.
## Model Details
- **Field:** Description
- **Model Architecture** T5 (Text-to-Text Transfer Transformer)
- **Task** Text Summarization for News
- **Input Format** Free-form order text (includes Order ID, Customer, Items, etc.)
- **Quantization** 8-bit (int8) using bitsandbytes
- **Framework** Hugging Face Transformers
- **Base Model** t5-base
- **Dataset** Custom
## Usage
## Installation
```sh
pip install transformers accelerate bitsandbytes torch
```
### Loading the Model
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "AventIQ-AI/T5-News-Summarization"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name, load_in_8bit=True, device_map="auto")
def test_summarization(model, tokenizer):
user_text = input("\nEnter your News text:\n")
inputs = tokenizer("summarize: " + user_text, return_tensors="pt", truncation=True, max_length=512).to(model.device)
output = model.generate(
**inputs,
max_new_tokens=100,
num_beams=5,
length_penalty=0.8,
early_stopping=True
)
summary = tokenizer.decode(output[0], skip_special_tokens=True)
return summary
print("\n📝 **Model Summary:**")
print(test_summarization(model, tokenizer))
```
## ROUGE Evaluation Results
After fine-tuning the **T5-Small** model for text summarization, we obtained the following **ROUGE** scores:
| **Metric** | **Score** | **Meaning** |
|-------------|-----------|-------------|
| **ROUGE-1** | **0.4125** (~41%) | Overlap of **unigrams** between reference and summary. |
| **ROUGE-2** | **0.2167** (~22%) | Overlap of **bigrams**, indicating fluency. |
| **ROUGE-L** | **0.3421** (~34%) | Longest common subsequence matching structure. |
| **ROUGE-Lsum** | **0.3644** (~36%) | Sentence-level summarization effectiveness. |
## Fine-Tuning Details
### Dataset
Custom-labeled food order dataset containing fields like Order ID, Customer, and Order Details. The model was trained to extract clean, natural summaries from noisy or inconsistent order formats.
### Training
- Number of epochs: 3
- Batch size: 4
- Evaluation strategy: epoch
- Learning rate: 3e-5
### Quantization
Post-training 8-bit quantization using bitsandbytes library with Hugging Face integration. This reduced the model size and improved inference speed with negligible impact on summarization quality.
## Repository Structure
```
.
├── model/ # Contains the quantized model files
├── tokenizer_config/ # Tokenizer configuration and vocabulary files
├── model.safetensors/ # Quantized model weights
├── README.md # Model documentation
```
## Limitations
- The model may misinterpret or misformat input with excessive noise or missing key fields.
- Quantized versions may show slight accuracy loss compared to full-precision models.
- Best suited for English-language food order formats.
## Contributing
Contributions are welcome! If you have suggestions, feature requests, or improvements, feel free to open an issue or submit a pull request. |