| --- |
| license: mit |
| library_name: transformers |
| pipeline_tag: text-classification |
| tags: |
| - transformers |
| - pytorch |
| - text-classification |
| - news-classification |
| - ag-news |
| - nlp |
| --- |
| |
| # News Classification using RoBERTa |
|
|
| ## Model Description |
|
|
| This model is a fine-tuned RoBERTa model for multiclass news classification. It predicts the category of a news headline or article among four classes: |
|
|
| * World |
| * Sports |
| * Business |
| * Sci/Tech |
|
|
| The model was fine-tuned on the AG News dataset using Hugging Face Transformers and PyTorch. |
|
|
| ### Model Details |
|
|
| * **Developed by:** MhoOmm |
| * **Model Type:** RoBERTa for Sequence Classification |
| * **Language:** English |
| * **License:** MIT |
| * **Finetuned From:** roberta-base |
|
|
| ## Labels |
|
|
| | Label ID | Category | |
| | -------- | -------- | |
| | 0 | World | |
| | 1 | Sports | |
| | 2 | Business | |
| | 3 | Sci/Tech | |
|
|
| ## Intended Use |
|
|
| ### Direct Use |
|
|
| This model can be used for: |
|
|
| * News article categorization |
| * Content organization systems |
| * News recommendation pipelines |
| * NLP learning projects |
| * Text classification demonstrations |
|
|
| ### Out-of-Scope Use |
|
|
| This model is not intended for: |
|
|
| * Medical diagnosis |
| * Financial advice |
| * Legal decision making |
| * Safety-critical systems |
| * Classification of non-English news |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer |
| from transformers import AutoModelForSequenceClassification |
| import torch |
| |
| model_name = "MhoOmm/news-classifier" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForSequenceClassification.from_pretrained(model_name) |
| |
| text = "Apple launches a new AI-powered processor" |
| |
| inputs = tokenizer( |
| text, |
| return_tensors="pt", |
| truncation=True |
| ) |
| |
| with torch.no_grad(): |
| outputs = model(**inputs) |
| |
| prediction = outputs.logits.argmax(-1).item() |
| |
| print(model.config.id2label[prediction]) |
| ``` |
|
|
| ## Training Data |
|
|
| The model was fine-tuned on the AG News dataset. |
|
|
| ### Dataset Statistics |
|
|
| * Training Samples: 120,000 |
| * Test Samples: 7,600 |
| * Classes: 4 |
|
|
| ### Categories |
|
|
| * World |
| * Sports |
| * Business |
| * Sci/Tech |
|
|
| ## Training Procedure |
|
|
| ### Preprocessing |
|
|
| * RoBERTa tokenizer |
| * Maximum sequence length: 128 |
| * Truncation enabled |
| * Dynamic padding |
|
|
| ### Hyperparameters |
|
|
| * Learning Rate: 2e-5 |
| * Batch Size: 16 |
| * Epochs: 3 |
| * Optimizer: AdamW |
| * Framework: PyTorch |
| * Trainer: Hugging Face Trainer |
|
|
| ## Evaluation |
|
|
| ### Metrics |
|
|
| The model was evaluated using classification accuracy. |
|
|
| ### Example Predictions |
|
|
| | Input | Prediction | |
| | --------------------------------------------- | ---------- | |
| | India defeats Australia in the cricket final | Sports | |
| | Stock market reaches a new all-time high | Business | |
| | Apple unveils a new AI-powered device | Sci/Tech | |
| | Global leaders meet to discuss climate policy | World | |
|
|
| ### Results |
|
|
| * Accuracy: **94.76%** |
|
|
| ## Limitations |
|
|
| * Trained only on English-language news articles. |
| * May perform poorly on informal text such as tweets and social media posts. |
| * Restricted to the four AG News categories. |
| * Performance may decrease on domains significantly different from the training data. |
|
|
| ## Architecture |
|
|
| * Base Model: RoBERTa Base |
| * Hidden Size: 768 |
| * Transformer Layers: 12 |
| * Attention Heads: 12 |
| * Classification Head: Linear Layer (768 → 4) |
|
|
| ## Software |
|
|
| * PyTorch |
| * Transformers |
| * Hugging Face Hub |
| * Gradio |
|
|
| ## Author |
|
|
| Created by **MhoOmm** as an end-to-end NLP project demonstrating: |
|
|
| * Transformer fine-tuning |
| * Text classification |
| * Model deployment with Hugging Face |
| * Interactive inference using Gradio |
|
|
| ## Citation |
|
|
| If you use this model, please cite the original RoBERTa paper: |
|
|
| **RoBERTa: A Robustly Optimized BERT Pretraining Approach** |
|
|
| Yinhan Liu, Myle Ott, Naman Goyal, et al. |
|
|
| https://arxiv.org/abs/1907.11692 |
|
|
