README.md

---
'[object Object]': null
license: apache-2.0
datasets:
- maryzhang/hw1-24679-image-dataset
language:
- en
metrics:
- accuracy
---

# Model Card for {{ model_id | default("Model ID", true) }}

<!-- Provide a quick summary of what the model is/does. -->

This is finetuned version of DistilBERT that is used for sentiment analysis on NFL news titles. 

## Model Details

### Model Description

This model uses the DistilBERT model to classify NFL news article titles as positive or negative. 

- **Developed by:** Devin DeCosmo
- **Model type:** Binary Sentiment Analysis
- **Language(s) (NLP):** English
- **License:** MIT
- **Finetuned from model:** DistilBERT

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

This is used for sentiment analysis of NFL articles, but could possibly be used for other article titles. 

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

The direct use is to classify NFL articles as positive or negative.

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

If the dataset was expanded, this could be used for sentiment analysis on other types of articles or find other features like bias towards a team or player.

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

This is trained off a small dataset of 100 titles, this small dataset could be liable to overfitting and is not robust. 

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

The small dataset size means this model is not highly generalizable. 

## How to Get Started with the Model

Use the code below to get started with the model.

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

James-kramer/football_news

This is the training dataset used. 
It consists of 100 original titles used for validation along with 1000 synthetic pieces of data from training. 

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
This model was trained with DistilBERT using binary classification, a training split of 80%, and 5 epochs.
I initially used more but this converged extremely quickly. 


## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->
James-kramer/football_news
The testing data was the 'original' split, the 100 original titles in this set. 

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

This dataset is evaluating whether the food is positive, "1", or negative, "0".

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

The testing metric used was accuracy to ensure the highest accuracy of the model possible. 
I also considered testing time. This small langauge model ran extremely quickly with 102 steps per second. 

### Results

After training with the initial dataset, this model reached an accuracy of 100% in validation. 
This is likely due to the simplicity of the task, binary classification, along with distilBERT being made for tasks such as this.

#### Summary

This model reached a high accuracy with our current model, but this perfomance can not be confirmed to continue as the dataset was very small.
Additional testing with more samples would be highly beneficial.