ddecosmo commited on
Commit
bfd9203
·
verified ·
1 Parent(s): 3a61b2b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +93 -49
README.md CHANGED
@@ -1,71 +1,115 @@
1
  ---
2
- library_name: transformers
3
  license: apache-2.0
4
- base_model: distilbert-base-uncased
5
- tags:
6
- - generated_from_trainer
 
7
  metrics:
8
  - accuracy
9
- - f1
10
- - precision
11
- - recall
12
- model-index:
13
- - name: finetuned_model
14
- results: []
15
  ---
16
 
17
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
18
- should probably proofread and complete it, then remove this comment. -->
19
 
20
- # finetuned_model
21
 
22
- This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the james-kramer/football_news dataset.
23
- It achieves the following results on the evaluation set:
24
- - Loss: 0.0000
25
- - Accuracy: 1.0
26
- - F1: 1.0
27
- - Precision: 1.0
28
- - Recall: 1.0
29
 
30
- ## Model description
31
 
32
- More information needed
33
 
34
- ## Intended uses & limitations
35
 
36
- More information needed
 
 
 
 
37
 
38
- ## Training and evaluation data
39
 
40
- More information needed
41
 
42
- ## Training procedure
43
 
44
- ### Training hyperparameters
45
 
46
- The following hyperparameters were used during training:
47
- - learning_rate: 2e-05
48
- - train_batch_size: 8
49
- - eval_batch_size: 8
50
- - seed: 42
51
- - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
52
- - lr_scheduler_type: linear
53
- - num_epochs: 5
54
 
55
- ### Training results
56
 
57
- | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
58
- |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
59
- | 0.0002 | 1.0 | 80 | 0.0710 | 0.9875 | 0.9875 | 0.9878 | 0.9875 |
60
- | 0.0001 | 2.0 | 160 | 0.0001 | 1.0 | 1.0 | 1.0 | 1.0 |
61
- | 0.0001 | 3.0 | 240 | 0.0001 | 1.0 | 1.0 | 1.0 | 1.0 |
62
- | 0.0001 | 4.0 | 320 | 0.0000 | 1.0 | 1.0 | 1.0 | 1.0 |
63
- | 0.0001 | 5.0 | 400 | 0.0000 | 1.0 | 1.0 | 1.0 | 1.0 |
64
 
 
65
 
66
- ### Framework versions
67
 
68
- - Transformers 4.56.1
69
- - Pytorch 2.8.0+cu126
70
- - Datasets 4.0.0
71
- - Tokenizers 0.22.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ '[object Object]': null
3
  license: apache-2.0
4
+ datasets:
5
+ - maryzhang/hw1-24679-image-dataset
6
+ language:
7
+ - en
8
  metrics:
9
  - accuracy
 
 
 
 
 
 
10
  ---
11
 
12
+ # Model Card for {{ model_id | default("Model ID", true) }}
 
13
 
14
+ <!-- Provide a quick summary of what the model is/does. -->
15
 
16
+ This is finetuned version of DistilBERT that is used for sentiment analysis on NFL news titles.
 
 
 
 
 
 
17
 
18
+ ## Model Details
19
 
20
+ ### Model Description
21
 
22
+ This model uses the DistilBERT model to classify NFL news article titles as positive or negative.
23
 
24
+ - **Developed by:** Devin DeCosmo
25
+ - **Model type:** Binary Sentiment Analysis
26
+ - **Language(s) (NLP):** English
27
+ - **License:** MIT
28
+ - **Finetuned from model:** DistilBERT
29
 
30
+ ## Uses
31
 
32
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
33
 
34
+ This is used for sentiment analysis of NFL articles, but could possibly be used for other article titles.
35
 
36
+ ### Direct Use
37
 
38
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
 
 
 
 
 
 
39
 
40
+ The direct use is to classify NFL articles as positive or negative.
41
 
42
+ ### Out-of-Scope Use
 
 
 
 
 
 
43
 
44
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
45
 
46
+ If the dataset was expanded, this could be used for sentiment analysis on other types of articles or find other features like bias towards a team or player.
47
 
48
+ ## Bias, Risks, and Limitations
49
+
50
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
51
+
52
+ This is trained off a small dataset of 100 titles, this small dataset could be liable to overfitting and is not robust.
53
+
54
+ ### Recommendations
55
+
56
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
57
+
58
+ The small dataset size means this model is not highly generalizable.
59
+
60
+ ## How to Get Started with the Model
61
+
62
+ Use the code below to get started with the model.
63
+
64
+ ## Training Details
65
+
66
+ ### Training Data
67
+
68
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
69
+
70
+ James-kramer/football_news
71
+
72
+ This is the training dataset used.
73
+ It consists of 100 original titles used for validation along with 1000 synthetic pieces of data from training.
74
+
75
+ ### Training Procedure
76
+
77
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
78
+ This model was trained with DistilBERT using binary classification, a training split of 80%, and 5 epochs.
79
+ I initially used more but this converged extremely quickly.
80
+
81
+
82
+ ## Evaluation
83
+
84
+ <!-- This section describes the evaluation protocols and provides the results. -->
85
+
86
+ ### Testing Data, Factors & Metrics
87
+
88
+ #### Testing Data
89
+
90
+ <!-- This should link to a Dataset Card if possible. -->
91
+ James-kramer/football_news
92
+ The testing data was the 'original' split, the 100 original titles in this set.
93
+
94
+ #### Factors
95
+
96
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
97
+
98
+ This dataset is evaluating whether the food is positive, "1", or negative, "0".
99
+
100
+ #### Metrics
101
+
102
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
103
+
104
+ The testing metric used was accuracy to ensure the highest accuracy of the model possible.
105
+ I also considered testing time. This small langauge model ran extremely quickly with 102 steps per second.
106
+
107
+ ### Results
108
+
109
+ After training with the initial dataset, this model reached an accuracy of 100% in validation.
110
+ This is likely due to the simplicity of the task, binary classification, along with distilBERT being made for tasks such as this.
111
+
112
+ #### Summary
113
+
114
+ This model reached a high accuracy with our current model, but this perfomance can not be confirmed to continue as the dataset was very small.
115
+ Additional testing with more samples would be highly beneficial.