Update README.md

README.md

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # DSPFirst-Finetuning-5
 
-This model is a fine-tuned version of [ahotrod/electra_large_discriminator_squad2_512](https://huggingface.co/ahotrod/electra_large_discriminator_squad2_512) on an unknown dataset.
+This model is a fine-tuned version of [ahotrod/electra_large_discriminator_squad2_512](https://huggingface.co/ahotrod/electra_large_discriminator_squad2_512) on a generated Questions and Answers dataset from the DSPFirst textbook based on the SQuAD 2.0 format.<br />
 It achieves the following results on the evaluation set:
 - Loss: 0.9496
 - Exact: 64.0557
@@ -24,15 +24,76 @@ It achieves the following results on the evaluation set:
 
 More information needed
 
+## More accurate metrics:
+
+### Before fine-tuning:
+
+```
+ 'HasAns_exact': 53.09537088678193,
+ 'HasAns_f1': 58.61604504258551,
+ 'HasAns_total': 1793,
+ 'NoAns_exact': 86.11111111111111,
+ 'NoAns_f1': 86.11111111111111,
+ 'NoAns_total': 288,
+ 'best_exact': 57.66458433445459,
+ 'best_exact_thresh': 0.0,
+ 'best_f1': 62.42122477720136,
+ 'best_f1_thresh': 0.0,
+ 'exact': 57.66458433445459,
+ 'f1': 62.42122477720133,
+ 'total': 2081
+```
+
+### After fine-tuning:
+
+```
+ 'HasAns_exact': 64.138315672058,
+ 'HasAns_f1': 71.25733612355444,
+ 'HasAns_total': 1793,
+ 'NoAns_exact': 63.19444444444444,
+ 'NoAns_f1': 63.19444444444444,
+ 'NoAns_total': 288,
+ 'best_exact': 63.95963479096588,
+ 'best_exact_thresh': 0.0,
+ 'best_f1': 70.09341838997268,
+ 'best_f1_thresh': 0.0,
+ 'exact': 64.00768861124459,
+ 'f1': 70.14147221025135,
+ 'total': 2081
+```
+
+# Dataset
+A visualization of the dataset can be found [here](https://github.gatech.edu/pages/VIP-ITS/textbook_SQuAD_explore/explore/textbookv1.0/textbook/).<br />
+The split between train and test is 65% and 35% respectively.
+```
+DatasetDict({
+    train: Dataset({
+        features: ['id', 'title', 'context', 'question', 'answers'],
+        num_rows: 3863
+    })
+    test: Dataset({
+        features: ['id', 'title', 'context', 'question', 'answers'],
+        num_rows: 2081
+    })
+})
+```
+
 ## Intended uses & limitations
 
-More information needed
+This model is fine-tuned to answer questions from the DSPFirst textbook. I'm not really sure what I am doing so you should review before using it.<br />
+Also, you should improve the Dataset either by using a **better generated questions and answers model** (currently using https://github.com/patil-suraj/question_generation) or perform **data augmentation** to increase dataset size.
 
 ## Training and evaluation data
 
-More information needed
+- `batch_size` of 6 results in 14.82 GB VRAM
+- Utilizes `gradient_accumulation_steps` to get total batch size to 514 (batch size should be at least 256)
+- 4.52 GB RAM
+- 30% of the total questions is dedicated for evaluating.
 
 ## Training procedure
+- The model was trained from [Google Colab](https://colab.research.google.com/drive/1dJXNstk2NSenwzdtl9xA8AqjP4LL-Ks_?usp=sharing)
+- Utilizes Tesla P100 16GB, took 6.3 hours to train
+- `load_best_model_at_end` is enabled in TrainingArguments
 
 ### Training hyperparameters
 
@@ -47,6 +108,11 @@ The following hyperparameters were used during training:
 - lr_scheduler_type: linear
 - num_epochs: 10
 
+### Model hyperparameters
+
+- hidden_dropout_prob: 0.36
+- attention_probs_dropout_prob = 0.36
+
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Exact   | F1      | Combined |