| --- |
| license: apache-2.0 |
| base_model: allenai/longformer-base-4096 |
| tags: |
| - generated_from_trainer |
| datasets: |
| - essays_su_g |
| metrics: |
| - accuracy |
| model-index: |
| - name: longformer-spans |
| results: |
| - task: |
| name: Token Classification |
| type: token-classification |
| dataset: |
| name: essays_su_g |
| type: essays_su_g |
| config: spans |
| split: train[80%:100%] |
| args: spans |
| metrics: |
| - name: Accuracy |
| type: accuracy |
| value: 0.9382309279843586 |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # longformer-spans |
|
|
| This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 0.1841 |
| - B: {'precision': 0.8358744394618834, 'recall': 0.8935762224352828, 'f1-score': 0.8637627432808155, 'support': 1043.0} |
| - I: {'precision': 0.9433073515392811, 'recall': 0.9695677233429395, 'f1-score': 0.9562572833470712, 'support': 17350.0} |
| - O: {'precision': 0.9409526006227655, 'recall': 0.8843485800997182, 'f1-score': 0.9117729228362295, 'support': 9226.0} |
| - Accuracy: 0.9382 |
| - Macro avg: {'precision': 0.9067114638746433, 'recall': 0.9158308419593135, 'f1-score': 0.9105976498213719, 'support': 27619.0} |
| - Weighted avg: {'precision': 0.9384636765600096, 'recall': 0.9382309279843586, 'f1-score': 0.9379045364930165, 'support': 27619.0} |
|
|
| ## Model description |
|
|
| More information needed |
|
|
| ## Intended uses & limitations |
|
|
| More information needed |
|
|
| ## Training and evaluation data |
|
|
| More information needed |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 2e-05 |
| - train_batch_size: 8 |
| - eval_batch_size: 8 |
| - seed: 42 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - num_epochs: 7 |
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | B | I | O | Accuracy | Macro avg | Weighted avg | |
| |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:| |
| | No log | 1.0 | 41 | 0.2970 | {'precision': 0.8171557562076749, 'recall': 0.34707574304889743, 'f1-score': 0.48721399730820997, 'support': 1043.0} | {'precision': 0.8802934137966912, 'recall': 0.9752737752161383, 'f1-score': 0.9253527288636114, 'support': 17350.0} | {'precision': 0.9304752325873774, 'recall': 0.8021894645566876, 'f1-score': 0.861583236321304, 'support': 9226.0} | 0.8937 | {'precision': 0.8759748008639145, 'recall': 0.7081796609405745, 'f1-score': 0.7580499874977084, 'support': 27619.0} | {'precision': 0.8946720981551954, 'recall': 0.893732575400992, 'f1-score': 0.8875050140583102, 'support': 27619.0} | |
| | No log | 2.0 | 82 | 0.2228 | {'precision': 0.7610474631751227, 'recall': 0.8916586768935763, 'f1-score': 0.8211920529801324, 'support': 1043.0} | {'precision': 0.9182955222264335, 'recall': 0.9775216138328531, 'f1-score': 0.946983444540607, 'support': 17350.0} | {'precision': 0.9614026236125126, 'recall': 0.8261435074788641, 'f1-score': 0.8886557071237029, 'support': 9226.0} | 0.9237 | {'precision': 0.8802485363380229, 'recall': 0.898441266068431, 'f1-score': 0.8856104015481474, 'support': 27619.0} | {'precision': 0.9267569578974372, 'recall': 0.9237119374343749, 'f1-score': 0.9227489636830115, 'support': 27619.0} | |
| | No log | 3.0 | 123 | 0.1807 | {'precision': 0.845437616387337, 'recall': 0.8705656759348035, 'f1-score': 0.8578176665092113, 'support': 1043.0} | {'precision': 0.9587634878973461, 'recall': 0.9474351585014409, 'f1-score': 0.9530656616901, 'support': 17350.0} | {'precision': 0.9035106382978724, 'recall': 0.9205506178192066, 'f1-score': 0.9119510361859765, 'support': 9226.0} | 0.9356 | {'precision': 0.9025705808608517, 'recall': 0.9128504840851503, 'f1-score': 0.907611454795096, 'support': 27619.0} | {'precision': 0.9360269053132667, 'recall': 0.9355516130200224, 'f1-score': 0.9357345782375959, 'support': 27619.0} | |
| | No log | 4.0 | 164 | 0.2177 | {'precision': 0.8223028105167725, 'recall': 0.8696069031639502, 'f1-score': 0.8452935694315005, 'support': 1043.0} | {'precision': 0.9182645433864154, 'recall': 0.9771181556195966, 'f1-score': 0.9467776164414164, 'support': 17350.0} | {'precision': 0.9526943133846536, 'recall': 0.8316713635378279, 'f1-score': 0.8880787037037038, 'support': 9226.0} | 0.9245 | {'precision': 0.8977538890959472, 'recall': 0.8927988074404581, 'f1-score': 0.8933832965255403, 'support': 27619.0} | {'precision': 0.9261417645247878, 'recall': 0.9244722835729027, 'f1-score': 0.9233370852871574, 'support': 27619.0} | |
| | No log | 5.0 | 205 | 0.1864 | {'precision': 0.8298059964726632, 'recall': 0.9022051773729626, 'f1-score': 0.8644924207625172, 'support': 1043.0} | {'precision': 0.9426901899089786, 'recall': 0.9670317002881844, 'f1-score': 0.9547058154091271, 'support': 17350.0} | {'precision': 0.9384137216530448, 'recall': 0.8835898547582918, 'f1-score': 0.9101769664489477, 'support': 9226.0} | 0.9367 | {'precision': 0.9036366360115622, 'recall': 0.9176089108064795, 'f1-score': 0.909791734206864, 'support': 27619.0} | {'precision': 0.9369987126692769, 'recall': 0.9367102357073029, 'f1-score': 0.9364243522452534, 'support': 27619.0} | |
| | No log | 6.0 | 246 | 0.1768 | {'precision': 0.8413417951042611, 'recall': 0.8897411313518696, 'f1-score': 0.8648648648648648, 'support': 1043.0} | {'precision': 0.9434724091520862, 'recall': 0.9696829971181556, 'f1-score': 0.9563981581490535, 'support': 17350.0} | {'precision': 0.9409258406264395, 'recall': 0.885649252113592, 'f1-score': 0.9124511446119487, 'support': 9226.0} | 0.9386 | {'precision': 0.908580014960929, 'recall': 0.9150244601945391, 'f1-score': 0.9112380558752889, 'support': 27619.0} | {'precision': 0.9387648936131638, 'recall': 0.9385929975741337, 'f1-score': 0.938261209968861, 'support': 27619.0} | |
| | No log | 7.0 | 287 | 0.1841 | {'precision': 0.8358744394618834, 'recall': 0.8935762224352828, 'f1-score': 0.8637627432808155, 'support': 1043.0} | {'precision': 0.9433073515392811, 'recall': 0.9695677233429395, 'f1-score': 0.9562572833470712, 'support': 17350.0} | {'precision': 0.9409526006227655, 'recall': 0.8843485800997182, 'f1-score': 0.9117729228362295, 'support': 9226.0} | 0.9382 | {'precision': 0.9067114638746433, 'recall': 0.9158308419593135, 'f1-score': 0.9105976498213719, 'support': 27619.0} | {'precision': 0.9384636765600096, 'recall': 0.9382309279843586, 'f1-score': 0.9379045364930165, 'support': 27619.0} | |
|
|
|
|
| ### Framework versions |
|
|
| - Transformers 4.37.2 |
| - Pytorch 2.2.0+cu121 |
| - Datasets 2.17.0 |
| - Tokenizers 0.15.2 |
|
|
