File size: 1,942 Bytes
a966321 0a25127 a966321 0a25127 a966321 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
library_name: transformers
datasets:
- cardiffnlp/tweet_eval
language:
- en
metrics:
- f1
base_model:
- OuteAI/Lite-Oute-1-300M-Instruct
pipeline_tag: text-classification
---
Модель `OuteAI/Lite-Oute-1-300M-Instruct` дообученная на датасете `cardiffnlp/tweet_eval`, задача классификации сентимента твита, вывести одно из трех слов -
`negative`, `neutral`, `positive`.
## Дообучение
Модель дообучалась при помощи DoRA.
- Ранг LoRA = `16`
- `alpha=32`
- DoRA применялась только к весам Key, Value в attention
- `BATCH_SIZE = 16`
- `LEARNING_RATE = 2e-4`
- `NUM_EPOCHS = 2`
- `AdamW`
- `weight_decay=0.01`
## Метрика на валидации
F1=0.51
**Tweet:** "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin" \
**Label:** positive\
**Output:** \
positive \
positive \
positive \
pos
**Tweet:** "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"\
**Label:** neutral\
**Output:** \
neutral \
neutral \
neutral \
neut
**Tweet:** Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.\
**Label:** neutral\
**Output:** \
neutral \
positive \
positive \
pos
**Tweet:** Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays\
**Label:** neutral\
**Output:** \
neutral \
neutral \
neutral \
neut
**Tweet:** @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"\
**Label:** positive\
**Output:** \
neutral \
negative \
negative \
negative
|