README.md

---
library_name: transformers
datasets:
- cardiffnlp/tweet_eval
language:
- en
metrics:
- f1
base_model:
- OuteAI/Lite-Oute-1-300M-Instruct
pipeline_tag: text-classification
---

Модель `OuteAI/Lite-Oute-1-300M-Instruct` дообученная на датасете `cardiffnlp/tweet_eval`, задача классификации сентимента твита, вывести одно из трех слов - 
`negative`, `neutral`, `positive`. 

## Дообучение

Модель дообучалась при помощи DoRA.

- Ранг LoRA = `16`
- `alpha=32`
- DoRA применялась только к весам Key, Value в attention
- `BATCH_SIZE = 16`
- `LEARNING_RATE = 2e-4`
- `NUM_EPOCHS = 2`
- `AdamW`
-  `weight_decay=0.01`

## Метрика на валидации

F1=0.51


![image/png](https://cdn-uploads.huggingface.co/production/uploads/67b331dfe2883deef7c92e6f/4RaY2SmZF6HnjlHiTlXfs.png)


**Tweet:** "QT @user In the original draft of the 7th book, Remus Lupin survived the Battle of Hogwarts. #HappyBirthdayRemusLupin" \
**Label:** positive\
**Output:** \
positive \
positive \
positive \
pos

**Tweet:** "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"\
**Label:** neutral\
**Output:** \
neutral \
neutral \
neutral \
neut

**Tweet:** Sorry bout the stream last night I crashed out but will be on tonight for sure. Then back to Minecraft in pc tomorrow night.\
**Label:** neutral\
**Output:** \
neutral \
positive \
positive \
pos

**Tweet:** Chase Headley's RBI double in the 8th inning off David Price snapped a Yankees streak of 33 consecutive scoreless innings against Blue Jays\
**Label:** neutral\
**Output:** \
neutral \
neutral \
neutral \
neut

**Tweet:** @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"\
**Label:** positive\
**Output:** \
neutral \
negative \
negative \
negative