File size: 1,069 Bytes
0f74f3e ccc639c 0f74f3e 12931f9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # SentiWSP
## For paper: Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis
We propose **SentiWSP**, a novel **Senti**ment-aware pre-trained language model with combined **W**ord-level and **S**entence-level **P**re-training tasks.
The word level pre-training task detects replaced sentiment words, via a generator-discriminator framework, to enhance the PLM's knowledge about sentiment words.
The sentence level pre-training task further strengthens the discriminator via a contrastive learning framework, with similar sentences as negative samples, to encode sentiments in a sentence.
## Fine-tunning
You can also load our model in huggingface ([https://huggingface.co/shuaifan/SentiWSP-base](https://huggingface.co/shuaifan/SentiWSP-base)) to fine-tunning in sentiment analysis tasks:
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("shuaifan/SentiWSP-base")
model = AutoModelForSequenceClassification.from_pretrained("shuaifan/SentiWSP-base")
``` |