| # SentiWSP | |
| ## For paper: Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis | |
| We propose **SentiWSP**, a novel **Senti**ment-aware pre-trained language model with combined **W**ord-level and **S**entence-level **P**re-training tasks. | |
| The word level pre-training task detects replaced sentiment words, via a generator-discriminator framework, to enhance the PLM's knowledge about sentiment words. | |
| The sentence level pre-training task further strengthens the discriminator via a contrastive learning framework, with similar sentences as negative samples, to encode sentiments in a sentence. | |
| ## Fine-tunning | |
| You can also load our model in huggingface ([https://huggingface.co/shuaifan/SentiWSP-base](https://huggingface.co/shuaifan/SentiWSP-base)) to fine-tunning in sentiment analysis tasks: | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification | |
| import torch | |
| tokenizer = AutoTokenizer.from_pretrained("shuaifan/SentiWSP-base") | |
| model = AutoModelForSequenceClassification.from_pretrained("shuaifan/SentiWSP-base") | |
| ``` |