---
library_name: transformers
tags: []
---
---
# phi2-2b-absa: Fine-Tuned Aspect-Based Sentiment Analysis Model

## Model Description

The **phi2-2b-absa** model is a fine-tuned aspect-based sentiment analysis (ABSA) model based on the Microsoft Phi-2 model. It has been trained on the **semeval2016-full-absa-reviews-english-translated-resampled** dataset. The model predicts sentiments towards different aspects mentioned in a given sentence.

## Fine-Tuning Details

The fine tuning can be revisited on [Google Colab](https://colab.research.google.com/drive/1n3ykETLpHQPXwPhUcOe-z9cG3ThrDkSi?usp=sharing).

### Dataset
- **Name:** semeval2016-full-absa-reviews-english-translated-resampled
- **Description:** Annotated dataset for ABSA containing sentences, aspects, sentiments, and additional contextual text. It is split into train and test sets.

### Model Architecture
- **Base Model:** Microsoft Phi-2
- **Fine-Tuned Model:** phi2-2b-absa

### Fine-Tuning Parameters
- **LoRA Attention Dimension (lora_r):** 64
- **LoRA Scaling Parameter (lora_alpha):** 16
- **LoRA Dropout Probability (lora_dropout):** 0.1

### BitsAndBytes Quantization
- **Activate 4-bit Precision:** True
- **Compute Dtype for 4-bit Models:** float16
- **Quantization Type:** nf4

### Training Parameters
- **Number of Training Epochs:** 1
- **Batch Size per GPU for Training:** 4
- **Batch Size per GPU for Evaluation:** 4
- **Gradient Accumulation Steps:** 1
- **Learning Rate:** 2e-4
- **Weight Decay:** 0.001
- **Optimizer:** PagedAdamW (32-bit)
- **Learning Rate Scheduler:** Cosine

### SFT Parameters
- **Maximum Sequence Length:** None
- **Packing:** False

## How to Use

```
from transformers import AutoTokenizer, pipeline
import torch

model = "Alpaca69B/llama-2-7b-absa-semeval-2016"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.float16,
    device="auto",
)

input_sentence = "the first thing that attracts attention is the warm reception and the smiling receptionists."
sequences = pipeline(
    f'### Human: {input_sentence} ### Assistant: aspect:',
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
)
sequences[0]['generated_text']

```

Testing can be seen on [Google Colab](https://colab.research.google.com/drive/1eKdZYYWiivyeCQDsocGBstVODMLZyT-_?usp=sharing)

## Acknowledgments

- The fine-tuning process and model development were performed by Ben Kampmann.
---