Qwen2.5-3B Fine-tuned for Sentiment Analysis

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct on the Sentiment140 dataset for sentiment classification.

Model Description

This model was trained to classify tweets as either "positive" or "negative". It was fine-tuned using the Parameter-Efficient Fine-Tuning (PEFT) technique, specifically LoRA (Low-Rank Adaptation), on a sample of 25,000 tweets. The training was performed in a Google Colab environment using an A100 GPU.

Intended Uses & Limitations

This model is intended for academic and demonstration purposes to analyze the sentiment of short, English-language text similar to tweets.

Limitations:

The model is only trained on two labels (positive, negative) and may not be suitable for more nuanced sentiment analysis.
The training data is from 2009 and may not capture modern slang or context.
The model is not intended for production use without further testing and evaluation.

Training Procedure

The model was trained for one epoch with a learning rate of 2e-4. The training process involved 4-bit quantization to reduce memory usage and make the fine-tuning process more efficient.

Training Hyperparameters

Framework: Transformers
Base Model: Qwen/Qwen2.5-3B-Instruct
Quantization: 4-bit (nf4)
LoRA r: 16
LoRA alpha: 32
Learning Rate: 0.0002
Epochs: 1
Batch Size: 8

Downloads last month: 1