DornierDo17/MiniRoBERTa_SST2

This model is a fine-tuned version of my base MiniRoBERTa (17.7M parameters) model. The goal of this fine-tuning experiment was to demonstrate that a RoBERTa-style model, built entirely from scratch and trained on a single GPU with limited compute, can still learn meaningful patterns and adapt effectively to downstream tasks.

The model was fine-tuned on the SST-2 sentiment classification dataset and achieved an accuracy of 80%, which is a strong result given the scale and simplicity of the pretraining setup.

This validates that the model has learned generalizable representations and can be adapted successfully to real-world NLP tasks through fine-tuning.

Highlights:

• Fine-tuned from scratch-trained RoBERTa (17.7M) model
• Dataset: SST-2 (Stanford Sentiment Treebank)
• Accuracy: 80%
• Trained on: Single GPU (limited compute)