metadata
license: mit
datasets:
- stanfordnlp/sst2
language:
- en
metrics:
- accuracy
base_model:
- DornierDo17/RoBERTa_17.7M
pipeline_tag: text-classification
library_name: transformers
tags:
- '#sst2'
- '#text-classification'
- '#ai'
This model is a fine-tuned version of my base MiniRoBERTa (17.7M parameters) model. The goal of this fine-tuning experiment was to demonstrate that a RoBERTa-style model, built entirely from scratch and trained on a single GPU with limited compute, can still learn meaningful patterns and adapt effectively to downstream tasks.
The model was fine-tuned on the SST-2 sentiment classification dataset and achieved an accuracy of 80%, which is a strong result given the scale and simplicity of the pretraining setup.
This validates that the model has learned generalizable representations and can be adapted successfully to real-world NLP tasks through fine-tuning.
Highlights:
- Fine-tuned from scratch-trained RoBERTa (17.7M) model
- Dataset: SST-2 (Stanford Sentiment Treebank)
- Accuracy: 80%
- Trained on: Single GPU (limited compute)