my-test-model

This model is a fine-tuned version of distilbert-base-uncased on imdb dataset.

Model description

This model is a fine-tuned version of DistilBERT-base-uncased for binary sentiment analysis on movie reviews. Key specifications:

Task: Sentiment classification (positive/negative)

Base Architecture: 6-layer distilled Transformer model

Parameters: ~66 million (standard DistilBERT configuration)

Output Labels:

0 → "NEGATIVE"

1 → "POSITIVE"

Intended uses & limitations

Acceptable Use Cases ✅ Sentiment analysis of English movie reviews

Educational/research purposes for text classification

Baseline model for entertainment industry applications

Integration in sentiment analysis pipelines

Limitations ⚠️ Language Restriction: Only supports English text

Domain Specificity: Optimized for movie reviews - performance degrades on other text types

Bias Risks: May reflect demographic/cultural biases in training data

Length Constraint: Maximum input length of 256 tokens (longer texts are truncated)

Not Suitable For:

Multilingual text analysis

Sarcasm/irony detection

Fine-grained sentiment analysis (e.g., detecting anger, excitement)

Training and evaluation data

Training Data Dataset: IMDB Movie Reviews

Size: 25,000 labeled examples

Class Distribution:

Positive: 12,500 (50%)

Negative: 12,500 (50%)

Preprocessing:

Lowercasing

DistilBERT tokenization (WordPiece)

Dynamic padding

Evaluation Data Test Set: Official IMDB test split (25,000 examples)

Training procedure

TrainingArguments( num_train_epochs=3, per_device_train_batch_size=16, per_device_eval_batch_size=64, learning_rate=2e-5, weight_decay=0.01, evaluation_strategy="epoch", save_strategy="epoch", metric_for_best_model="accuracy" )

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 64
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.2497	1.0	1563	0.2486	0.9026	0.9024
0.1496	2.0	3126	0.2896	0.9135	0.9135
0.1222	3.0	4689	0.3448	0.9130	0.9130

Framework versions

• Transformers 4.52.3
• Pytorch 2.7.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1