File size: 1,090 Bytes
9246cbd 248e56f 993d637 248e56f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
license: mit
datasets:
- stanfordnlp/sst2
language:
- en
metrics:
- accuracy
tags:
- pytorch
- nlp
- text-classification
- sst2
---
# CLSE-v1 by Lloid
Custom Encoder Transformer trained on SST-2 sentiment classification from scratch.
Accuracy: 77% on SST-2 validation set.
## Usage
```python
import torch
from transformers import AutoTokenizer
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("lloid-labs/CLSE-v1")
# Load model (copy the Model class from the repo)
model = Model(vocab_size=30522, d_model=256, n_heads=8, N_layers=4, T=128, out_features=2)
model.load_state_dict(torch.load("model.pth", map_location="cpu"))
model.eval()
# Inference
sentence = "This movie was great!"
inputs = tokenizer(sentence, return_tensors="pt", padding="max_length",
truncation=True, max_length=128)
with torch.no_grad():
logits = model(inputs["input_ids"])
pred = torch.argmax(logits, dim=-1).item()
print("Positive" if pred == 1 else "Negative")
```
## Architecture
- 4 Encoder layers
- 8 attention heads
- d_model: 256
- Trained from scratch on SST-2 |