Rasooli/kabert-sentiment-persian

🔹 Usage

This model uses a custom PyTorch KABERT architecture and therefore cannot be loaded via Hugging Face pipeline or AutoModel. Inference must be performed manually using PyTorch, as shown below.

Requirements pip install torch transformers huggingface_hub

Load model and tokenizer import torch import torch.nn as nn from huggingface_hub import hf_hub_download from transformers import AutoTokenizer, AutoModel

REPO_ID = "Rasooli/kabert-sentiment-persian" BASE_MODEL = "HooshvareLab/bert-base-parsbert-uncased" MAX_LEN = 192

Download trained weights

weights_path = hf_hub_download( repo_id=REPO_ID, filename="pytorch_model.bin" )

Load tokenizer

tokenizer = AutoTokenizer.from_pretrained(REPO_ID)

Load base BERT encoder

bert = AutoModel.from_pretrained(BASE_MODEL)

KABERT architecture (gate-based fusion) class GateFusion(nn.Module): def init(self, d_bert, num_labels): super().init() self.gate = nn.Linear(d_bert * 2, d_bert) self.out = nn.Linear(d_bert, num_labels)

def forward(self, H):
    cls = H[:, 0, :]
    avg = H.mean(dim=1)
    G = torch.sigmoid(self.gate(torch.cat([cls, avg], dim=-1)))
    fused = G * cls + (1 - G) * avg
    return self.out(fused)

class KABERT(nn.Module): def init(self, bert_model, num_labels=3): super().init() self.bert = bert_model self.fusion = GateFusion(self.bert.config.hidden_size, num_labels)

def forward(self, enc):
    out = self.bert(**enc)
    return self.fusion(out.last_hidden_state)

Load trained weights model = KABERT(bert, num_labels=3) state_dict = torch.load(weights_path, map_location="cpu") model.load_state_dict(state_dict, strict=True) model.eval()

Sentiment prediction labels = ["Negative", "Positive", "Neutral"] # 0, 1, 2

def predict_sentiment(text): enc = tokenizer( text, return_tensors="pt", truncation=True, padding=True, max_length=MAX_LEN ) with torch.no_grad(): logits = model(enc) pred = int(torch.argmax(logits, dim=1).item()) return labels[pred]

print(predict_sentiment("این محصول بسیار عالی است")) # Positive print(predict_sentiment("خیلی بد بود و پیشنهاد نمی‌کنم")) # Negative print(predict_sentiment("معمولی بود، نه خوب نه بد")) # Neutral

🔹 Label Mapping

The model was trained with the following label encoding:

ID Sentiment 0 Negative 1 Positive 2 Neutral 🔹 Notes

This repository contains a custom research architecture.

Hugging Face inference widgets are not enabled for non-PreTrainedModel classes.

The model is fully reproducible using the provided code.

you can test my model usin this part

labels = ["Negative", "Positive", "Neutral"] # 0,1,2

def predict_kabert(text: str, max_length: int = 192): enc = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=max_length) with torch.no_grad(): logits = model(enc) pred = int(torch.argmax(logits, dim=1).item()) return pred, labels[pred]

print(predict_kabert("کیفیت خیلی عالی بود"))

(1, 'Positive')