PolyAI/banking77
Updated • 7.93k • 70
How to use Anant1213/bert-base-banking77-student-pseudo-v1 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-classification", model="Anant1213/bert-base-banking77-student-pseudo-v1") # Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("Anant1213/bert-base-banking77-student-pseudo-v1")
model = AutoModelForSequenceClassification.from_pretrained("Anant1213/bert-base-banking77-student-pseudo-v1")anant1213/bert-base-banking77-student-pseudo-v1 (BERT Student — Pseudo-Label Distilled)
This repository contains a BERT-base intent classifier fine-tuned on the BANKING77 dataset using a teacher pseudo-labeling distillation approach (Path 1). A fine-tuned BERT-large teacher generates predicted intent labels for each training example; the BERT-base student is then fine-tuned on these teacher-generated labels and evaluated on the original test set.
BertForSequenceClassification (BERT encoder + classification head)google-bert/bert-base-uncased :contentReference[oaicite:1]{index=1}google-bert/bert-large-uncased fine-tuned on BANKING77 :contentReference[oaicite:2]{index=2}google-bert/bert-base-uncased :contentReference[oaicite:3]{index=3} google-bert/bert-large-uncased :contentReference[oaicite:4]{index=4} PolyAI/banking77 :contentReference[oaicite:5]{index=5}from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
repo_id = "anant1213/bert-base-banking77-student-pseudo-v1"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
text = "I was charged twice for the same card payment"
inputs = tokenizer(text, return_tensors="pt", truncation=True)
with torch.no_grad():
logits = model(**inputs).logits
pred_id = int(torch.argmax(logits, dim=-1))
print("Predicted label id:", pred_id)
Base model
google-bert/bert-base-uncased