|
|
--- |
|
|
base_model: |
|
|
- microsoft/MiniLM-L12-H384-uncased |
|
|
language: |
|
|
- en |
|
|
library_name: transformers |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# Fine-tuned LoRA Classifier on MiniLM for IAB Multi-Label Classification |
|
|
|
|
|
This is a fine-tuned LoRA (Low-Rank Adaptation) classifier based on MiniLM (microsoft/MiniLM-L12-H384-uncased), designed for multi-label content classification using the IAB content taxonomy. The model can assign one or more categories to input text — making it suitable for tasks such as content classification. |
|
|
|
|
|
🔍 Model Details |
|
|
Model Description |
|
|
|
|
|
This model is based on microsoft/MiniLM-L12-H384-uncased, a compact and efficient transformer model optimized for fast inference and low memory footprint. It has been fine-tuned using LoRA for multi-label classification over 20 IAB categories plus an "inconclusive" fallback class. |
|
|
|
|
|
The model predicts multiple applicable content labels from: |
|
|
|
|
|
inconclusive |
|
|
animals |
|
|
arts |
|
|
autos |
|
|
business |
|
|
career |
|
|
education |
|
|
fashion |
|
|
finance |
|
|
food |
|
|
government |
|
|
health |
|
|
hobbies |
|
|
home |
|
|
news |
|
|
realestate |
|
|
society |
|
|
sports |
|
|
tech |
|
|
travel |
|
|
|
|
|
Key Configuration: |
|
|
|
|
|
Base Model: microsoft/MiniLM-L12-H384-uncased |
|
|
Task: Multi-label content classification |
|
|
Label Count: 21 (multi-hot vector) |
|
|
Language: English |
|
|
Fine-tuning Method: PEFT with LoRA |
|
|
LoRA Config: |
|
|
r=16 |
|
|
lora_alpha=16 |
|
|
lora_dropout=0.1 |
|
|
target_modules=["query", "key"] |
|
|
Developed by: Mozilla |
|
|
License: Apache-2.0 |
|
|
|
|
|
📦 Model Sources: |
|
|
Demo (optional): [Hugging Face Space](https://huggingface.co/spaces/chidamnat2002/iab_content_classifier) |
|
|
|
|
|
|
|
|
📥 Usage: |
|
|
``` |
|
|
from transformers import AutoTokenizer, AutoModelForSequenceClassification |
|
|
import torch |
|
|
|
|
|
model = AutoModelForSequenceClassification.from_pretrained("Mozilla/content-multilabel-iab-classifier") |
|
|
tokenizer = AutoTokenizer.from_pretrained("Mozilla/content-multilabel-iab-classifier") |
|
|
|
|
|
label_list = [ |
|
|
'inconclusive', |
|
|
'animals', |
|
|
'arts', |
|
|
'autos', |
|
|
'business', |
|
|
'career', |
|
|
'education', |
|
|
'fashion', |
|
|
'finance', |
|
|
'food', |
|
|
'government', |
|
|
'health', |
|
|
'hobbies', |
|
|
'home', |
|
|
'news', |
|
|
'realestate', |
|
|
'society', |
|
|
'sports', |
|
|
'tech', |
|
|
'travel' |
|
|
] |
|
|
label2id = {label: idx for idx, label in enumerate(label_list)} |
|
|
id2label = {idx: label for label, idx in label2id.items()} |
|
|
|
|
|
text = "Discover the latest trends in AI and wearable technology." |
|
|
|
|
|
with torch.no_grad(): |
|
|
inputs = tokenizer(text, return_tensors="pt", truncation=True) |
|
|
outputs = model(**inputs) |
|
|
probs = torch.sigmoid(outputs.logits).squeeze().cpu().numpy() |
|
|
predicted_labels = [(id2label[i], round(p, 3)) for i, p in enumerate(probs) if p >= 0.5] |
|
|
``` |
|
|
|
|
|
📖 Citation |
|
|
|
|
|
If you use this model, please cite it as: |
|
|
``` |
|
|
@misc{mozilla_iab_multilabel_lora, |
|
|
title = {Fine-tuned LoRA Classifier on MiniLM for IAB Multi-Label Classification}, |
|
|
author = {Mozilla}, |
|
|
year = {2025}, |
|
|
url = {https://huggingface.co/mozilla/content-multilabel-iab-classifier}, |
|
|
license = {Apache-2.0} |
|
|
} |
|
|
``` |
