my420254/hipro-lora-smp2020

HiPro-LoRA — SMP2020

Adaptive Hierarchical State Pooling + Tail-aware Prototype Memory Bank for low-resource sentiment classification.

Model Details

Item	Value
Base model	hfl/chinese-macbert-base
LoRA rank	16
LoRA alpha	32
Num labels	6
Labels	happy, angry, sad, fear, surprise, neutral
Test Macro-F1	71.48%
Test Accuracy	71.33%

Usage

import torch
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("hfl/chinese-macbert-base")
# Load model architecture first, then load state dict
model.load_state_dict(torch.load("model.pt", map_location="cpu"))
model.eval()

Architecture

• AHSP: Learnable layer-wise attention + gated pooling over transformer hidden states
• TPMB: Per-class memory bank with tail-aware contrastive prototype regularization
• LoRA: Low-rank adaptation on query/value projection matrices

Citation

If you use this model, please cite the HiPro-LoRA paper.