NiDeYingZiD/BHGap-ckpt

BHGap · Trained Checkpoints

A Deep Iterative Prompting and Multi-stage Alignment Framework for Dynamic Facial Expression Recognition

🤗 This repository hosts the trained model weights for BHGap (WWW 2026 Oral). For the full code, training and evaluation pipeline, please visit the GitHub repository.

Frozen audio-visual encoders + SDIC (iterative cross-modal prompting) + MSC2F (coarse-to-fine alignment) + lightweight fusion.

📖 Abstract

Dynamic Facial Expression Recognition (DFER) is a crucial part of affective computing, with broad applications in human-computer interaction and social media content analysis. Effectively integrating audio-visual signals remains the core challenge, as existing approaches are constrained by (1) shallow, static fusion that fails to capture the dynamic co-evolution of features, and (2) implicit, coarse alignment that cannot bridge the modality gap.

BHGap reformulates audio-visual collaboration from a one-shot fusion event into a continuous, reciprocal generation process spanning every layer of frozen backbone encoders:

• SDIC — an SSM (Mamba)-based Cross-Modal Prompt Generator dynamically produces guidance prompts for the counterpart modality at each encoding layer, enabling deep and fine-grained feature co-evolution.
• MSC2F — a coarse-to-fine alignment module that combines low-rank adversarial alignment (macro-level distribution & spatio-temporal congruity) with MMD-driven implicit differentiation (micro-level statistical & semantic consistency).

It achieves state-of-the-art performance on DFEW and MAFW using raw audio-visual inputs only.

📊 Results

Evaluated under 5-fold cross-validation (WAR = Weighted Average Recall, UAR = Unweighted Average Recall).

Dataset	Modality	WAR (%)	UAR (%)
DFEW	Audio + Visual	78.80	69.03
MAFW	Audio + Visual	59.97	47.68

📦 Checkpoints

This repository provides the trained BHGap checkpoints for DFEW and MAFW (5-fold). Each fold produces the best models selected by WAR / UAR (model_best_war.pth, model_best_uar.pth); see the Files and versions tab for the exact contents.

🚀 Usage

Download the checkpoints from the Hub:

huggingface-cli download NiDeYingZiD/BHGap-ckpt --local-dir ./checkpoints

from huggingface_hub import snapshot_download

snapshot_download(repo_id="NiDeYingZiD/BHGap-ckpt", local_dir="./checkpoints")

Then evaluate with the code from the GitHub repository:

python evaluate.py --dataset DFEW --checkpoint ./checkpoints/model_best_war.pth --fold 5

Note — These are task checkpoints for the BHGap pipeline (frozen MAE-Face / AudioMAE backbones + trainable SDIC & MSC2F modules), not a standalone transformers model. Please load them through the BHGap code.

📌 Citation

@inproceedings{zhang2026bhgap,
  title     = {BHGap: A Deep Iterative Prompting and Multi-stage Alignment Framework for Dynamic Facial Expression Recognition},
  author    = {Zhang, Yichi and Han, Yunqi and Ding, Jiayue and Chen, Liangyu},
  booktitle = {Proceedings of the ACM Web Conference 2026 (WWW '26)},
  year      = {2026},
  address   = {Dubai, United Arab Emirates},
  publisher = {Association for Computing Machinery},
  doi       = {10.1145/3774904.3792417}
}

🙏 Acknowledgements

Built upon MAE-Face, AudioMAE, Mamba, MMA-DFER, and the DFEW / MAFW datasets. Thank you for these excellent efforts!