Add pipeline tag and library name metadata

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
  license: mit
 
 
3
  tags:
4
  - diffusion
5
  - transformers
@@ -25,9 +27,9 @@ tags:
25
  _Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)_
26
 
27
  ## Abstract
28
- Recent multimodal face generation models address the spatial control limitations of text-to-image diffusion models by augmenting text-based conditioning with spatial priors such as segmentation masks, sketches, or edge maps. However, existing approaches typically append auxiliary control modules or stitch together separate uni-modal networks.
29
 
30
- We introduce **MMFace-DiT**, a unified dual-stream diffusion transformer engineered for synergistic multimodal face synthesis. Its core novelty lies in a dual-stream transformer block that processes spatial (mask/sketch) and semantic (text) tokens in parallel, deeply fusing them through a shared **Rotary Position-Embedded (RoPE) Attention** mechanism. Furthermore, a novel **Modality Embedder** enables a single cohesive model to dynamically adapt to varying spatial conditions without retraining. MMFace-DiT achieves a 40% improvement in visual fidelity and prompt alignment over five state-of-the-art multimodal face generation models.
31
 
32
  ## Repository Contents
33
  This repository contains the trained model checkpoints for MMFace-DiT. The models are provided for both Diffusion and Rectified Flow Matching (Flow) paradigms across different resolutions.
@@ -54,7 +56,7 @@ python sample_flow.py \
54
  --guidance_scale 7.5
55
  ```
56
 
57
- ## Citation:
58
  If you find this work helpful for your research, please cite our CVPR paper:
59
 
60
  ```bibtex
 
1
  ---
2
  license: mit
3
+ library_name: diffusers
4
+ pipeline_tag: text-to-image
5
  tags:
6
  - diffusion
7
  - transformers
 
27
  _Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)_
28
 
29
  ## Abstract
30
+ Recent multimodal face generation models address the spatial control limitations of text-to-image diffusion models by augmenting text-based conditioning with spatial priors such as segmentation masks, sketches, or edge maps. However, most existing approaches typically extend pre-trained text-to-image pipelines by appending auxiliary control modules or stitching together separate uni-modal networks.
31
 
32
+ We introduce **MMFace-DiT**, a unified dual-stream diffusion transformer engineered for synergistic multimodal face synthesis. Its core novelty lies in a dual-stream transformer block that processes spatial (mask/sketch) and semantic (text) tokens in parallel, deeply fusing them through a shared **Rotary Position-Embedded (RoPE) Attention** mechanism. Furthermore, a novel **Modality Embedder** enables a single cohesive model to dynamically adapt to varying spatial conditions without retraining. MMFace-DiT achieves a 40% improvement in visual fidelity and prompt alignment over six state-of-the-art multimodal face generation models, establishing a flexible new paradigm for end-to-end controllable generative modeling.
33
 
34
  ## Repository Contents
35
  This repository contains the trained model checkpoints for MMFace-DiT. The models are provided for both Diffusion and Rectified Flow Matching (Flow) paradigms across different resolutions.
 
56
  --guidance_scale 7.5
57
  ```
58
 
59
+ ## Citation
60
  If you find this work helpful for your research, please cite our CVPR paper:
61
 
62
  ```bibtex