Instructions to use BharathK333/MMFace-DiT-Models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use BharathK333/MMFace-DiT-Models with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("BharathK333/MMFace-DiT-Models", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Transformers
How to use BharathK333/MMFace-DiT-Models with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("BharathK333/MMFace-DiT-Models", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add pipeline tag and library name metadata
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
tags:
|
| 4 |
- diffusion
|
| 5 |
- transformers
|
|
@@ -25,9 +27,9 @@ tags:
|
|
| 25 |
_Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)_
|
| 26 |
|
| 27 |
## Abstract
|
| 28 |
-
Recent multimodal face generation models address the spatial control limitations of text-to-image diffusion models by augmenting text-based conditioning with spatial priors such as segmentation masks, sketches, or edge maps. However, existing approaches typically
|
| 29 |
|
| 30 |
-
We introduce **MMFace-DiT**, a unified dual-stream diffusion transformer engineered for synergistic multimodal face synthesis. Its core novelty lies in a dual-stream transformer block that processes spatial (mask/sketch) and semantic (text) tokens in parallel, deeply fusing them through a shared **Rotary Position-Embedded (RoPE) Attention** mechanism. Furthermore, a novel **Modality Embedder** enables a single cohesive model to dynamically adapt to varying spatial conditions without retraining. MMFace-DiT achieves a 40% improvement in visual fidelity and prompt alignment over
|
| 31 |
|
| 32 |
## Repository Contents
|
| 33 |
This repository contains the trained model checkpoints for MMFace-DiT. The models are provided for both Diffusion and Rectified Flow Matching (Flow) paradigms across different resolutions.
|
|
@@ -54,7 +56,7 @@ python sample_flow.py \
|
|
| 54 |
--guidance_scale 7.5
|
| 55 |
```
|
| 56 |
|
| 57 |
-
## Citation
|
| 58 |
If you find this work helpful for your research, please cite our CVPR paper:
|
| 59 |
|
| 60 |
```bibtex
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
pipeline_tag: text-to-image
|
| 5 |
tags:
|
| 6 |
- diffusion
|
| 7 |
- transformers
|
|
|
|
| 27 |
_Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)_
|
| 28 |
|
| 29 |
## Abstract
|
| 30 |
+
Recent multimodal face generation models address the spatial control limitations of text-to-image diffusion models by augmenting text-based conditioning with spatial priors such as segmentation masks, sketches, or edge maps. However, most existing approaches typically extend pre-trained text-to-image pipelines by appending auxiliary control modules or stitching together separate uni-modal networks.
|
| 31 |
|
| 32 |
+
We introduce **MMFace-DiT**, a unified dual-stream diffusion transformer engineered for synergistic multimodal face synthesis. Its core novelty lies in a dual-stream transformer block that processes spatial (mask/sketch) and semantic (text) tokens in parallel, deeply fusing them through a shared **Rotary Position-Embedded (RoPE) Attention** mechanism. Furthermore, a novel **Modality Embedder** enables a single cohesive model to dynamically adapt to varying spatial conditions without retraining. MMFace-DiT achieves a 40% improvement in visual fidelity and prompt alignment over six state-of-the-art multimodal face generation models, establishing a flexible new paradigm for end-to-end controllable generative modeling.
|
| 33 |
|
| 34 |
## Repository Contents
|
| 35 |
This repository contains the trained model checkpoints for MMFace-DiT. The models are provided for both Diffusion and Rectified Flow Matching (Flow) paradigms across different resolutions.
|
|
|
|
| 56 |
--guidance_scale 7.5
|
| 57 |
```
|
| 58 |
|
| 59 |
+
## Citation
|
| 60 |
If you find this work helpful for your research, please cite our CVPR paper:
|
| 61 |
|
| 62 |
```bibtex
|