Add metadata and improve model card

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +14 -3
README.md CHANGED
@@ -1,3 +1,9 @@
 
 
 
 
 
 
1
  <div align ="center">
2
  <h1> Proteus-ID </h1>
3
  <h3> Proteus-ID: ID-Consistent and Motion-Coherent Video Customization </h3>
@@ -8,6 +14,12 @@
8
  [![arXiv](https://img.shields.io/badge/arXiv-2506.23729-b31b1b.svg)](https://arxiv.org/abs/2506.23729)&nbsp;
9
  </div>
10
 
 
 
 
 
 
 
11
  Authors: [Guiyu Zhang](https://grenoble-zhang.github.io/)<sup>1</sup>, [Chen Shi](https://scholar.google.com.hk/citations?user=o-K_AoYAAAAJ&hl=en)<sup>1</sup>, Zijian Jiang<sup>1</sup>, Xunzhi Xiang<sup>2</sup>, Jingjing Qian<sup>1</sup>, [Shaoshuai Shi](https://shishaoshuai.com/)<sup>3</sup>, [Li Jiang†](https://llijiang.github.io/)<sup>1</sup>
12
 
13
  <sup>1</sup> The Chinese University of Hong Kong, Shenzhen&emsp;<sup>2</sup> Nanjing University&emsp;
@@ -25,8 +37,7 @@ Authors: [Guiyu Zhang](https://grenoble-zhang.github.io/)<sup>1</sup>, [Chen Shi
25
  ```bash
26
  # 0. Clone the repo
27
  git clone --depth=1 https://github.com/grenoble-zhang/Proteus-ID.git
28
-
29
- cd /nfs/dataset-ofs-voyager-research/guiyuzhang/Opensource/code/Proteus-ID-main
30
 
31
  # 1. Create conda environment
32
  conda create -n proteusid python=3.11.0
@@ -89,4 +100,4 @@ If you find our work useful in your research, please consider citing our paper:
89
 
90
  ## Acknowledgement
91
 
92
- Thansk for these excellent opensource works and models: [CogVideoX](https://github.com/THUDM/CogVideo); [ConsisID](https://github.com/PKU-YuanGroup/ConsisID); [diffusers](https://github.com/huggingface/diffusers).
 
1
+ ---
2
+ pipeline_tag: image-to-video
3
+ library_name: diffusers
4
+ license: apache-2.0
5
+ ---
6
+
7
  <div align ="center">
8
  <h1> Proteus-ID </h1>
9
  <h3> Proteus-ID: ID-Consistent and Motion-Coherent Video Customization </h3>
 
14
  [![arXiv](https://img.shields.io/badge/arXiv-2506.23729-b31b1b.svg)](https://arxiv.org/abs/2506.23729)&nbsp;
15
  </div>
16
 
17
+ **Proteus-ID** is a novel diffusion-based framework for identity-consistent and motion-coherent video customization. It addresses the challenges of maintaining subject identity while generating fluid, natural motion by introducing:
18
+
19
+ - **Multimodal Identity Fusion (MIF)**: Unifies visual and textual cues into a joint identity representation.
20
+ - **Time-Aware Identity Injection (TAII)**: Dynamically modulates identity conditioning across denoising steps for fine-detail reconstruction.
21
+ - **Adaptive Motion Learning (AML)**: Enhances motion realism using optical-flow-derived heatmaps.
22
+
23
  Authors: [Guiyu Zhang](https://grenoble-zhang.github.io/)<sup>1</sup>, [Chen Shi](https://scholar.google.com.hk/citations?user=o-K_AoYAAAAJ&hl=en)<sup>1</sup>, Zijian Jiang<sup>1</sup>, Xunzhi Xiang<sup>2</sup>, Jingjing Qian<sup>1</sup>, [Shaoshuai Shi](https://shishaoshuai.com/)<sup>3</sup>, [Li Jiang†](https://llijiang.github.io/)<sup>1</sup>
24
 
25
  <sup>1</sup> The Chinese University of Hong Kong, Shenzhen&emsp;<sup>2</sup> Nanjing University&emsp;
 
37
  ```bash
38
  # 0. Clone the repo
39
  git clone --depth=1 https://github.com/grenoble-zhang/Proteus-ID.git
40
+ cd Proteus-ID
 
41
 
42
  # 1. Create conda environment
43
  conda create -n proteusid python=3.11.0
 
100
 
101
  ## Acknowledgement
102
 
103
+ Thanks for these excellent opensource works and models: [CogVideoX](https://github.com/THUDM/CogVideo); [ConsisID](https://github.com/PKU-YuanGroup/ConsisID); [diffusers](https://github.com/huggingface/diffusers).