new project named scepter SCEdit

README.md

@@ -1,3 +1,72 @@
 ---
 license: apache-2.0
+language:
+- en
+tags:
+- diffusion model
+- stable diffusion
+- SCEdit
+- Scepter
+- Scepter studio
+- Controllable
+- ControlNet
+- Lora
 ---
+<p align="center">
+
+  <h2 align="center">🪄SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing</h2>
+  <p align="center">
+    <strong>Zeyinzi Jiang</strong>
+    ·
+    <strong>Chaojie Mao</strong>
+    ·
+    <strong>Yulin Pan</strong>
+    ·
+    <strong>Zhen Han</strong>
+    ·
+    <strong>Jingfeng Zhang</strong>
+    <br>
+    <br>
+        <a href="https://arxiv.org/abs/2312.11392"><img src='https://img.shields.io/badge/arXiv-SCEdit-red' alt='Paper PDF'></a>
+        <a href='https://scedit.github.io/'><img src='https://img.shields.io/badge/Project_Page-SCEdit-green' alt='Project Page'></a>
+        <a href='https://github.com/modelscope/scepter'><img src='https://img.shields.io/badge/scepter-SCEdit-yellow'></a>
+        <a href='https://github.com/modelscope/swift'><img src='https://img.shields.io/badge/swift-SCEdit-blue'></a>
+    <br>
+    <b>Alibaba Group</b>
+  </p>
+  
+  <table align="center">
+    <tr>
+    <td>
+      <img src="assets/figures/show.jpg">
+    </td>
+    </tr>
+  </table>
+
+SCEdit is an efficient generative fine-tuning framework proposed by Alibaba TongYi Vision Intelligence Lab. This framework enhances the fine-tuning capabilities for text-to-image generation downstream tasks and enables quick adaptation to specific generative scenarios, **saving 30%-50% of training memory costs compared to LoRA**. Furthermore, it can be directly extended to controllable image generation tasks, **requiring only 7.9% of the parameters that ControlNet needs for conditional generation and saving 30% of memory usage**. It supports various conditional generation tasks including edge maps, depth maps, segmentation maps, poses, color maps, and image completion.
+#### Code Example
+```shell
+git clone https://github.com/modelscope/scepter.git
+cd scepter
+PYTHONPATH=. python scepter/tools/run_train.py --cfg scepter/methods/SCEdit/t2i_sdxl_1024_sce.yaml
+```
+
+To prepare  the training dataset.
+
+```python
+# pip install modelscope
+from modelscope.msdatasets import MsDataset
+ms_train_dataset = MsDataset.load('style_custom_dataset', namespace='damo', subset_name='3D', split='train_short')
+print(next(iter(ms_train_dataset)))
+```
+
+## BibTeX
+
+```bibtex
+@article{jiang2023scedit,
+    title = {SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing},
+    author = {Jiang, Zeyinzi and Mao, Chaojie and Pan, Yulin and Han, Zhen and Zhang, Jingfeng},
+    year = {2023},
+    journal = {arXiv preprint arXiv:2312.11392}  
+}
+```