Enhance model card: Add metadata, paper abstract, and links

README.md

@@ -1,92 +1,101 @@
-
-# An efficient watermarking method for latent diffusion models via low-rank adaptation 
-
-Code for our paper "An efficient watermarking method for latent diffusion models via low-rank adaptation".
-
-You can download the paper via: [[ArXiv]](https://arxiv.org/abs/2410.20202)
-
-
-## 😀Summary
-
-A lightweight parameter fine-tuning strategy with low-rank adaptation and dynamic loss weight adjustment enables efficient watermark embedding in large-scale models while minimizing impact on image quality and maintaining robustness.
-
-![image](diagram.png)
-
-## 🍉Requirement
-
-```shell
-pip install -r requirements.txt
-```
-
-## 🐬Preparation
-
-### Clone
-
-```shell
-git clone https://github.com/MrDongdongLin/EW-LoRA
-```
-
-### Create an anaconda environment [Optional]:
-
-```shell
-conda create -n ewlora python==3.8.18
-conda activate ewlora
-pip install -r requirements.txt
-```
-
-### Prepare the training data:
-
-* Download the dataset files [here](https://cocodataset.org/).
-* Extract them to the `data` folder.
-* The directory structure will be as follows:
-
-```shell
-coco2017
-└── train
-   ├── img1.jpg
-   ├── img2.jpg
-   └── img3.jpg
-└── test
-   ├── img4.jpg
-   ├── img5.jpg
-   └── img6.jpg
-```
-
-### Usage
-
-#### Training
-
-```shell
-cd ./watermarker/stable_signature
-CUDA_VISIBLE_DEVICES=0 python train_SS.py --num_keys 1 \
---train_dir ./Datasets/coco2017/train2017 \
---val_dir ./Datasets/coco2017/val2017 \
---ldm_config ./watermarker/stable_signature/configs/stable-diffusion/v1-inference.yaml \
---ldm_ckpt ../models/ldm_ckpts/sd-v1-4-full-ema.ckpt \
---msg_decoder_path ../models/wm_encdec/hidden/ckpts/dec_48b_whit.torchscript.pt \
---output_dir ./watermarker/stable_signature/outputs/ \
---task_name train_SS_fix_weights \
---do_validation \
---val_frep 50 \
---batch_size 4 \
---lambda_i 1.0 --lambda_w 0.2 \
---steps 20000 --val_size 100 \
---warmup_steps 20 \
---save_img_freq 100 \
---log_freq 1 --debug
-```
-
-## Citation
-
-If this work is helpful, please cite as:
-
-```latex
-@article{linEfficientWatermarkingMethod2024,
-  title     = {An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation},
-  author    = {Lin, Dongdong and Li, Yue and Tondi, Benedetta and Li, Bin and Barni, Mauro},
-  year      = {2024},
-  month     = oct,
-  number    = {arXiv:2410.20202},
-  eprint    = {2410.20202},
-}
+---
+pipeline_tag: text-to-image
+library_name: diffusers
+license: unknown
+---
+
+# An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation and Dynamic Loss Weighting
+
+This repository hosts the model and code for the paper [An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation and Dynamic Loss Weighting](https://huggingface.co/papers/2410.20202).
+
+**Abstract:**
+The rapid proliferation of Deep Neural Networks (DNNs) is driving a surge in model watermarking technologies, as the trained models themselves constitute valuable intellectual property. Existing watermarking approaches primarily focus on modifying model parameters or altering sampling behaviors. However, with the emergence of increasingly large models, improving the efficiency of watermark embedding becomes essential to manage increasing computational demands. Prioritizing efficiency not only optimizes resource utilization, making the watermarking process more applicable for large models, but also mitigates potential degradation of model performance. In this paper, we propose an efficient watermarking method for Latent Diffusion Models (LDMs) based on Low-Rank Adaptation (LoRA). The core idea is to introduce trainable low-rank parameters into the frozen LDM to embed watermark, thereby preserving the integrity of the original model weights. Furthermore, a dynamic loss weight scheduler is designed to adaptively balance the objectives of generative quality and watermark fidelity, enabling the model to achieve effective watermark embedding with minimal impact on quality of the generated images. Experimental results show that the proposed method ensures fast and accurate watermark embedding and a high quality of the generated images, at the same time maintaining a level of robustness aligned - in some cases superior - with state-of-the-art approaches. Moreover, the method generalizes well across different datasets and base LDMs.
+
+**Code:** Find the official implementation on GitHub: [https://github.com/MrDongdongLin/EW-LoRA](https://github.com/MrDongdongLin/EW-LoRA)
+
+You can download the paper via: [[ArXiv]](https://arxiv.org/abs/2410.20202)
+
+## 😀Summary
+
+A lightweight parameter fine-tuning strategy with low-rank adaptation and dynamic loss weight adjustment enables efficient watermark embedding in large-scale models while minimizing impact on image quality and maintaining robustness.
+
+![image](https://github.com/MrDongdongLin/EW-LoRA/raw/main/diagram.png)
+
+## 🍉Requirement
+
+```shell
+pip install -r requirements.txt
+```
+
+## 🐬Preparation
+
+### Clone
+
+```shell
+git clone https://github.com/MrDongdongLin/EW-LoRA
+```
+
+### Create an anaconda environment [Optional]:
+
+```shell
+conda create -n ewlora python==3.8.18
+conda activate ewlora
+pip install -r requirements.txt
+```
+
+### Prepare the training data:
+
+* Download the dataset files [here](https://cocodataset.org/).
+* Extract them to the `data` folder.
+* The directory structure will be as follows:
+
+```shell
+coco2017
+└── train
+   ├── img1.jpg
+   ├── img2.jpg
+   └── img3.jpg
+└── test
+   ├── img4.jpg
+   ├── img5.jpg
+   └── img6.jpg
+```
+
+### Usage
+
+#### Training
+
+```shell
+cd ./watermarker/stable_signature
+CUDA_VISIBLE_DEVICES=0 python train_SS.py --num_keys 1 \
+--train_dir ./Datasets/coco2017/train2017 \
+--val_dir ./Datasets/coco2017/val2017 \
+--ldm_config ./watermarker/stable_signature/configs/stable-diffusion/v1-inference.yaml \
+--ldm_ckpt ../models/ldm_ckpts/sd-v1-4-full-ema.ckpt \
+--msg_decoder_path ../models/wm_encdec/hidden/ckpts/dec_48b_whit.torchscript.pt \
+--output_dir ./watermarker/stable_signature/outputs/ \
+--task_name train_SS_fix_weights \
+--do_validation \
+--val_frep 50 \
+--batch_size 4 \
+--lambda_i 1.0 --lambda_w 0.2 \
+--steps 20000 --val_size 100 \
+--warmup_steps 20 \
+--save_img_freq 100 \
+--log_freq 1 --debug
+```
+
+## Citation
+
+If this work is helpful, please cite as:
+
+```latex
+@article{linEfficientWatermarkingMethod2024,
+  title     = {An Efficient Watermarking Method for Latent Diffusion Models via Low-Rank Adaptation},
+  author    = {Lin, Dongdong and Li, Yue and Tondi, Benedetta and Li, Bin and Barni, Mauro},
+  year      = {2024},
+  month     = oct,
+  number    = {arXiv:2410.20202},
+  eprint    = {2410.20202},
+}
 ```