Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CAMOUFLaGE: Controllable AnoniMizatiOn throUgh diFfusion-based image coLlection GEneration
|
| 2 |
+
|
| 3 |
+
Code 
|
| 4 |
+
|
| 5 |
+
Official implementations of ["Latent Diffusion Models for Attribute-Preserving Image Anonymization"](#latent-diffusion-models-for-attribute-preserving-image-anonymization)
|
| 6 |
+
and ["Harnessing Foundation Models for Image Anonymization"](#harnessing-foundation-models-for-image-anonymization).
|
| 7 |
+
|
| 8 |
+
## Latent Diffusion Models for Attribute-Preserving Image Anonymization
|
| 9 |
+
|
| 10 |
+
[[Paper]](https://arxiv.org/abs/2403.14790)
|
| 11 |
+
|
| 12 |
+
This paper presents, to the best of our knowledge, the first approach to image anonymization based on
|
| 13 |
+
Latent Diffusion Models (LDMs). Every element of a scene is maintained to convey the same meaning, yet
|
| 14 |
+
manipulated in a way that makes re-identification difficult. We propose two LDMs for this purpose:
|
| 15 |
+
|
| 16 |
+
- *CAMOUFLaGE-Base*
|
| 17 |
+
- *CAMOFULaGE-Light*
|
| 18 |
+
|
| 19 |
+
The former solution achieves superior performance on most metrics and benchmarks, while the latter cuts
|
| 20 |
+
the inference time in half at the cost of fine-tuning a lightweight module.
|
| 21 |
+
|
| 22 |
+
Compared to state-of-the-art, we anonymize complex scenes by introducing variations in the faces, bodies,
|
| 23 |
+
and background elements.
|
| 24 |
+
|
| 25 |
+
#### CAMOUFLaGE-Base
|
| 26 |
+
|
| 27 |
+
CAMOUFLaGE-Base exploits a combination of pre-trained ControlNets and introduces an anonymizazion guidance based on
|
| 28 |
+
the original image.
|
| 29 |
+
|
| 30 |
+

|
| 31 |
+
|
| 32 |
+
More details on its usage can be found [here](CAMOUFLaGE-Base-v1-0).
|
| 33 |
+
|
| 34 |
+
#### CAMOUFLaGE-Light
|
| 35 |
+
|
| 36 |
+
CAMOUFLaGE-Light trains a lightweight IP-Adapter to encode key elements of the scene and facial attributes of each
|
| 37 |
+
person.
|
| 38 |
+
|
| 39 |
+

|
| 40 |
+
|
| 41 |
+
More details on its usage can be found [here](CAMOUFLaGE_light).
|
| 42 |
+
|
| 43 |
+
## Harnessing Foundation Models for Image Anonymization
|
| 44 |
+
|
| 45 |
+
[[Paper]]()
|
| 46 |
+
|
| 47 |
+
We explore how foundation models can be leveraged to solve tasks, specifically focusing on anonymization,
|
| 48 |
+
without the requirement for training or fine-tuning. By bypassing traditional pipelines, we demonstrate the
|
| 49 |
+
efficiency and effectiveness of this approach in achieving anonymization objectives directly from the
|
| 50 |
+
foundation model’s inherent knowledge.
|
| 51 |
+
|
| 52 |
+
#### CAMOUFLaGE-Face
|
| 53 |
+
|
| 54 |
+
We examine how foundation models can generate anonymized images directly from textual descriptions. Two models
|
| 55 |
+
were employed for information extraction: FACER, used to identify the 40 CelebA-HQ attributes, and DeepFace,
|
| 56 |
+
used to determine ethnicity and age. Using this rich information, we craft captions to guide the generation process.
|
| 57 |
+
Classifier-free guidance was employed to push the image content in the direction of the positive prompt P and far
|
| 58 |
+
from the negative prompt ¬P.
|
| 59 |
+
|
| 60 |
+

|
| 61 |
+
|
| 62 |
+
More details on its usage can be found [here](GEM2024).
|
| 63 |
+
|
| 64 |
+
## Citation
|
| 65 |
+
|
| 66 |
+
If you find CAMOUFLaGE-Base and/or CAMOUFLaGE-Light useful, please cite:
|
| 67 |
+
|
| 68 |
+
```
|
| 69 |
+
@misc{camouflage,
|
| 70 |
+
title={Latent Diffusion Models for Attribute-Preserving Image Anonymization},
|
| 71 |
+
author={Luca Piano and Pietro Basci and Fabrizio Lamberti and Lia Morra},
|
| 72 |
+
year={2024},
|
| 73 |
+
eprint={2403.14790},
|
| 74 |
+
archivePrefix={arXiv},
|
| 75 |
+
primaryClass={cs.CV}
|
| 76 |
+
}
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
If you find CAMOUFLaGE-Face useful, please cite:
|
| 80 |
+
|
| 81 |
+
```
|
| 82 |
+
@inproceedings{pianoGEM24,
|
| 83 |
+
title={Harnessing Foundation Models for Image Anonymization},
|
| 84 |
+
author={Piano, Luca and Basci, Pietro and Lamberti, Fabrizio and Morra, Lia},
|
| 85 |
+
booktitle={2024 IEEE CTSoc Gaming, Entertainment and Media},
|
| 86 |
+
year={2024},
|
| 87 |
+
organization={IEEE}
|
| 88 |
+
}
|
| 89 |
+
```
|
| 90 |
+
|