|
|
--- |
|
|
license: cc-by-nc-4.0 |
|
|
library_name: diffusion-single-file |
|
|
base_model: |
|
|
- black-forest-labs/FLUX.1-dev |
|
|
pipeline_tag: image-to-image |
|
|
--- |
|
|
|
|
|
<div align="center"> |
|
|
<a href="https://fotographer.ai/zen-control"> |
|
|
<picture> |
|
|
<source media="(max-width: 424px" srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/banner_sm.avif" type="image/avif"> |
|
|
<source media="(min-width: 425px" srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/banner.avif" type="image/avif"> |
|
|
<source media="(max-width: 424px" srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/banner_sm.webp" type="image/webp"> |
|
|
<source media="(min-width: 425px" srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/banner.webp" type="image/webp"> |
|
|
<img alt="ZenCtrl Banner" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/banner.png" /> |
|
|
</picture> |
|
|
</a> |
|
|
<h1>ZenCtrl</h1> |
|
|
</div> |
|
|
|
|
|
We are making an Agent that can automate the whole personalized visual content creation process. It will need to perform multiple types of tasks, including designing tasks and training a model for it's own use. |
|
|
This agent will handle the data, the models and ensure the quality of the outputs. |
|
|
|
|
|
<div align="center" style="line-height: 1;"> |
|
|
<a href="https://github.com/FotographerAI/ZenCtrl/tree/main" target="_blank" style="margin: 2px;" name="github_repo_link"><img src="https://img.shields.io/badge/GitHub-Repo-181717.svg" alt="GitHub Repo" style="display: inline-block; vertical-align: middle;"></a> |
|
|
<a href="https://huggingface.co/spaces/fotographerai/ZenCtrl" target="_blank" name="huggingface_space_link"><img src="https://img.shields.io/badge/🤗_HuggingFace-Space-ffbd45.svg" alt="HuggingFace Space" style="display: inline-block; vertical-align: middle;"></a> |
|
|
<a href="https://discord.com/invite/b9RuYQ3F8k" target="_blank" style="margin: 2px;" name="discord_link"><img src="https://img.shields.io/badge/Discord-Join-7289da.svg?logo=discord" alt="Discord" style="display: inline-block; vertical-align: middle;"></a> |
|
|
<a href="https://fotographer.ai/zen-control" target="_blank" style="margin: 2px;" name="lp_link"><img src="https://img.shields.io/badge/Website-Landing_Page-blue" alt="LP" style="display: inline-block; vertical-align: middle;"></a> |
|
|
<a href="https://x.com/FotographerAI" target="_blank" style="margin: 2px;" name="twitter_link"><img src="https://img.shields.io/twitter/follow/FotographerAI?style=social" alt="X" style="display: inline-block; vertical-align: middle;"></a> |
|
|
</div> |
|
|
|
|
|
Here are some randomly picked weights we trained on image-conditionned text to image generation for spatially aligned and non-aligned tasks. |
|
|
|
|
|
We are training multiple tasks in various conditions and settings. We will share more details soon on the weights and how to run them, but these weights were finetuned with versions of OminiControl Pipelines. |
|
|
Most, if not all should work with OminiControl so you can add them to your project and load them as adapters. |
|
|
the adapter name is the string in the folder name after "-". |
|
|
|
|
|
## 📦 Github page |
|
|
|
|
|
https://github.com/FotographerAI/ZenCtrl/tree/main |
|
|
|
|
|
<div style="display: grid; grid-template-columns: repeat(4, 1fr); grid-template-rows: repeat(2, 1fr); grid-column-gap: 1em; grid-row-gap: 1em;"> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_1.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_1.webp" type="image/webp" /> |
|
|
<img alt="bottle on top of a rock" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_1.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_2.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_2.webp" type="image/webp" /> |
|
|
<img alt="bottle on top of a rock" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/bottle_2.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_1.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_1.webp" type="image/webp" /> |
|
|
<img alt="speaker" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_1.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_2.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_2.webp" type="image/webp" /> |
|
|
<img alt="speaker" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/speaker_2.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_1.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_1.webp" type="image/webp" /> |
|
|
<img alt="chair" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_1.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_2.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_2.webp" type="image/webp" /> |
|
|
<img alt="chair" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/chair_2.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_1.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_1.webp" type="image/webp" /> |
|
|
<img alt="handcream" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_1.png"/> |
|
|
</picture> |
|
|
<picture> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_2.avif" type="image/avif" /> |
|
|
<source srcset="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_2.webp" type="image/webp" /> |
|
|
<img alt="handcream" src="https://storage.googleapis.com/fotographer-cdn/app-static-assets/zen_ctrl/handcream_2.png"/> |
|
|
</picture> |
|
|
</div> |
|
|
|
|
|
--- |
|
|
|
|
|
Here are the Controls, Tasks and Categories we are taining for and we plan to make them all open source, including the video models. |
|
|
|
|
|
Controls: |
|
|
|
|
|
Preprocessing |
|
|
- bg remove, matting, reshaping, segmentation… |
|
|
Control Models |
|
|
- Shape (canny, Hed, scribble, depth..), Pose, Mask, Camera view |
|
|
Post-Processing models |
|
|
- Enhancement, Color fixing, blending |
|
|
Editing Models |
|
|
- in painting (including removal, masked blending, replacing etc…, adding), Outpainting, Motion/Transformation , Relighting |
|
|
|
|
|
Tasks: |
|
|
|
|
|
Background generation |
|
|
Controlled Background Generation |
|
|
Subject consistent in context Generation |
|
|
Object placement |
|
|
Video Generation |
|
|
In context Image/Video Generation |
|
|
multi-object/subject Merging/Blending |
|
|
|
|
|
Categories: |
|
|
|
|
|
Product photography |
|
|
Fashion Accessory / Shoes, Hats etc…. fitting |
|
|
Virtual Try-on |
|
|
People |
