Tan commited on
Commit
12d4366
Β·
verified Β·
1 Parent(s): 1c2a997

Add comprehensive model card

Browse files
Files changed (1) hide show
  1. README.md +183 -3
README.md CHANGED
@@ -1,3 +1,183 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: flux-1-dev-non-commercial-license
4
+ license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
5
+ language:
6
+ - en
7
+ tags:
8
+ - flux
9
+ - diffusers
10
+ - lora
11
+ - kontext
12
+ - web-design
13
+ - ui-ux
14
+ - design-transformation
15
+ - instruction-following
16
+ base_model: black-forest-labs/FLUX.1-dev
17
+ pipeline_tag: text-to-image
18
+ ---
19
+
20
+ # InstructDesign-Kontext
21
+
22
+ Fine-tuned FLUX.1 Kontext [dev] model for transforming web interfaces with natural language instructions.
23
+
24
+ ## Model Description
25
+
26
+ InstructDesign-Kontext is a LoRA fine-tuned version of FLUX.1 Kontext [dev] specialized in:
27
+ - Web interface transformations
28
+ - UI/UX redesigns
29
+ - Style transfers (dark mode, glassmorphism, etc.)
30
+ - Device mockup generation
31
+ - Design system applications
32
+
33
+ ### Training Details
34
+
35
+ - **Base Model**: FLUX.1 Kontext [dev]
36
+ - **Training Method**: LoRA fine-tuning
37
+ - **Dataset**: 937 web interface transformation pairs
38
+ - **Training Steps**: 10,000
39
+ - **Training Time**: ~2 weeks (iterative)
40
+
41
+ ## File Structure
42
+
43
+ ```
44
+ β”œβ”€β”€ unet/
45
+ β”‚ └── flux1-kontext-dev.safetensors (23GB)
46
+ β”œβ”€β”€ loras/
47
+ β”‚ └── flux_kontext_lora_v4_consolidated_000010000.safetensors (2.6GB)
48
+ β”œβ”€β”€ clip/
49
+ β”‚ β”œβ”€β”€ clip_l.safetensors (235MB)
50
+ β”‚ └── t5xxl_fp8_e4m3fn_scaled.safetensors (4.9GB)
51
+ β”œβ”€β”€ vae/
52
+ β”‚ └── ae.safetensors (320MB)
53
+ └── workflow/
54
+ └── workflow_api.json
55
+ ```
56
+
57
+ ## Usage with ComfyUI
58
+
59
+ ### 1. Download Models
60
+
61
+ Download all model files to your ComfyUI models directory:
62
+
63
+ ```bash
64
+ # Main model
65
+ ComfyUI/models/diffusion_models/flux1-kontext-dev.safetensors
66
+
67
+ # LoRA
68
+ ComfyUI/models/loras/flux_kontext_lora_v4_consolidated_000010000.safetensors
69
+
70
+ # CLIP encoders
71
+ ComfyUI/models/clip/clip_l.safetensors
72
+ ComfyUI/models/clip/t5xxl_fp8_e4m3fn_scaled.safetensors
73
+
74
+ # VAE
75
+ ComfyUI/models/vae/ae.safetensors
76
+ ```
77
+
78
+ ### 2. Load Workflow
79
+
80
+ Import the `workflow/workflow_api.json` into ComfyUI.
81
+
82
+ ### 3. Configure Parameters
83
+
84
+ Default optimal settings:
85
+ - **Steps**: 20
86
+ - **CFG**: 1.0
87
+ - **Guidance**: 5.0
88
+ - **Sampler**: er_sde
89
+ - **Scheduler**: kl_optimal
90
+ - **LoRA Strength**: 1.0
91
+
92
+ ## API Usage
93
+
94
+ See the [GitHub repository](https://github.com/tercumantanumut/instructdesign-kontext) for full API implementation.
95
+
96
+ ```python
97
+ # Example API request
98
+ import requests
99
+
100
+ response = requests.post("http://localhost:8000/api/generate", json={
101
+ "positive_prompt": "Transform this interface to dark mode with purple accents",
102
+ "input_image": "website.png",
103
+ "seed": 42,
104
+ "steps": 20,
105
+ "cfg": 1.0,
106
+ "guidance": 5.0
107
+ })
108
+ ```
109
+
110
+ ## Transformation Capabilities
111
+
112
+ ### Style Transformations
113
+ - Dark/light mode conversion
114
+ - Glassmorphism effects
115
+ - Neumorphic design
116
+ - Material Design application
117
+ - iOS/Android style transfer
118
+
119
+ ### UI/UX Improvements
120
+ - Layout modernization
121
+ - Accessibility enhancements
122
+ - Color scheme updates
123
+ - Typography improvements
124
+ - Interactive element additions
125
+
126
+ ### Mockup Generation
127
+ - Device frame placement
128
+ - Environment contextualization
129
+ - Billboard/poster mockups
130
+ - Multi-device presentations
131
+
132
+ ## Example Prompts
133
+
134
+ ```
135
+ "Transform this website to dark mode with purple gradient accents"
136
+ "Apply glassmorphism effect with blurred backgrounds"
137
+ "Convert to material design 3 with rounded corners"
138
+ "Place this interface on an iPhone 15 Pro in a coffee shop"
139
+ "Modernize this dated interface with current design trends"
140
+ ```
141
+
142
+ ## Performance
143
+
144
+ - **Inference Time**: ~45-70 seconds (NVIDIA GPU 16GB+)
145
+ - **VRAM Requirements**: 16GB minimum, 24GB recommended
146
+ - **Quality**: 85%+ instruction adherence rate
147
+
148
+ ## Docker Deployment
149
+
150
+ Complete Docker setup available in the GitHub repository with:
151
+ - ComfyUI container
152
+ - FastAPI wrapper
153
+ - Queue management
154
+ - Auto-scaling workers
155
+
156
+ ## Limitations
157
+
158
+ - Requires significant VRAM (16GB+)
159
+ - Best results with clear, specific instructions
160
+ - Limited to single image transformations (no batch in single inference)
161
+
162
+ ## Citation
163
+
164
+ If you use this model, please cite:
165
+
166
+ ```bibtex
167
+ @misc{instructdesign-kontext,
168
+ title={InstructDesign-Kontext: Web Interface Transformation with FLUX},
169
+ author={tercumantanumut},
170
+ year={2024},
171
+ publisher={Hugging Face}
172
+ }
173
+ ```
174
+
175
+ ## License
176
+
177
+ This model is subject to the FLUX.1 [dev] Non-Commercial License.
178
+
179
+ ## Acknowledgments
180
+
181
+ - FLUX.1 team for the base model
182
+ - ComfyUI for the inference framework
183
+ - 10 weeks of iterative development