Update README.md
Browse files
README.md
CHANGED
|
@@ -26,6 +26,8 @@ tags:
|
|
| 26 |
|
| 27 |
# Rico Diffusion Model Card
|
| 28 |
|
|
|
|
|
|
|
| 29 |
I fine-tuned a Stable Diffusion 1.5 model to generate mobile UI mockups at 384x640 with GLIGEN (https://gligen.github.io) to control UI component positions. However, there are designs generated at 448x576, primarily modal dialogs.
|
| 30 |
|
| 31 |
I used EveryDream2 (https://github.com/victorchall/EveryDream2trainer) to fine-tune the model on the Rico Dataset (http://www.interactionmining.org/rico.html) of UI Screenshots where I wrote a Python notebook to parse over the Semantic Hierarchies of the dataset to create the captions for each screenshot as well as using the Play Store and UI Metadata to use the app categories as extra tags. I have also cropped each UI component of a given screenshot (with exceptions) and labeled them accordingly so that I can train the model on individual UI components first before going for the whole screenshot. I also used BLIP-2 (https://huggingface.co/Salesforce/blip2-opt-2.7b-coco) to add color names to the UI components in the caption as well as general labelling for certain components.
|
|
|
|
| 26 |
|
| 27 |
# Rico Diffusion Model Card
|
| 28 |
|
| 29 |
+
|
| 30 |
+
|
| 31 |
I fine-tuned a Stable Diffusion 1.5 model to generate mobile UI mockups at 384x640 with GLIGEN (https://gligen.github.io) to control UI component positions. However, there are designs generated at 448x576, primarily modal dialogs.
|
| 32 |
|
| 33 |
I used EveryDream2 (https://github.com/victorchall/EveryDream2trainer) to fine-tune the model on the Rico Dataset (http://www.interactionmining.org/rico.html) of UI Screenshots where I wrote a Python notebook to parse over the Semantic Hierarchies of the dataset to create the captions for each screenshot as well as using the Play Store and UI Metadata to use the app categories as extra tags. I have also cropped each UI component of a given screenshot (with exceptions) and labeled them accordingly so that I can train the model on individual UI components first before going for the whole screenshot. I also used BLIP-2 (https://huggingface.co/Salesforce/blip2-opt-2.7b-coco) to add color names to the UI components in the caption as well as general labelling for certain components.
|