update

README.md

@@ -1,5 +1,5 @@
 ---
-title: GT4SD - Polymer Blocks
+title: GT4SD - Diffusers (image)
 emoji: 💡
 colorFrom: green
 colorTo: blue

app.py

@@ -2,30 +2,32 @@ import logging
 import pathlib
 import gradio as gr
 import pandas as pd
-from gt4sd.algorithms.generation.polymer_blocks import (
-    PolymerBlocksGenerator,
-    PolymerBlocks,
+from gt4sd.algorithms.generation.diffusion import (
+    DiffusersGenerationAlgorithm,
+    DDPMGenerator,
+    DDIMGenerator,
+    ScoreSdeGenerator,
+    LDMTextToImageGenerator,
+    LDMGenerator,
+    StableDiffusionGenerator,
 )
-
 from gt4sd.algorithms.registry import ApplicationsRegistry
 
-from utils import draw_grid_generate
-
 logger = logging.getLogger(__name__)
 logger.addHandler(logging.NullHandler())
 
 
-def run_inference(algorithm_version: str, length: float, number_of_samples: int):
+def run_inference(model_type: str, prompt: str):
 
-    config = PolymerBlocksGenerator(
-        algorithm_version=algorithm_version,
-        batch_size=32,
-        generated_length=length,
-    )
-    model = PolymerBlocks(config)
-    samples = list(model.sample(number_of_samples))
+    config = eval(f"{model_type}(prompt={prompt})")
+    if config.modality != "token2image" and prompt != "":
+        raise ValueError(
+            f"{model_type} is an unconditional generative model, please remove prompt (not={prompt})"
+        )
+    model = DiffusersGenerationAlgorithm(config)
+    image = list(model.sample(1))[0]
 
-    return draw_grid_generate(samples=samples, n_cols=5, seeds=[])
+    return image
 
 
 if __name__ == "__main__":
@@ -33,11 +35,10 @@ if __name__ == "__main__":
     # Preparation (retrieve all available algorithms)
     all_algos = ApplicationsRegistry.list_available()
     algos = [
-        x["algorithm_version"]
-        for x in list(
-            filter(lambda x: "PolymerBlocks" in x["algorithm_name"], all_algos)
-        )
+        x["algorithm_application"]
+        for x in list(filter(lambda x: "Diff" in x["algorithm_name"], all_algos))
     ]
+    algos = [a for a in algos if not "GeoDiff" in a]
 
     # Load metadata
     metadata_root = pathlib.Path(__file__).parent.joinpath("model_cards")
@@ -53,21 +54,14 @@ if __name__ == "__main__":
 
     demo = gr.Interface(
         fn=run_inference,
-        title="Polymer Blocks",
+        title="Diffusion-based image generators",
         inputs=[
-            gr.Dropdown(algos, label="Algorithm version", value="v0"),
-            gr.Slider(
-                minimum=5,
-                maximum=400,
-                value=100,
-                label="Maximal sequence length",
-                step=1,
-            ),
-            gr.Slider(
-                minimum=1, maximum=50, value=10, label="Number of samples", step=1
+            gr.Dropdown(
+                algos, label="Diffusion model", value="StableDiffusionGenerator"
             ),
+            gr.Textbox(label="Text prompt", placeholder="A blue tree", lines=1),
         ],
-        outputs=gr.HTML(label="Output"),
+        outputs=gr.outputs.Image(type="pil"),
         article=article,
         description=description,
         examples=examples.values.tolist(),

model_cards/article.md

@@ -1,40 +1,45 @@
 # Model documentation & parameters
 
-**Algorithm Version**: Which model version to use.
+**Diffusion model**: Which model version to use. 
 
-**Maximal sequence length**: The maximal number of SMILES tokens in the generated molecule.
+**Prompt**: The text prompt used, only applies to *conditional* diffusion image generators. These are `LDMTextToImageGenerator` and `StableDiffusionGenerator`. The other four models (`DDPMGenerator`, `DDPMGenerator`, `LDMGenerator` and `ScoreSdeGenerator`) are *unconditional*.
 
-**Number of samples**: How many samples should be generated (between 1 and 50).
 
 
+# Model card -- Image diffusion models
 
-# Model card -- PolymerBlocks
+**Model Details**: Six diffusion models for image generation:
+- `LDMTextToImageGenerator`
+- `StableDiffusionGenerator`
+- `DDPMGenerator`
+- `DDPMGenerator`
+- `LDMGenerator`
+- `ScoreSdeGenerator`
+For details, see the [Diffusers docs](https://huggingface.co/docs/diffusers/index)
 
-**Model Details**: *PolymerBlocks* is a sequence-based molecular generator tuned to generate blocks of polymers (e.g., catalysts and monomers). The model relies on a Variational Autoencoder architecture as described in [Born et al. (2021; *iScience*)](https://www.sciencedirect.com/science/article/pii/S2589004221002376)
+**Developers**: Various developers of above models, wrapped by Diffusers developers into [`diffusers`](https://github.com/huggingface/diffusers)
 
-**Developers**: Matteo Manica and colleagues from IBM Research.
+**Distributors**: Diffusers code integrated into GT4SD.
 
-**Distributors**: Original authors' code integrated into GT4SD.
+**Model date**: 2022.
 
-**Model date**: Not yet published.
+**Model version**: Diffusion models, checkpoints provided and distributed by [`diffusers`](https://github.com/huggingface/diffusers).
 
-**Model version**: Only initial model version.
-
-**Model type**: A sequence-based molecular generator tuned to generate blocks of polymers (e.g., catalysts and monomers).
+**Model type**: Various, see [`diffusers`](https://github.com/huggingface/diffusers) docs.
 
 **Information about training algorithms, parameters, fairness constraints or other applied approaches, and features**: 
 N.A.
 
 **Paper or other resource for more information**: 
-TBD
+N.A.
 
 **License**: MIT
 
-**Where to send questions or comments about the model**: Open an issue on [GT4SD repository](https://github.com/GT4SD/gt4sd-core).
+**Where to send questions or comments about the model**: Open an issue on [`diffusers`](https://github.com/huggingface/diffusers) repo.
 
-**Intended Use. Use cases that were envisioned during development**: Chemical research, in particular drug discovery.
+**Intended Use. Use cases that were envisioned during development**: Computer vision researchers experimenting with image generative models.
 
-**Primary intended uses/users**: Researchers and computational chemists using the model for model comparison or research exploration purposes.
+**Primary intended uses/users**: Computer vision researchers 
 
 **Out-of-scope use cases**: Production-level inference, producing molecules with harmful properties.
 
@@ -51,10 +56,12 @@ Model card prototype inspired by [Mitchell et al. (2019)](https://dl.acm.org/doi
 ## Citation
 TBD, temporarily please cite:
 ```bib
-@article{manica2022gt4sd,
-  title={GT4SD: Generative Toolkit for Scientific Discovery},
-  author={Manica, Matteo and Cadow, Joris and Christofidellis, Dimitrios and Dave, Ashish and Born, Jannis and Clarke, Dean and Teukam, Yves Gaetan Nana and Hoffman, Samuel C and Buchan, Matthew and Chenthamarakshan, Vijil and others},
-  journal={arXiv preprint arXiv:2207.03928},
-  year={2022}
+@misc{von-platen-etal-2022-diffusers,
+  author = {Patrick von Platen and Suraj Patil and Anton Lozhkov and Pedro Cuenca and Nathan Lambert and Kashif Rasul and Mishig Davaadorj and Thomas Wolf},
+  title = {Diffusers: State-of-the-art diffusion models},
+  year = {2022},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/huggingface/diffusers}}
 }
 ```

model_cards/description.md

@@ -1,6 +1,6 @@
 <img align="right" src="https://raw.githubusercontent.com/GT4SD/gt4sd-core/main/docs/_static/gt4sd_logo.png" alt="logo" width="120" >
 
-*PolymerBlocks* is a sequence-based molecular generator tuned to generate blocks of polymers (e.g., catalysts and monomers). The model relies on a Variational Autoencoder architecture as described in [Born et al. (2021; *iScience*)](https://www.sciencedirect.com/science/article/pii/S2589004221002376)
+This UI provides access to various diffusion-based image generators implemented in the [`diffusers`](https://github.com/huggingface/diffusers) library, wrapped and re-distributed by GT4SD. 
 
 For **examples** and **documentation** of the model parameters, please see below.
 Moreover, we provide a **model card** ([Mitchell et al. (2019)](https://dl.acm.org/doi/abs/10.1145/3287560.3287596?casa_token=XD4eHiE2cRUAAAAA:NL11gMa1hGPOUKTAbtXnbVQBDBbjxwcjGECF_i-WC_3g1aBgU1Hbz_f2b4kI_m1in-w__1ztGeHnwHs)) at the bottom of this page.

model_cards/examples.csv

@@ -1 +1,2 @@
-v0,100,10
+LDMGenerator,
+LDMTextToImageGenerator,Generative models on the moon