import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("schrum2/MarioDiffusion-MiniLM-single-negative0", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]Details on the code used to produce and use this model are available at:
https://github.com/schrum2/MarioDiffusion
That repo has instructions to check out this model and apply it to the generation of Super Mario Bros. level scenes. There is also an interactive GUI for constructing complete levels out of model-generated scenes.
This model makes use of https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1 as a text embedding model for use with diffusion to generate Mario levels. It also makes use of negative guidance during diffusion training.
To see a model using sentence-transformers/multi-qa-MiniLM-L6-cos-v1 without negative guidance, see https://huggingface.co/schrum2/MarioDiffusion-MiniLM-single-regular0. To see a model that uses negative guidance with a simple token-based transformer model for text embedding, see https://huggingface.co/schrum2/MarioDiffusion-MLM-negative0.
- Downloads last month
- 3