Update README.md
Browse filesDocument the free model-first ZeroGPU pipeline
README.md
CHANGED
|
@@ -31,7 +31,7 @@ If a requested role has no deterministic hook, assets are still generated for re
|
|
| 31 |
|
| 32 |
Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
|
| 33 |
|
| 34 |
-
The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset
|
| 35 |
|
| 36 |
Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
|
| 37 |
|
|
@@ -39,17 +39,39 @@ The app combines the game type, perspective, theme, and pasted code into explici
|
|
| 39 |
|
| 40 |
If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
|
| 41 |
|
| 42 |
-
##
|
| 43 |
|
| 44 |
-
The
|
| 45 |
|
| 46 |
```text
|
| 47 |
-
|
| 48 |
```
|
| 49 |
|
| 50 |
-
|
| 51 |
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
```text
|
| 55 |
USE_CONTROLNET_FOR_BACKGROUNDS=1
|
|
@@ -58,7 +80,7 @@ CONTROLNET_MODEL=diffusers/controlnet-depth-sdxl-1.0-small
|
|
| 58 |
CONTROLNET_STEPS=24
|
| 59 |
```
|
| 60 |
|
| 61 |
-
This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet
|
| 62 |
|
| 63 |
For an API-backed refinement that works from a CPU Space, enable:
|
| 64 |
|
|
@@ -69,10 +91,10 @@ HF_STRUCTURE_PROVIDER=fal-ai
|
|
| 69 |
HF_TOKEN=<Space secret with Inference Providers permission>
|
| 70 |
```
|
| 71 |
|
| 72 |
-
This sends the deterministic background as an image-to-image layout reference
|
| 73 |
|
| 74 |
The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
|
| 75 |
|
| 76 |
-
|
| 77 |
|
| 78 |
`black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.
|
|
|
|
| 31 |
|
| 32 |
Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
|
| 33 |
|
| 34 |
+
The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset carries a normalized camera contract (`top_down`, `side_view`, `isometric`, `first_person`, or `front_view`). Top-down background prompts explicitly forbid a sky, horizon, eye-level view, and front-facing facades.
|
| 35 |
|
| 36 |
Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
|
| 37 |
|
|
|
|
| 39 |
|
| 40 |
If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
|
| 41 |
|
| 42 |
+
## Default neural image pipeline
|
| 43 |
|
| 44 |
+
The deployed Space is designed for Hugging Face ZeroGPU and uses a real diffusion model for both sprites and backgrounds:
|
| 45 |
|
| 46 |
```text
|
| 47 |
+
segmind/SSD-1B + latent-consistency/lcm-lora-ssd-1b
|
| 48 |
```
|
| 49 |
|
| 50 |
+
SSD-1B is an Apache-2.0 distilled SDXL model. The LCM adapter reduces generation to four denoising steps. The weights run inside the Space's ZeroGPU allocation, so the app does not consume Hugging Face Inference Provider credits and does not require a paid image API.
|
| 51 |
|
| 52 |
+
Sprites use text-to-image generation followed by transparency extraction. Backgrounds use image-to-image generation from a deterministic, perspective-specific layout guide. This gives the model explicit top-down, side-view, or isometric structure instead of relying on prompt wording alone. Background negative prompts reject characters and generic particle overlays; the procedural lava guide no longer contains the circular effects that previously appeared as repeated particles.
|
| 53 |
+
|
| 54 |
+
The deployed Space is fail-closed: if the primary neural model cannot run, generation stops with a visible error instead of silently returning procedural art and calling it model output. The procedural renderer remains available for local development and emergency diagnostics only.
|
| 55 |
+
|
| 56 |
+
ZeroGPU is free for eligible personal accounts, but it is quota-limited rather than unlimited: free accounts currently receive five minutes of GPU time per day. Queueing or a quota message is therefore possible even though no inference credits or payment are required.
|
| 57 |
+
|
| 58 |
+
The primary pipeline can be configured with:
|
| 59 |
+
|
| 60 |
+
```text
|
| 61 |
+
USE_PRIMARY_IMAGE_MODEL=1
|
| 62 |
+
REQUIRE_PRIMARY_IMAGE_MODEL=1
|
| 63 |
+
PRIMARY_IMAGE_MODEL=segmind/SSD-1B
|
| 64 |
+
PRIMARY_LORA_MODEL=latent-consistency/lcm-lora-ssd-1b
|
| 65 |
+
PRIMARY_IMAGE_STEPS=4
|
| 66 |
+
PRIMARY_GUIDANCE_SCALE=1.5
|
| 67 |
+
PRIMARY_BACKGROUND_STRENGTH=0.72
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
`HF_TOKEN` is not required to download these public image models. It remains useful for the optional prompt-model provider and for the UI's token-validity check.
|
| 71 |
+
|
| 72 |
+
## Optional generation tiers
|
| 73 |
+
|
| 74 |
+
For a larger dedicated GPU Space, the SDXL Depth ControlNet path remains available:
|
| 75 |
|
| 76 |
```text
|
| 77 |
USE_CONTROLNET_FOR_BACKGROUNDS=1
|
|
|
|
| 80 |
CONTROLNET_STEPS=24
|
| 81 |
```
|
| 82 |
|
| 83 |
+
This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet after the primary model path. It intentionally refuses to run on CPU-only hardware. The small ControlNet adapter is experimental, so perspective still needs human review.
|
| 84 |
|
| 85 |
For an API-backed refinement that works from a CPU Space, enable:
|
| 86 |
|
|
|
|
| 91 |
HF_TOKEN=<Space secret with Inference Providers permission>
|
| 92 |
```
|
| 93 |
|
| 94 |
+
This sends the deterministic background as an image-to-image layout reference and asks the model to preserve its projection and walkable geometry. `FLUX.2-klein-4B` is available through a Hugging Face Inference Provider. This option is disabled by default because provider calls consume account inference credits; it is not needed for the free ZeroGPU pipeline.
|
| 95 |
|
| 96 |
The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
|
| 97 |
|
| 98 |
+
When running locally with `REQUIRE_PRIMARY_IMAGE_MODEL=0`, model failure may fall back to the style-aware Pillow renderer for development. The public Space should keep the requirement enabled so its generated images are provably model-backed.
|
| 99 |
|
| 100 |
`black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.
|