LeafCat79 commited on
Commit
8be5bab
·
verified ·
1 Parent(s): 84d05ed

Update README.md

Browse files

Document the free model-first ZeroGPU pipeline

Files changed (1) hide show
  1. README.md +31 -9
README.md CHANGED
@@ -31,7 +31,7 @@ If a requested role has no deterministic hook, assets are still generated for re
31
 
32
  Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
33
 
34
- The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset now carries a normalized camera contract (`top_down`, `side_view`, `isometric`, `first_person`, or `front_view`). Top-down background prompts explicitly forbid a sky, horizon, eye-level view, and front-facing facades. The free procedural renderer also routes top-down and isometric backgrounds through dedicated geometry, so camera compliance does not depend only on prompt wording.
35
 
36
  Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
37
 
@@ -39,17 +39,39 @@ The app combines the game type, perspective, theme, and pasted code into explici
39
 
40
  If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
41
 
42
- ## Background generation tiers
43
 
44
- The default tier is deliberately free and deterministic:
45
 
46
  ```text
47
- camera-aware procedural background
48
  ```
49
 
50
- It runs on CPU Basic and is the active production-safe fallback. It is deterministic drawing code, not a neural image-generation model. The UI reports this explicitly for every role. It fixes camera geometry, but it cannot match the semantic richness of a large image model.
51
 
52
- For a GPU Space, enable the production depth-control path:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
54
  ```text
55
  USE_CONTROLNET_FOR_BACKGROUNDS=1
@@ -58,7 +80,7 @@ CONTROLNET_MODEL=diffusers/controlnet-depth-sdxl-1.0-small
58
  CONTROLNET_STEPS=24
59
  ```
60
 
61
- This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet before falling back. It intentionally refuses to run on CPU-only hardware: loading SDXL on CPU Basic is not a credible interactive production configuration. The small ControlNet adapter is experimental, so perspective still needs human review.
62
 
63
  For an API-backed refinement that works from a CPU Space, enable:
64
 
@@ -69,10 +91,10 @@ HF_STRUCTURE_PROVIDER=fal-ai
69
  HF_TOKEN=<Space secret with Inference Providers permission>
70
  ```
71
 
72
- This sends the deterministic background as an image-to-image layout reference, asks the model to preserve its projection and walkable geometry, and falls back safely if the request fails. `FLUX.2-klein-4B` is Apache-2.0 licensed and available through a Hugging Face Inference Provider. It is not unlimited-free infrastructure: a free Hugging Face user currently receives only a small monthly inference credit, after which purchased credits are required. Keep the flag disabled unless paid API usage and token handling are acceptable.
73
 
74
  The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
75
 
76
- If every enabled model path fails, the app uses the local style-aware Pillow fallback, which varies watercolor, vector/neon, claymation, oil-painting, anime, pixel-art looks, and common game sprite silhouettes.
77
 
78
  `black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.
 
31
 
32
  Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
33
 
34
+ The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset carries a normalized camera contract (`top_down`, `side_view`, `isometric`, `first_person`, or `front_view`). Top-down background prompts explicitly forbid a sky, horizon, eye-level view, and front-facing facades.
35
 
36
  Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
37
 
 
39
 
40
  If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
41
 
42
+ ## Default neural image pipeline
43
 
44
+ The deployed Space is designed for Hugging Face ZeroGPU and uses a real diffusion model for both sprites and backgrounds:
45
 
46
  ```text
47
+ segmind/SSD-1B + latent-consistency/lcm-lora-ssd-1b
48
  ```
49
 
50
+ SSD-1B is an Apache-2.0 distilled SDXL model. The LCM adapter reduces generation to four denoising steps. The weights run inside the Space's ZeroGPU allocation, so the app does not consume Hugging Face Inference Provider credits and does not require a paid image API.
51
 
52
+ Sprites use text-to-image generation followed by transparency extraction. Backgrounds use image-to-image generation from a deterministic, perspective-specific layout guide. This gives the model explicit top-down, side-view, or isometric structure instead of relying on prompt wording alone. Background negative prompts reject characters and generic particle overlays; the procedural lava guide no longer contains the circular effects that previously appeared as repeated particles.
53
+
54
+ The deployed Space is fail-closed: if the primary neural model cannot run, generation stops with a visible error instead of silently returning procedural art and calling it model output. The procedural renderer remains available for local development and emergency diagnostics only.
55
+
56
+ ZeroGPU is free for eligible personal accounts, but it is quota-limited rather than unlimited: free accounts currently receive five minutes of GPU time per day. Queueing or a quota message is therefore possible even though no inference credits or payment are required.
57
+
58
+ The primary pipeline can be configured with:
59
+
60
+ ```text
61
+ USE_PRIMARY_IMAGE_MODEL=1
62
+ REQUIRE_PRIMARY_IMAGE_MODEL=1
63
+ PRIMARY_IMAGE_MODEL=segmind/SSD-1B
64
+ PRIMARY_LORA_MODEL=latent-consistency/lcm-lora-ssd-1b
65
+ PRIMARY_IMAGE_STEPS=4
66
+ PRIMARY_GUIDANCE_SCALE=1.5
67
+ PRIMARY_BACKGROUND_STRENGTH=0.72
68
+ ```
69
+
70
+ `HF_TOKEN` is not required to download these public image models. It remains useful for the optional prompt-model provider and for the UI's token-validity check.
71
+
72
+ ## Optional generation tiers
73
+
74
+ For a larger dedicated GPU Space, the SDXL Depth ControlNet path remains available:
75
 
76
  ```text
77
  USE_CONTROLNET_FOR_BACKGROUNDS=1
 
80
  CONTROLNET_STEPS=24
81
  ```
82
 
83
+ This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet after the primary model path. It intentionally refuses to run on CPU-only hardware. The small ControlNet adapter is experimental, so perspective still needs human review.
84
 
85
  For an API-backed refinement that works from a CPU Space, enable:
86
 
 
91
  HF_TOKEN=<Space secret with Inference Providers permission>
92
  ```
93
 
94
+ This sends the deterministic background as an image-to-image layout reference and asks the model to preserve its projection and walkable geometry. `FLUX.2-klein-4B` is available through a Hugging Face Inference Provider. This option is disabled by default because provider calls consume account inference credits; it is not needed for the free ZeroGPU pipeline.
95
 
96
  The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
97
 
98
+ When running locally with `REQUIRE_PRIMARY_IMAGE_MODEL=0`, model failure may fall back to the style-aware Pillow renderer for development. The public Space should keep the requirement enabled so its generated images are provably model-backed.
99
 
100
  `black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.