Spaces:
Running on Zero
Running on Zero
Upgrade generation to full-quality SDXL
Browse files
README.md
CHANGED
|
@@ -84,15 +84,15 @@ If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first u
|
|
| 84 |
The deployed Space is designed for Hugging Face ZeroGPU and uses real neural models for generation and transparent cutout extraction:
|
| 85 |
|
| 86 |
```text
|
| 87 |
-
|
| 88 |
ZhengPeng7/BiRefNet (transparent-cutout alpha only)
|
| 89 |
```
|
| 90 |
|
| 91 |
-
|
| 92 |
|
| 93 |
-
Sprites and backgrounds are generated directly from their written prompts. No procedurally drawn sprite, palette, silhouette, camera guide, or background layout is passed into the production image generator.
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
The deployed Space is fail-closed: if the primary neural model cannot run, generation stops with a visible error instead of silently returning procedural art and calling it model output. The procedural renderer remains available for local development and diagnostics only and cannot influence successful public generations.
|
| 98 |
|
|
@@ -105,16 +105,19 @@ The primary pipeline can be configured with:
|
|
| 105 |
```text
|
| 106 |
USE_PRIMARY_IMAGE_MODEL=1
|
| 107 |
REQUIRE_PRIMARY_IMAGE_MODEL=1
|
| 108 |
-
PRIMARY_IMAGE_MODEL=
|
| 109 |
-
|
|
|
|
| 110 |
FOREGROUND_MODEL_ID=ZhengPeng7/BiRefNet
|
| 111 |
FOREGROUND_MODEL_REVISION=e2bf8e4460fc8fa32bba5ea4d94b3233d367b0e4
|
| 112 |
-
PRIMARY_IMAGE_STEPS=
|
| 113 |
-
PRIMARY_SPRITE_STEPS=
|
| 114 |
PRIMARY_SPRITE_ATTEMPTS=3
|
| 115 |
-
PRIMARY_GUIDANCE_SCALE=
|
| 116 |
```
|
| 117 |
|
|
|
|
|
|
|
| 118 |
`HF_TOKEN` is not required to download these public image models. It remains useful for the optional prompt-model provider and for the UI's token-validity check.
|
| 119 |
|
| 120 |
## Optional generation tiers
|
|
|
|
| 84 |
The deployed Space is designed for Hugging Face ZeroGPU and uses real neural models for generation and transparent cutout extraction:
|
| 85 |
|
| 86 |
```text
|
| 87 |
+
stabilityai/stable-diffusion-xl-base-1.0 (native full-step inference)
|
| 88 |
ZhengPeng7/BiRefNet (transparent-cutout alpha only)
|
| 89 |
```
|
| 90 |
|
| 91 |
+
SDXL Base is a 3B-parameter Open RAIL++ text-to-image model. Production generation intentionally runs without an LCM acceleration adapter: backgrounds use 30 denoising steps, sprites use 32, and classifier-free guidance is 8.0. This is slower than the earlier SSD-1B + LCM configuration, but it preserves the base model's full denoising process, makes negative prompts effective, and gives strict perspective and composition instructions substantially more influence. The weights run inside the Space's free ZeroGPU allocation, so the app does not consume Hugging Face Inference Provider credits and does not require a paid image API.
|
| 92 |
|
| 93 |
+
Sprites and backgrounds are generated directly from their written prompts. No procedurally drawn sprite, palette, silhouette, camera guide, or background layout is passed into the production image generator. Strict top-down sprites use a native 1024×1024 canvas, an orthographic 90-degree overhead instruction at the front of the positive prompt, and an effective negative prompt that rejects front, oblique, three-quarter, eye-level, and side-elevation views. Other perspectives use camera-appropriate native SDXL aspect ratios. BiRefNet predicts only the alpha mask for transparent cutouts while preserving SDXL's generated RGB artwork. After neural transparency extraction, sprites with zero or multiple significant foreground components, retained scene-like alpha, implausible silhouettes, or inadequate scale are regenerated with a new seed up to `PRIMARY_SPRITE_ATTEMPTS` times. If every attempt fails, the deployed Space returns an explicit model-generation error rather than returning a character sheet, environment-contaminated cutout, or obvious upright portrait. Background negative prompts reject characters and generic particle overlays.
|
| 94 |
|
| 95 |
+
SDXL uses two fixed text encoders with limited prompt context. Production prompts therefore retain a conservative word budget, place the camera and single-subject contract first, and compact the free-form description. This prevents critical geometry constraints from being silently truncated behind verbose style text.
|
| 96 |
|
| 97 |
The deployed Space is fail-closed: if the primary neural model cannot run, generation stops with a visible error instead of silently returning procedural art and calling it model output. The procedural renderer remains available for local development and diagnostics only and cannot influence successful public generations.
|
| 98 |
|
|
|
|
| 105 |
```text
|
| 106 |
USE_PRIMARY_IMAGE_MODEL=1
|
| 107 |
REQUIRE_PRIMARY_IMAGE_MODEL=1
|
| 108 |
+
PRIMARY_IMAGE_MODEL=stabilityai/stable-diffusion-xl-base-1.0
|
| 109 |
+
PRIMARY_IMAGE_REVISION=
|
| 110 |
+
PRIMARY_LORA_MODEL=
|
| 111 |
FOREGROUND_MODEL_ID=ZhengPeng7/BiRefNet
|
| 112 |
FOREGROUND_MODEL_REVISION=e2bf8e4460fc8fa32bba5ea4d94b3233d367b0e4
|
| 113 |
+
PRIMARY_IMAGE_STEPS=30
|
| 114 |
+
PRIMARY_SPRITE_STEPS=32
|
| 115 |
PRIMARY_SPRITE_ATTEMPTS=3
|
| 116 |
+
PRIMARY_GUIDANCE_SCALE=8.0
|
| 117 |
```
|
| 118 |
|
| 119 |
+
`PRIMARY_LORA_MODEL` remains an optional compatibility switch for an LCM acceleration adapter. When it is non-empty and the step/guidance variables are not explicitly overridden, the app automatically returns to LCM-safe defaults of 6 background steps, 8 sprite steps, and guidance 1.5. That mode is faster but is no longer the production default because quality and contract adherence take priority.
|
| 120 |
+
|
| 121 |
`HF_TOKEN` is not required to download these public image models. It remains useful for the optional prompt-model provider and for the UI's token-validity check.
|
| 122 |
|
| 123 |
## Optional generation tiers
|
app.py
CHANGED
|
@@ -74,17 +74,22 @@ class IntegrationReport:
|
|
| 74 |
|
| 75 |
|
| 76 |
HF_TOKEN = os.environ.get("HF_TOKEN", "")
|
| 77 |
-
PRIMARY_IMAGE_MODEL = os.environ.get(
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
FOREGROUND_MODEL_ID = os.environ.get("FOREGROUND_MODEL_ID", "ZhengPeng7/BiRefNet")
|
| 80 |
FOREGROUND_MODEL_REVISION = os.environ.get(
|
| 81 |
"FOREGROUND_MODEL_REVISION",
|
| 82 |
"e2bf8e4460fc8fa32bba5ea4d94b3233d367b0e4",
|
| 83 |
)
|
| 84 |
-
PRIMARY_IMAGE_STEPS = int(os.environ.get("PRIMARY_IMAGE_STEPS", "6"))
|
| 85 |
-
PRIMARY_SPRITE_STEPS = int(os.environ.get("PRIMARY_SPRITE_STEPS", "8"))
|
| 86 |
PRIMARY_SPRITE_ATTEMPTS = max(1, int(os.environ.get("PRIMARY_SPRITE_ATTEMPTS", "3")))
|
| 87 |
-
PRIMARY_GUIDANCE_SCALE = float(os.environ.get("PRIMARY_GUIDANCE_SCALE", "1.5"))
|
| 88 |
USE_PRIMARY_IMAGE_MODEL = os.environ.get("USE_PRIMARY_IMAGE_MODEL", "1") == "1"
|
| 89 |
REQUIRE_PRIMARY_IMAGE_MODEL = os.environ.get(
|
| 90 |
"REQUIRE_PRIMARY_IMAGE_MODEL",
|
|
@@ -1779,7 +1784,8 @@ def diffusion_negative_prompt(spec: AssetSpec) -> str:
|
|
| 1779 |
camera_negative = {
|
| 1780 |
"top_down": (
|
| 1781 |
", sky, horizon, eye-level view, front view, front-facing facade, upright portrait, "
|
| 1782 |
-
"standing toward camera, face looking at viewer,
|
|
|
|
| 1783 |
),
|
| 1784 |
"isometric": ", eye-level view, front view, inconsistent perspective, horizon",
|
| 1785 |
"side_view": ", overhead view, top-down view, isometric view",
|
|
@@ -1861,7 +1867,7 @@ def initialize_primary_image_model() -> None:
|
|
| 1861 |
|
| 1862 |
try:
|
| 1863 |
import torch
|
| 1864 |
-
from diffusers import AutoPipelineForText2Image
|
| 1865 |
|
| 1866 |
model_kwargs = {
|
| 1867 |
"torch_dtype": torch.float16,
|
|
@@ -1870,11 +1876,16 @@ def initialize_primary_image_model() -> None:
|
|
| 1870 |
}
|
| 1871 |
if HF_TOKEN:
|
| 1872 |
model_kwargs["token"] = HF_TOKEN
|
|
|
|
|
|
|
| 1873 |
|
| 1874 |
text_pipe = AutoPipelineForText2Image.from_pretrained(PRIMARY_IMAGE_MODEL, **model_kwargs)
|
| 1875 |
-
|
| 1876 |
-
|
| 1877 |
-
|
|
|
|
|
|
|
|
|
|
| 1878 |
text_pipe = text_pipe.to("cuda")
|
| 1879 |
if hasattr(text_pipe, "set_progress_bar_config"):
|
| 1880 |
text_pipe.set_progress_bar_config(disable=True)
|
|
@@ -1947,8 +1958,9 @@ def primary_diffusion_prompt(spec: AssetSpec) -> str:
|
|
| 1947 |
subject = (spec.group or spec.role).replace("_", " ")
|
| 1948 |
if spec.camera == "top_down":
|
| 1949 |
return compact_prompt_words(
|
| 1950 |
-
f"
|
| 1951 |
-
"Show top surfaces and a compact foreshortened silhouette
|
|
|
|
| 1952 |
f"{description}. Exactly one complete isolated subject, centered on uniform white. "
|
| 1953 |
f"{variation}Crisp game-ready details. No scene.",
|
| 1954 |
55,
|
|
@@ -2633,7 +2645,7 @@ def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str,
|
|
| 2633 |
|
| 2634 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2635 |
png_content, error = primary_diffusion_png(spec, index, run_id)
|
| 2636 |
-
source =
|
| 2637 |
if not is_background:
|
| 2638 |
source += f" + {FOREGROUND_MODEL_ID} neural alpha"
|
| 2639 |
if error:
|
|
@@ -2879,14 +2891,23 @@ def model_source_note(image_model: str) -> str:
|
|
| 2879 |
if "via fal-ai" in lowered or "via replicate" in lowered:
|
| 2880 |
return "routed image-to-image refinement—preserves a layout guide but consumes account inference credit"
|
| 2881 |
if "zerogpu" in lowered:
|
|
|
|
|
|
|
| 2882 |
return "direct locally hosted text-to-image diffusion—no procedural guide or per-image API credit; visual review required"
|
| 2883 |
return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
|
| 2884 |
|
| 2885 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2886 |
def model_configuration_summary() -> str:
|
| 2887 |
prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
|
| 2888 |
sprite_chain = []
|
| 2889 |
-
primary_source =
|
| 2890 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2891 |
sprite_chain.append(f"direct text-to-image {primary_source} → neural alpha {FOREGROUND_MODEL_ID}")
|
| 2892 |
if USE_DIFFUSION_FOR_SPRITES:
|
|
@@ -2918,7 +2939,9 @@ def model_configuration_summary() -> str:
|
|
| 2918 |
remote_fallback = " → ".join(remote_sources) if remote_sources else "disabled"
|
| 2919 |
neural_models = []
|
| 2920 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2921 |
-
neural_models.append(
|
|
|
|
|
|
|
| 2922 |
neural_models.append(FOREGROUND_MODEL_ID)
|
| 2923 |
if USE_DIFFUSION_FOR_SPRITES or USE_DIFFUSION_FOR_BACKGROUNDS:
|
| 2924 |
neural_models.append(FREE_IMAGE_MODEL)
|
|
@@ -3045,7 +3068,8 @@ def estimate_generation_gpu_duration(
|
|
| 3045 |
output_count = max(1, len(resolve_role_lines(html_code or "", roles or "")[0]))
|
| 3046 |
except AssetManifestError:
|
| 3047 |
output_count = 1
|
| 3048 |
-
|
|
|
|
| 3049 |
|
| 3050 |
|
| 3051 |
@gpu_task(duration=estimate_generation_gpu_duration)
|
|
@@ -3143,7 +3167,8 @@ def estimate_regeneration_gpu_duration(state: dict, selected_role: str, approved
|
|
| 3143 |
count = sum(1 for spec in specs if (spec.group or spec.role) == group)
|
| 3144 |
else:
|
| 3145 |
count = 1
|
| 3146 |
-
|
|
|
|
| 3147 |
|
| 3148 |
|
| 3149 |
@gpu_task(duration=estimate_regeneration_gpu_duration)
|
|
|
|
| 74 |
|
| 75 |
|
| 76 |
HF_TOKEN = os.environ.get("HF_TOKEN", "")
|
| 77 |
+
PRIMARY_IMAGE_MODEL = os.environ.get(
|
| 78 |
+
"PRIMARY_IMAGE_MODEL",
|
| 79 |
+
"stabilityai/stable-diffusion-xl-base-1.0",
|
| 80 |
+
)
|
| 81 |
+
PRIMARY_IMAGE_REVISION = os.environ.get("PRIMARY_IMAGE_REVISION", "").strip()
|
| 82 |
+
PRIMARY_LORA_MODEL = os.environ.get("PRIMARY_LORA_MODEL", "").strip()
|
| 83 |
+
PRIMARY_USES_LCM = bool(PRIMARY_LORA_MODEL)
|
| 84 |
FOREGROUND_MODEL_ID = os.environ.get("FOREGROUND_MODEL_ID", "ZhengPeng7/BiRefNet")
|
| 85 |
FOREGROUND_MODEL_REVISION = os.environ.get(
|
| 86 |
"FOREGROUND_MODEL_REVISION",
|
| 87 |
"e2bf8e4460fc8fa32bba5ea4d94b3233d367b0e4",
|
| 88 |
)
|
| 89 |
+
PRIMARY_IMAGE_STEPS = int(os.environ.get("PRIMARY_IMAGE_STEPS", "6" if PRIMARY_USES_LCM else "30"))
|
| 90 |
+
PRIMARY_SPRITE_STEPS = int(os.environ.get("PRIMARY_SPRITE_STEPS", "8" if PRIMARY_USES_LCM else "32"))
|
| 91 |
PRIMARY_SPRITE_ATTEMPTS = max(1, int(os.environ.get("PRIMARY_SPRITE_ATTEMPTS", "3")))
|
| 92 |
+
PRIMARY_GUIDANCE_SCALE = float(os.environ.get("PRIMARY_GUIDANCE_SCALE", "1.5" if PRIMARY_USES_LCM else "8.0"))
|
| 93 |
USE_PRIMARY_IMAGE_MODEL = os.environ.get("USE_PRIMARY_IMAGE_MODEL", "1") == "1"
|
| 94 |
REQUIRE_PRIMARY_IMAGE_MODEL = os.environ.get(
|
| 95 |
"REQUIRE_PRIMARY_IMAGE_MODEL",
|
|
|
|
| 1784 |
camera_negative = {
|
| 1785 |
"top_down": (
|
| 1786 |
", sky, horizon, eye-level view, front view, front-facing facade, upright portrait, "
|
| 1787 |
+
"standing toward camera, face looking at viewer, three-quarter view, oblique view, "
|
| 1788 |
+
"side elevation, low camera, vanishing point"
|
| 1789 |
),
|
| 1790 |
"isometric": ", eye-level view, front view, inconsistent perspective, horizon",
|
| 1791 |
"side_view": ", overhead view, top-down view, isometric view",
|
|
|
|
| 1867 |
|
| 1868 |
try:
|
| 1869 |
import torch
|
| 1870 |
+
from diffusers import AutoPipelineForText2Image
|
| 1871 |
|
| 1872 |
model_kwargs = {
|
| 1873 |
"torch_dtype": torch.float16,
|
|
|
|
| 1876 |
}
|
| 1877 |
if HF_TOKEN:
|
| 1878 |
model_kwargs["token"] = HF_TOKEN
|
| 1879 |
+
if PRIMARY_IMAGE_REVISION:
|
| 1880 |
+
model_kwargs["revision"] = PRIMARY_IMAGE_REVISION
|
| 1881 |
|
| 1882 |
text_pipe = AutoPipelineForText2Image.from_pretrained(PRIMARY_IMAGE_MODEL, **model_kwargs)
|
| 1883 |
+
if PRIMARY_LORA_MODEL:
|
| 1884 |
+
from diffusers import LCMScheduler
|
| 1885 |
+
|
| 1886 |
+
text_pipe.scheduler = LCMScheduler.from_config(text_pipe.scheduler.config)
|
| 1887 |
+
text_pipe.load_lora_weights(PRIMARY_LORA_MODEL)
|
| 1888 |
+
text_pipe.fuse_lora()
|
| 1889 |
text_pipe = text_pipe.to("cuda")
|
| 1890 |
if hasattr(text_pipe, "set_progress_bar_config"):
|
| 1891 |
text_pipe.set_progress_bar_config(disable=True)
|
|
|
|
| 1958 |
subject = (spec.group or spec.role).replace("_", " ")
|
| 1959 |
if spec.camera == "top_down":
|
| 1960 |
return compact_prompt_words(
|
| 1961 |
+
f"Strict orthographic top-down game sprite. Camera at 90 degrees directly above the {subject}, "
|
| 1962 |
+
"looking straight down. Show only top-facing surfaces and a compact foreshortened silhouette. "
|
| 1963 |
+
"No visible frontal face, chest, facade, or side elevation. "
|
| 1964 |
f"{description}. Exactly one complete isolated subject, centered on uniform white. "
|
| 1965 |
f"{variation}Crisp game-ready details. No scene.",
|
| 1966 |
55,
|
|
|
|
| 2645 |
|
| 2646 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2647 |
png_content, error = primary_diffusion_png(spec, index, run_id)
|
| 2648 |
+
source = primary_model_source()
|
| 2649 |
if not is_background:
|
| 2650 |
source += f" + {FOREGROUND_MODEL_ID} neural alpha"
|
| 2651 |
if error:
|
|
|
|
| 2891 |
if "via fal-ai" in lowered or "via replicate" in lowered:
|
| 2892 |
return "routed image-to-image refinement—preserves a layout guide but consumes account inference credit"
|
| 2893 |
if "zerogpu" in lowered:
|
| 2894 |
+
if "lcm" not in lowered:
|
| 2895 |
+
return "full-step locally hosted diffusion—stronger prompt guidance and effective negative prompting; visual review required"
|
| 2896 |
return "direct locally hosted text-to-image diffusion—no procedural guide or per-image API credit; visual review required"
|
| 2897 |
return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
|
| 2898 |
|
| 2899 |
|
| 2900 |
+
def primary_model_source() -> str:
|
| 2901 |
+
model_chain = PRIMARY_IMAGE_MODEL
|
| 2902 |
+
if PRIMARY_LORA_MODEL:
|
| 2903 |
+
model_chain += f" + {PRIMARY_LORA_MODEL}"
|
| 2904 |
+
return f"{model_chain} on ZeroGPU"
|
| 2905 |
+
|
| 2906 |
+
|
| 2907 |
def model_configuration_summary() -> str:
|
| 2908 |
prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
|
| 2909 |
sprite_chain = []
|
| 2910 |
+
primary_source = primary_model_source()
|
| 2911 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2912 |
sprite_chain.append(f"direct text-to-image {primary_source} → neural alpha {FOREGROUND_MODEL_ID}")
|
| 2913 |
if USE_DIFFUSION_FOR_SPRITES:
|
|
|
|
| 2939 |
remote_fallback = " → ".join(remote_sources) if remote_sources else "disabled"
|
| 2940 |
neural_models = []
|
| 2941 |
if USE_PRIMARY_IMAGE_MODEL:
|
| 2942 |
+
neural_models.append(
|
| 2943 |
+
f"{PRIMARY_IMAGE_MODEL} + {PRIMARY_LORA_MODEL}" if PRIMARY_LORA_MODEL else PRIMARY_IMAGE_MODEL
|
| 2944 |
+
)
|
| 2945 |
neural_models.append(FOREGROUND_MODEL_ID)
|
| 2946 |
if USE_DIFFUSION_FOR_SPRITES or USE_DIFFUSION_FOR_BACKGROUNDS:
|
| 2947 |
neural_models.append(FREE_IMAGE_MODEL)
|
|
|
|
| 3068 |
output_count = max(1, len(resolve_role_lines(html_code or "", roles or "")[0]))
|
| 3069 |
except AssetManifestError:
|
| 3070 |
output_count = 1
|
| 3071 |
+
per_output = max(36, int(max(PRIMARY_IMAGE_STEPS, PRIMARY_SPRITE_STEPS) * 1.5))
|
| 3072 |
+
return min(300, 40 + output_count * per_output)
|
| 3073 |
|
| 3074 |
|
| 3075 |
@gpu_task(duration=estimate_generation_gpu_duration)
|
|
|
|
| 3167 |
count = sum(1 for spec in specs if (spec.group or spec.role) == group)
|
| 3168 |
else:
|
| 3169 |
count = 1
|
| 3170 |
+
per_output = max(36, int(max(PRIMARY_IMAGE_STEPS, PRIMARY_SPRITE_STEPS) * 1.5))
|
| 3171 |
+
return min(300, 35 + max(1, count) * per_output)
|
| 3172 |
|
| 3173 |
|
| 3174 |
@gpu_task(duration=estimate_regeneration_gpu_duration)
|