Enforce camera perspective in background generation
Browse files
README.md
CHANGED
|
@@ -31,10 +31,48 @@ If a requested role has no deterministic hook, assets are still generated for re
|
|
| 31 |
|
| 32 |
Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
|
| 33 |
|
| 34 |
-
The UI reports the configured prompt and image pipeline and labels every role with its actual source.
|
|
|
|
|
|
|
| 35 |
|
| 36 |
The app combines the game type, perspective, theme, and pasted code into explicit per-role image prompts, then generates PNG assets and embeds contract-compatible assets as base64 data URIs.
|
| 37 |
|
| 38 |
-
If the Space has an `HF_TOKEN` secret, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
-
|
|
|
|
| 31 |
|
| 32 |
Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
|
| 33 |
|
| 34 |
+
The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset now carries a normalized camera contract (`top_down`, `side_view`, `isometric`, `first_person`, or `front_view`). Top-down background prompts explicitly forbid a sky, horizon, eye-level view, and front-facing facades. The free procedural renderer also routes top-down and isometric backgrounds through dedicated geometry, so camera compliance does not depend only on prompt wording.
|
| 35 |
+
|
| 36 |
+
Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
|
| 37 |
|
| 38 |
The app combines the game type, perspective, theme, and pasted code into explicit per-role image prompts, then generates PNG assets and embeds contract-compatible assets as base64 data URIs.
|
| 39 |
|
| 40 |
+
If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
|
| 41 |
+
|
| 42 |
+
## Background generation tiers
|
| 43 |
+
|
| 44 |
+
The default tier is deliberately free and deterministic:
|
| 45 |
+
|
| 46 |
+
```text
|
| 47 |
+
camera-aware procedural background
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
It runs on CPU Basic and is the active production-safe fallback. It fixes camera geometry, but it cannot match the semantic richness of a large image model.
|
| 51 |
+
|
| 52 |
+
For a GPU Space, enable the production depth-control path:
|
| 53 |
+
|
| 54 |
+
```text
|
| 55 |
+
USE_CONTROLNET_FOR_BACKGROUNDS=1
|
| 56 |
+
CONTROLNET_BASE_MODEL=stabilityai/stable-diffusion-xl-base-1.0
|
| 57 |
+
CONTROLNET_MODEL=diffusers/controlnet-depth-sdxl-1.0-small
|
| 58 |
+
CONTROLNET_STEPS=24
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet before falling back. It intentionally refuses to run on CPU-only hardware: loading SDXL on CPU Basic is not a credible interactive production configuration. The small ControlNet adapter is experimental, so perspective still needs human review.
|
| 62 |
+
|
| 63 |
+
For an API-backed refinement that works from a CPU Space, enable:
|
| 64 |
+
|
| 65 |
+
```text
|
| 66 |
+
USE_HF_STRUCTURE_PROVIDER=1
|
| 67 |
+
HF_STRUCTURE_MODEL=black-forest-labs/FLUX.2-klein-4B
|
| 68 |
+
HF_STRUCTURE_PROVIDER=fal-ai
|
| 69 |
+
HF_TOKEN=<Space secret with Inference Providers permission>
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
This sends the deterministic background as an image-to-image layout reference, asks the model to preserve its projection and walkable geometry, and falls back safely if the request fails. `FLUX.2-klein-4B` is Apache-2.0 licensed and available through a Hugging Face Inference Provider. It is not unlimited-free infrastructure: a free Hugging Face user currently receives only a small monthly inference credit, after which purchased credits are required. Keep the flag disabled unless paid API usage and token handling are acceptable.
|
| 73 |
+
|
| 74 |
+
The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
|
| 75 |
+
|
| 76 |
+
If every enabled model path fails, the app uses the local style-aware Pillow fallback, which varies watercolor, vector/neon, claymation, oil-painting, anime, pixel-art looks, and common game sprite silhouettes.
|
| 77 |
|
| 78 |
+
`black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.
|
app.py
CHANGED
|
@@ -92,6 +92,7 @@ class AssetSpec:
|
|
| 92 |
filename: str
|
| 93 |
width: int
|
| 94 |
height: int
|
|
|
|
| 95 |
|
| 96 |
|
| 97 |
@dataclass
|
|
@@ -121,11 +122,20 @@ USE_HF_PROMPT_PROVIDER = os.environ.get("USE_HF_PROMPT_PROVIDER", "0") == "1"
|
|
| 121 |
USE_HF_IMAGE_PROVIDER = os.environ.get("USE_HF_IMAGE_PROVIDER", "0") == "1"
|
| 122 |
USE_DIFFUSION_FOR_SPRITES = os.environ.get("USE_DIFFUSION_FOR_SPRITES", "0") == "1"
|
| 123 |
USE_DIFFUSION_FOR_BACKGROUNDS = os.environ.get("USE_DIFFUSION_FOR_BACKGROUNDS", "0") == "1"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
HF_IMAGE_MODEL = os.environ.get("HF_IMAGE_MODEL", "black-forest-labs/FLUX.1-schnell")
|
| 125 |
HF_PROMPT_MODEL = os.environ.get("HF_PROMPT_MODEL", "Qwen/Qwen2.5-Coder-7B-Instruct:fastest")
|
| 126 |
HF_PROMPT_ENDPOINT = os.environ.get("HF_PROMPT_ENDPOINT", "https://router.huggingface.co/v1/chat/completions")
|
| 127 |
FREE_DIFFUSION_PIPE = None
|
| 128 |
FREE_DIFFUSION_ERROR = None
|
|
|
|
|
|
|
| 129 |
|
| 130 |
|
| 131 |
def slugify(value: str) -> str:
|
|
@@ -133,6 +143,44 @@ def slugify(value: str) -> str:
|
|
| 133 |
return value or "asset"
|
| 134 |
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
def interpret_style_hint(style_hint: str) -> StylePlan:
|
| 137 |
text = (style_hint or "").lower()
|
| 138 |
tags: list[str] = []
|
|
@@ -198,24 +246,20 @@ def interpret_style_hint(style_hint: str) -> StylePlan:
|
|
| 198 |
lighting = "balanced game lighting"
|
| 199 |
linework = "clear readable silhouette and controlled edges"
|
| 200 |
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
camera = "first-person game camera"
|
| 209 |
-
elif has("front-facing", "front facing"):
|
| 210 |
-
camera = "front-facing game camera"
|
| 211 |
-
else:
|
| 212 |
-
camera = "game-ready camera angle matching the role"
|
| 213 |
|
| 214 |
return StylePlan(medium, palette, texture, lighting, linework, camera, tuple(tags))
|
| 215 |
|
| 216 |
|
| 217 |
def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
|
| 218 |
plan = interpret_style_hint(style_hint)
|
|
|
|
| 219 |
slug = slugify(role)
|
| 220 |
is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
|
| 221 |
if is_background:
|
|
@@ -234,6 +278,7 @@ def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
|
|
| 234 |
f"{role} asset: {prompt}. Creative brief: {style_hint}. "
|
| 235 |
f"Style interpretation: {plan.medium}; {plan.palette}; "
|
| 236 |
f"{plan.texture}; {plan.lighting}; {plan.linework}; {plan.camera}. "
|
|
|
|
| 237 |
f"{asset_instruction} "
|
| 238 |
"Game asset, readable at small size, no text, no watermark."
|
| 239 |
)
|
|
@@ -397,8 +442,25 @@ def parse_assets(raw_roles: str, style_hint: str, prompt_map: dict[str, str] | N
|
|
| 397 |
is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
|
| 398 |
width, height = (800, 450) if is_background else (128, 128)
|
| 399 |
filename = f"sprite_{slug}.png"
|
| 400 |
-
|
| 401 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 402 |
return specs
|
| 403 |
|
| 404 |
|
|
@@ -577,15 +639,170 @@ def apply_style_finish(content: bytes, spec: AssetSpec, rng: random.Random) -> b
|
|
| 577 |
return out.getvalue()
|
| 578 |
|
| 579 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 580 |
def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
|
| 581 |
colors = palette_for(spec.prompt)
|
| 582 |
image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
|
| 583 |
draw = ImageDraw.Draw(image, "RGBA")
|
| 584 |
prompt = spec.prompt.lower()
|
| 585 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 586 |
if (
|
| 587 |
any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "hazard", "grid", "shooter"))
|
| 588 |
-
and not
|
| 589 |
):
|
| 590 |
image = Image.new("RGBA", (spec.width, spec.height), (18, 22, 31, 255))
|
| 591 |
draw = ImageDraw.Draw(image, "RGBA")
|
|
@@ -620,7 +837,7 @@ def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
|
|
| 620 |
image.save(out, format="PNG")
|
| 621 |
return out.getvalue()
|
| 622 |
|
| 623 |
-
if
|
| 624 |
image = Image.new("RGBA", (spec.width, spec.height), (3, 4, 18, 255))
|
| 625 |
draw = ImageDraw.Draw(image, "RGBA")
|
| 626 |
for y in range(spec.height):
|
|
@@ -1085,10 +1302,119 @@ def diffusion_negative_prompt(spec: AssetSpec) -> str:
|
|
| 1085 |
"multiple objects, cropped subject, close-up surface, fabric swatch, text, watermark"
|
| 1086 |
)
|
| 1087 |
if is_background_spec(spec):
|
| 1088 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1089 |
return common + ", landscape, environment scene, room, horizon, opaque background, drop shadow"
|
| 1090 |
|
| 1091 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1092 |
def polish_diffusion_asset(image: Image.Image, spec: AssetSpec) -> bytes:
|
| 1093 |
image = image.convert("RGBA")
|
| 1094 |
if not is_background_spec(spec):
|
|
@@ -1181,8 +1507,37 @@ def hf_image_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None
|
|
| 1181 |
return None, short_error(exc)
|
| 1182 |
|
| 1183 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1184 |
def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str, str | None, str]:
|
| 1185 |
-
|
|
|
|
| 1186 |
png_content = local_asset_png(spec, index, run_id)
|
| 1187 |
return (
|
| 1188 |
png_bytes_to_data_uri(png_content),
|
|
@@ -1190,7 +1545,15 @@ def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str,
|
|
| 1190 |
None,
|
| 1191 |
"shape-aware 2D sprite generator",
|
| 1192 |
)
|
| 1193 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1194 |
png_content = local_asset_png(spec, index, run_id)
|
| 1195 |
return (
|
| 1196 |
png_bytes_to_data_uri(png_content),
|
|
@@ -1199,20 +1562,42 @@ def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str,
|
|
| 1199 |
"scene-safe 2D background generator",
|
| 1200 |
)
|
| 1201 |
|
| 1202 |
-
png_content
|
| 1203 |
-
source =
|
| 1204 |
-
|
| 1205 |
-
|
| 1206 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1207 |
source = HF_IMAGE_MODEL
|
| 1208 |
-
|
|
|
|
|
|
|
| 1209 |
if png_content is None:
|
| 1210 |
png_content = local_asset_png(spec, index, run_id)
|
| 1211 |
source = "local procedural fallback"
|
| 1212 |
return (
|
| 1213 |
png_bytes_to_data_uri(png_content),
|
| 1214 |
write_gallery_image(png_content, spec.role),
|
| 1215 |
-
|
| 1216 |
source,
|
| 1217 |
)
|
| 1218 |
|
|
@@ -1369,20 +1754,45 @@ def model_source_note(image_model: str) -> str:
|
|
| 1369 |
lowered = image_model.lower()
|
| 1370 |
if "shape-aware" in lowered or "scene-safe" in lowered or "procedural" in lowered:
|
| 1371 |
return "procedural/heuristic output—predictable structure, limited open-ended semantic fidelity"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1372 |
return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
|
| 1373 |
|
| 1374 |
|
| 1375 |
def model_configuration_summary() -> str:
|
| 1376 |
prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
|
| 1377 |
-
|
| 1378 |
-
|
| 1379 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1380 |
return (
|
| 1381 |
"**Configured model pipeline:** "
|
| 1382 |
f"prompts: `{prompt_source}` · sprites: `{sprite_source}` · backgrounds: `{background_source}` · "
|
| 1383 |
f"remote image fallback: `{remote_fallback}`. "
|
| 1384 |
-
"
|
| 1385 |
-
"
|
| 1386 |
)
|
| 1387 |
|
| 1388 |
|
|
|
|
| 92 |
filename: str
|
| 93 |
width: int
|
| 94 |
height: int
|
| 95 |
+
camera: str = "auto"
|
| 96 |
|
| 97 |
|
| 98 |
@dataclass
|
|
|
|
| 122 |
USE_HF_IMAGE_PROVIDER = os.environ.get("USE_HF_IMAGE_PROVIDER", "0") == "1"
|
| 123 |
USE_DIFFUSION_FOR_SPRITES = os.environ.get("USE_DIFFUSION_FOR_SPRITES", "0") == "1"
|
| 124 |
USE_DIFFUSION_FOR_BACKGROUNDS = os.environ.get("USE_DIFFUSION_FOR_BACKGROUNDS", "0") == "1"
|
| 125 |
+
USE_CONTROLNET_FOR_BACKGROUNDS = os.environ.get("USE_CONTROLNET_FOR_BACKGROUNDS", "0") == "1"
|
| 126 |
+
CONTROLNET_BASE_MODEL = os.environ.get("CONTROLNET_BASE_MODEL", "stabilityai/stable-diffusion-xl-base-1.0")
|
| 127 |
+
CONTROLNET_MODEL = os.environ.get("CONTROLNET_MODEL", "diffusers/controlnet-depth-sdxl-1.0-small")
|
| 128 |
+
CONTROLNET_STEPS = int(os.environ.get("CONTROLNET_STEPS", "24"))
|
| 129 |
+
USE_HF_STRUCTURE_PROVIDER = os.environ.get("USE_HF_STRUCTURE_PROVIDER", "0") == "1"
|
| 130 |
+
HF_STRUCTURE_MODEL = os.environ.get("HF_STRUCTURE_MODEL", "black-forest-labs/FLUX.2-klein-4B")
|
| 131 |
+
HF_STRUCTURE_PROVIDER = os.environ.get("HF_STRUCTURE_PROVIDER", "fal-ai")
|
| 132 |
HF_IMAGE_MODEL = os.environ.get("HF_IMAGE_MODEL", "black-forest-labs/FLUX.1-schnell")
|
| 133 |
HF_PROMPT_MODEL = os.environ.get("HF_PROMPT_MODEL", "Qwen/Qwen2.5-Coder-7B-Instruct:fastest")
|
| 134 |
HF_PROMPT_ENDPOINT = os.environ.get("HF_PROMPT_ENDPOINT", "https://router.huggingface.co/v1/chat/completions")
|
| 135 |
FREE_DIFFUSION_PIPE = None
|
| 136 |
FREE_DIFFUSION_ERROR = None
|
| 137 |
+
CONTROLNET_PIPE = None
|
| 138 |
+
CONTROLNET_ERROR = None
|
| 139 |
|
| 140 |
|
| 141 |
def slugify(value: str) -> str:
|
|
|
|
| 143 |
return value or "asset"
|
| 144 |
|
| 145 |
|
| 146 |
+
def normalize_camera(text: str) -> str:
|
| 147 |
+
"""Convert UI and prompt camera wording into a stable generation contract."""
|
| 148 |
+
lowered = (text or "").lower()
|
| 149 |
+
explicit = re.search(r"required camera perspective:\s*([^.]*)", lowered)
|
| 150 |
+
camera_text = explicit.group(1) if explicit else lowered
|
| 151 |
+
if any(word in camera_text for word in ("top-down", "top down", "overhead", "bird's-eye", "birds-eye")):
|
| 152 |
+
return "top_down"
|
| 153 |
+
if any(word in camera_text for word in ("isometric", "3/4 view", "three-quarter")):
|
| 154 |
+
return "isometric"
|
| 155 |
+
if any(word in camera_text for word in ("side view", "side-view", "side-scroller", "side scroller", "platformer")):
|
| 156 |
+
return "side_view"
|
| 157 |
+
if any(word in camera_text for word in ("first-person", "first person")):
|
| 158 |
+
return "first_person"
|
| 159 |
+
if any(word in camera_text for word in ("front-facing", "front facing", "front view")):
|
| 160 |
+
return "front_view"
|
| 161 |
+
return "auto"
|
| 162 |
+
|
| 163 |
+
|
| 164 |
+
def camera_prompt_contract(camera: str, is_background: bool) -> str:
|
| 165 |
+
if camera == "top_down":
|
| 166 |
+
if is_background:
|
| 167 |
+
return (
|
| 168 |
+
"Strict orthographic overhead composition viewed straight down. Show roofs, ground, paths, and "
|
| 169 |
+
"objects from above. No visible sky, no horizon line, no eye-level view, no front-facing facade, "
|
| 170 |
+
"and no vanishing-point perspective. Keep traversal space readable across the whole frame."
|
| 171 |
+
)
|
| 172 |
+
return "Strict overhead view of the subject from directly above; not front-facing and not side-view."
|
| 173 |
+
if camera == "isometric":
|
| 174 |
+
return "Consistent isometric three-quarter projection with parallel axes and no eye-level horizon."
|
| 175 |
+
if camera == "side_view":
|
| 176 |
+
return "Strict side-view profile composition suitable for a 2D platformer."
|
| 177 |
+
if camera == "first_person":
|
| 178 |
+
return "First-person viewpoint with environment geometry aligned to the player's eye line."
|
| 179 |
+
if camera == "front_view":
|
| 180 |
+
return "Front-facing composition with the subject or environment oriented toward the camera."
|
| 181 |
+
return "Use one internally consistent game camera angle appropriate to the role."
|
| 182 |
+
|
| 183 |
+
|
| 184 |
def interpret_style_hint(style_hint: str) -> StylePlan:
|
| 185 |
text = (style_hint or "").lower()
|
| 186 |
tags: list[str] = []
|
|
|
|
| 246 |
lighting = "balanced game lighting"
|
| 247 |
linework = "clear readable silhouette and controlled edges"
|
| 248 |
|
| 249 |
+
camera = {
|
| 250 |
+
"top_down": "top-down readable game camera",
|
| 251 |
+
"side_view": "side-view platformer camera",
|
| 252 |
+
"isometric": "consistent isometric three-quarter game camera",
|
| 253 |
+
"first_person": "first-person game camera",
|
| 254 |
+
"front_view": "front-facing game camera",
|
| 255 |
+
}.get(normalize_camera(style_hint), "game-ready camera angle matching the role")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 256 |
|
| 257 |
return StylePlan(medium, palette, texture, lighting, linework, camera, tuple(tags))
|
| 258 |
|
| 259 |
|
| 260 |
def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
|
| 261 |
plan = interpret_style_hint(style_hint)
|
| 262 |
+
camera = normalize_camera(f"{style_hint} {prompt}")
|
| 263 |
slug = slugify(role)
|
| 264 |
is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
|
| 265 |
if is_background:
|
|
|
|
| 278 |
f"{role} asset: {prompt}. Creative brief: {style_hint}. "
|
| 279 |
f"Style interpretation: {plan.medium}; {plan.palette}; "
|
| 280 |
f"{plan.texture}; {plan.lighting}; {plan.linework}; {plan.camera}. "
|
| 281 |
+
f"Camera contract: {camera_prompt_contract(camera, is_background)} "
|
| 282 |
f"{asset_instruction} "
|
| 283 |
"Game asset, readable at small size, no text, no watermark."
|
| 284 |
)
|
|
|
|
| 442 |
is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
|
| 443 |
width, height = (800, 450) if is_background else (128, 128)
|
| 444 |
filename = f"sprite_{slug}.png"
|
| 445 |
+
camera = normalize_camera(f"{style_hint} {prompt}")
|
| 446 |
+
interpreted_prompt = (prompt_map or {}).get(role)
|
| 447 |
+
if interpreted_prompt:
|
| 448 |
+
full_prompt = (
|
| 449 |
+
f"{interpreted_prompt}. Camera contract: {camera_prompt_contract(camera, is_background)} "
|
| 450 |
+
"Game asset, no text, no watermark."
|
| 451 |
+
)
|
| 452 |
+
else:
|
| 453 |
+
full_prompt = build_asset_prompt(role, prompt, style_hint)
|
| 454 |
+
specs.append(
|
| 455 |
+
AssetSpec(
|
| 456 |
+
role=role,
|
| 457 |
+
prompt=full_prompt,
|
| 458 |
+
filename=filename,
|
| 459 |
+
width=width,
|
| 460 |
+
height=height,
|
| 461 |
+
camera=camera,
|
| 462 |
+
)
|
| 463 |
+
)
|
| 464 |
return specs
|
| 465 |
|
| 466 |
|
|
|
|
| 639 |
return out.getvalue()
|
| 640 |
|
| 641 |
|
| 642 |
+
def finish_png(image: Image.Image) -> bytes:
|
| 643 |
+
out = io.BytesIO()
|
| 644 |
+
image.save(out, format="PNG")
|
| 645 |
+
return out.getvalue()
|
| 646 |
+
|
| 647 |
+
|
| 648 |
+
def is_space_background_prompt(prompt: str) -> bool:
|
| 649 |
+
lowered = prompt.lower()
|
| 650 |
+
return any(
|
| 651 |
+
phrase in lowered
|
| 652 |
+
for phrase in ("outer space", "deep space", "starfield", "star field", "nebula", "galaxy", "cosmic")
|
| 653 |
+
)
|
| 654 |
+
|
| 655 |
+
|
| 656 |
+
def draw_top_down_background(spec: AssetSpec, rng: random.Random) -> bytes:
|
| 657 |
+
"""Draw an explicitly horizon-free overhead map for the free default path."""
|
| 658 |
+
prompt = spec.prompt.lower()
|
| 659 |
+
colors = palette_for(spec.prompt)
|
| 660 |
+
width, height = spec.width, spec.height
|
| 661 |
+
|
| 662 |
+
if any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "space station", "shooter")):
|
| 663 |
+
image = Image.new("RGBA", (width, height), (18, 22, 31, 255))
|
| 664 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 665 |
+
panel = 64
|
| 666 |
+
for y in range(0, height, panel):
|
| 667 |
+
for x in range(0, width, panel):
|
| 668 |
+
shade = 24 + ((x // panel + y // panel) % 2) * 12
|
| 669 |
+
draw.rectangle(
|
| 670 |
+
(x, y, x + panel - 2, y + panel - 2),
|
| 671 |
+
fill=(shade, shade + 4, shade + 14, 255),
|
| 672 |
+
outline=(58, 74, 92, 170),
|
| 673 |
+
)
|
| 674 |
+
for x in range(0, width, 96):
|
| 675 |
+
draw.line((x, 0, x, height), fill=(34, 215, 236, 95), width=2)
|
| 676 |
+
for y in range(0, height, 96):
|
| 677 |
+
draw.line((0, y, width, y), fill=(34, 215, 236, 95), width=2)
|
| 678 |
+
return finish_png(image)
|
| 679 |
+
|
| 680 |
+
if any(word in prompt for word in ("garden", "flower", "grass", "meadow", "forest", "jungle", "path")):
|
| 681 |
+
image = Image.new("RGBA", (width, height), (64, 132, 70, 255))
|
| 682 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 683 |
+
for _ in range(85):
|
| 684 |
+
x, y = rng.randint(0, width), rng.randint(0, height)
|
| 685 |
+
rx, ry = rng.randint(18, 70), rng.randint(12, 48)
|
| 686 |
+
draw.ellipse((x - rx, y - ry, x + rx, y + ry), fill=(78, rng.randint(135, 169), 76, 75))
|
| 687 |
+
|
| 688 |
+
path_points = [
|
| 689 |
+
(-40, int(height * 0.76)),
|
| 690 |
+
(int(width * 0.20), int(height * 0.66)),
|
| 691 |
+
(int(width * 0.43), int(height * 0.52)),
|
| 692 |
+
(int(width * 0.63), int(height * 0.37)),
|
| 693 |
+
(int(width * 0.83), int(height * 0.22)),
|
| 694 |
+
(width + 40, int(height * 0.13)),
|
| 695 |
+
]
|
| 696 |
+
draw.line(path_points, fill=(112, 91, 59, 155), width=126, joint="curve")
|
| 697 |
+
draw.line(path_points, fill=(198, 169, 112, 255), width=108, joint="curve")
|
| 698 |
+
for _ in range(31):
|
| 699 |
+
edge = rng.choice(("top", "bottom", "left", "right"))
|
| 700 |
+
if edge in ("top", "bottom"):
|
| 701 |
+
x = rng.randint(-24, width + 24)
|
| 702 |
+
y = rng.randint(-28, 94) if edge == "top" else rng.randint(height - 94, height + 28)
|
| 703 |
+
else:
|
| 704 |
+
x = rng.randint(-28, 86) if edge == "left" else rng.randint(width - 86, width + 28)
|
| 705 |
+
y = rng.randint(-24, height + 24)
|
| 706 |
+
radius = rng.randint(24, 53)
|
| 707 |
+
draw.ellipse(
|
| 708 |
+
(x - radius, y - radius, x + radius, y + radius),
|
| 709 |
+
fill=(28, rng.randint(82, 116), 48, 255),
|
| 710 |
+
outline=(18, 70, 37, 210),
|
| 711 |
+
width=3,
|
| 712 |
+
)
|
| 713 |
+
inner = int(radius * 0.66)
|
| 714 |
+
draw.ellipse((x - inner, y - inner, x + inner, y + inner), fill=(58, rng.randint(126, 166), 69, 245))
|
| 715 |
+
for _ in range(42):
|
| 716 |
+
x, y = rng.randint(0, width), rng.randint(0, height)
|
| 717 |
+
radius = rng.randint(2, 5)
|
| 718 |
+
draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=rng.choice(colors) + (220,))
|
| 719 |
+
return finish_png(image)
|
| 720 |
+
|
| 721 |
+
if any(word in prompt for word in ("dungeon", "castle", "stone", "crypt")):
|
| 722 |
+
image = Image.new("RGBA", (width, height), (48, 50, 62, 255))
|
| 723 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 724 |
+
tile = 54
|
| 725 |
+
for y in range(0, height, tile):
|
| 726 |
+
for x in range(0, width, tile):
|
| 727 |
+
shade = rng.randint(48, 78)
|
| 728 |
+
draw.rectangle(
|
| 729 |
+
(x, y, x + tile - 2, y + tile - 2),
|
| 730 |
+
fill=(shade, shade + 2, shade + 8, 255),
|
| 731 |
+
outline=(27, 29, 38, 190),
|
| 732 |
+
)
|
| 733 |
+
return finish_png(image)
|
| 734 |
+
|
| 735 |
+
if any(word in prompt for word in ("lava", "fire", "volcano")):
|
| 736 |
+
image = Image.new("RGBA", (width, height), (52, 23, 18, 255))
|
| 737 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 738 |
+
for _ in range(24):
|
| 739 |
+
x, y = rng.randint(-80, width), rng.randint(0, height)
|
| 740 |
+
points = [(x, y)]
|
| 741 |
+
for _ in range(rng.randint(2, 5)):
|
| 742 |
+
x += rng.randint(45, 125)
|
| 743 |
+
y += rng.randint(-45, 45)
|
| 744 |
+
points.append((x, y))
|
| 745 |
+
draw.line(points, fill=(255, 103, 35, 235), width=rng.randint(7, 15), joint="curve")
|
| 746 |
+
draw.line(points, fill=(255, 202, 64, 210), width=rng.randint(2, 5), joint="curve")
|
| 747 |
+
return finish_png(image)
|
| 748 |
+
|
| 749 |
+
if any(word in prompt for word in ("ocean", "sea", "water", "underwater")):
|
| 750 |
+
image = Image.new("RGBA", (width, height), (18, 114, 158, 255))
|
| 751 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 752 |
+
for _ in range(42):
|
| 753 |
+
x, y = rng.randint(0, width), rng.randint(0, height)
|
| 754 |
+
rx, ry = rng.randint(18, 75), rng.randint(7, 25)
|
| 755 |
+
draw.ellipse((x - rx, y - ry, x + rx, y + ry), outline=(176, 236, 230, 95), width=3)
|
| 756 |
+
return finish_png(image)
|
| 757 |
+
|
| 758 |
+
image = Image.new("RGBA", (width, height), colors[2] + (255,))
|
| 759 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 760 |
+
for _ in range(75):
|
| 761 |
+
x, y = rng.randint(0, width), rng.randint(0, height)
|
| 762 |
+
radius = rng.randint(8, 52)
|
| 763 |
+
draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=rng.choice(colors) + (rng.randint(45, 105),))
|
| 764 |
+
return finish_png(image)
|
| 765 |
+
|
| 766 |
+
|
| 767 |
+
def draw_isometric_background(spec: AssetSpec, rng: random.Random) -> bytes:
|
| 768 |
+
"""Draw a stable diamond-grid projection instead of an accidental horizon view."""
|
| 769 |
+
colors = palette_for(spec.prompt)
|
| 770 |
+
image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
|
| 771 |
+
draw = ImageDraw.Draw(image, "RGBA")
|
| 772 |
+
tile_w, tile_h = 96, 48
|
| 773 |
+
for row in range(-2, spec.height // tile_h + 4):
|
| 774 |
+
for col in range(-2, spec.width // tile_w + 4):
|
| 775 |
+
cx = col * tile_w + (row % 2) * (tile_w // 2)
|
| 776 |
+
cy = row * (tile_h // 2)
|
| 777 |
+
diamond = [(cx, cy), (cx + tile_w // 2, cy + tile_h // 2), (cx, cy + tile_h), (cx - tile_w // 2, cy + tile_h // 2)]
|
| 778 |
+
shade = colors[(row + col) % len(colors)] + (115,)
|
| 779 |
+
draw.polygon(diamond, fill=shade, outline=colors[3] + (150,))
|
| 780 |
+
for _ in range(14):
|
| 781 |
+
cx, cy = rng.randint(40, spec.width - 40), rng.randint(45, spec.height - 55)
|
| 782 |
+
size = rng.randint(16, 35)
|
| 783 |
+
top = [(cx, cy - size), (cx + size, cy - size // 2), (cx, cy), (cx - size, cy - size // 2)]
|
| 784 |
+
left = [(cx - size, cy - size // 2), (cx, cy), (cx, cy + size), (cx - size, cy + size // 2)]
|
| 785 |
+
right = [(cx + size, cy - size // 2), (cx, cy), (cx, cy + size), (cx + size, cy + size // 2)]
|
| 786 |
+
draw.polygon(left, fill=colors[2] + (235,))
|
| 787 |
+
draw.polygon(right, fill=colors[0] + (235,))
|
| 788 |
+
draw.polygon(top, fill=colors[1] + (245,))
|
| 789 |
+
return finish_png(image)
|
| 790 |
+
|
| 791 |
+
|
| 792 |
def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
|
| 793 |
colors = palette_for(spec.prompt)
|
| 794 |
image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
|
| 795 |
draw = ImageDraw.Draw(image, "RGBA")
|
| 796 |
prompt = spec.prompt.lower()
|
| 797 |
|
| 798 |
+
if spec.camera == "top_down" and not is_space_background_prompt(prompt):
|
| 799 |
+
return draw_top_down_background(spec, rng)
|
| 800 |
+
if spec.camera == "isometric":
|
| 801 |
+
return draw_isometric_background(spec, rng)
|
| 802 |
+
|
| 803 |
if (
|
| 804 |
any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "hazard", "grid", "shooter"))
|
| 805 |
+
and not is_space_background_prompt(prompt)
|
| 806 |
):
|
| 807 |
image = Image.new("RGBA", (spec.width, spec.height), (18, 22, 31, 255))
|
| 808 |
draw = ImageDraw.Draw(image, "RGBA")
|
|
|
|
| 837 |
image.save(out, format="PNG")
|
| 838 |
return out.getvalue()
|
| 839 |
|
| 840 |
+
if is_space_background_prompt(prompt):
|
| 841 |
image = Image.new("RGBA", (spec.width, spec.height), (3, 4, 18, 255))
|
| 842 |
draw = ImageDraw.Draw(image, "RGBA")
|
| 843 |
for y in range(spec.height):
|
|
|
|
| 1302 |
"multiple objects, cropped subject, close-up surface, fabric swatch, text, watermark"
|
| 1303 |
)
|
| 1304 |
if is_background_spec(spec):
|
| 1305 |
+
camera_negative = {
|
| 1306 |
+
"top_down": ", sky, horizon, eye-level view, front view, front-facing facade, vanishing point",
|
| 1307 |
+
"isometric": ", eye-level view, front view, inconsistent perspective, horizon",
|
| 1308 |
+
"side_view": ", overhead view, top-down view, isometric view",
|
| 1309 |
+
}.get(spec.camera, "")
|
| 1310 |
+
return (
|
| 1311 |
+
common
|
| 1312 |
+
+ ", person, character, player, hero, creature, monster, vehicle, mascot, foreground subject"
|
| 1313 |
+
+ camera_negative
|
| 1314 |
+
)
|
| 1315 |
return common + ", landscape, environment scene, room, horizon, opaque background, drop shadow"
|
| 1316 |
|
| 1317 |
|
| 1318 |
+
def build_camera_control_image(spec: AssetSpec, width: int = 1024, height: int = 576) -> Image.Image:
|
| 1319 |
+
"""Create a deterministic depth-like layout that encodes the requested camera."""
|
| 1320 |
+
image = Image.new("L", (width, height), 172)
|
| 1321 |
+
draw = ImageDraw.Draw(image)
|
| 1322 |
+
camera = spec.camera
|
| 1323 |
+
|
| 1324 |
+
if camera == "top_down":
|
| 1325 |
+
# Overhead maps use local depth islands and a constant-width traversal path.
|
| 1326 |
+
# There is deliberately no full-width horizontal boundary that could become a horizon.
|
| 1327 |
+
path = [
|
| 1328 |
+
(-60, int(height * 0.78)),
|
| 1329 |
+
(int(width * 0.18), int(height * 0.66)),
|
| 1330 |
+
(int(width * 0.43), int(height * 0.54)),
|
| 1331 |
+
(int(width * 0.68), int(height * 0.34)),
|
| 1332 |
+
(width + 60, int(height * 0.18)),
|
| 1333 |
+
]
|
| 1334 |
+
draw.line(path, fill=205, width=150, joint="curve")
|
| 1335 |
+
seed = random.Random(f"control|{spec.role}|{spec.prompt}")
|
| 1336 |
+
for _ in range(24):
|
| 1337 |
+
x, y = seed.randint(0, width), seed.randint(0, height)
|
| 1338 |
+
radius = seed.randint(22, 62)
|
| 1339 |
+
draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=seed.randint(55, 125))
|
| 1340 |
+
elif camera == "isometric":
|
| 1341 |
+
tile_w, tile_h = 128, 64
|
| 1342 |
+
for row in range(-2, height // (tile_h // 2) + 3):
|
| 1343 |
+
for col in range(-2, width // tile_w + 3):
|
| 1344 |
+
cx = col * tile_w + (row % 2) * (tile_w // 2)
|
| 1345 |
+
cy = row * (tile_h // 2)
|
| 1346 |
+
value = 125 + ((row + col) % 3) * 30
|
| 1347 |
+
diamond = [(cx, cy), (cx + tile_w // 2, cy + tile_h // 2), (cx, cy + tile_h), (cx - tile_w // 2, cy + tile_h // 2)]
|
| 1348 |
+
draw.polygon(diamond, fill=value, outline=95)
|
| 1349 |
+
elif camera == "side_view":
|
| 1350 |
+
horizon = int(height * 0.55)
|
| 1351 |
+
for y in range(height):
|
| 1352 |
+
value = int(230 - 125 * (y / max(1, height - 1)))
|
| 1353 |
+
draw.line((0, y, width, y), fill=value)
|
| 1354 |
+
draw.rectangle((0, horizon, width, min(height, horizon + 18)), fill=70)
|
| 1355 |
+
for x in range(0, width, 160):
|
| 1356 |
+
draw.rectangle((x, horizon - 90, x + 75, horizon), fill=115)
|
| 1357 |
+
else:
|
| 1358 |
+
# Front/first-person guide: converging floor rays and a real horizon are intentional.
|
| 1359 |
+
horizon = int(height * 0.48)
|
| 1360 |
+
for y in range(height):
|
| 1361 |
+
value = 225 if y < horizon else int(180 - 95 * ((y - horizon) / max(1, height - horizon)))
|
| 1362 |
+
draw.line((0, y, width, y), fill=value)
|
| 1363 |
+
vanishing = (width // 2, horizon)
|
| 1364 |
+
for x in range(-width, width * 2, width // 6):
|
| 1365 |
+
draw.line((x, height, *vanishing), fill=70, width=5)
|
| 1366 |
+
|
| 1367 |
+
return image.convert("RGB")
|
| 1368 |
+
|
| 1369 |
+
|
| 1370 |
+
def controlnet_background_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None, str | None]:
|
| 1371 |
+
"""Run SDXL Depth ControlNet when a CUDA Space explicitly enables it."""
|
| 1372 |
+
global CONTROLNET_PIPE, CONTROLNET_ERROR
|
| 1373 |
+
if CONTROLNET_ERROR:
|
| 1374 |
+
return None, CONTROLNET_ERROR
|
| 1375 |
+
try:
|
| 1376 |
+
import torch
|
| 1377 |
+
from diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline
|
| 1378 |
+
|
| 1379 |
+
if not torch.cuda.is_available():
|
| 1380 |
+
return None, "SDXL ControlNet requires CUDA; CPU Basic is not a production-viable runtime"
|
| 1381 |
+
|
| 1382 |
+
if CONTROLNET_PIPE is None:
|
| 1383 |
+
controlnet = ControlNetModel.from_pretrained(
|
| 1384 |
+
CONTROLNET_MODEL,
|
| 1385 |
+
variant="fp16",
|
| 1386 |
+
use_safetensors=True,
|
| 1387 |
+
torch_dtype=torch.float16,
|
| 1388 |
+
)
|
| 1389 |
+
CONTROLNET_PIPE = StableDiffusionXLControlNetPipeline.from_pretrained(
|
| 1390 |
+
CONTROLNET_BASE_MODEL,
|
| 1391 |
+
controlnet=controlnet,
|
| 1392 |
+
variant="fp16",
|
| 1393 |
+
use_safetensors=True,
|
| 1394 |
+
torch_dtype=torch.float16,
|
| 1395 |
+
)
|
| 1396 |
+
CONTROLNET_PIPE.enable_model_cpu_offload()
|
| 1397 |
+
|
| 1398 |
+
width, height = 1024, 576
|
| 1399 |
+
seed = abs(hash(f"controlnet|{spec.role}|{spec.prompt}|{index}|{run_id}")) % 2147483647
|
| 1400 |
+
generator = torch.Generator(device="cuda").manual_seed(seed)
|
| 1401 |
+
image = CONTROLNET_PIPE(
|
| 1402 |
+
prompt=spec.prompt,
|
| 1403 |
+
negative_prompt=diffusion_negative_prompt(spec),
|
| 1404 |
+
image=build_camera_control_image(spec, width, height),
|
| 1405 |
+
width=width,
|
| 1406 |
+
height=height,
|
| 1407 |
+
num_inference_steps=CONTROLNET_STEPS,
|
| 1408 |
+
guidance_scale=6.5,
|
| 1409 |
+
controlnet_conditioning_scale=0.65,
|
| 1410 |
+
generator=generator,
|
| 1411 |
+
).images[0]
|
| 1412 |
+
return polish_diffusion_asset(image, spec), None
|
| 1413 |
+
except Exception as exc:
|
| 1414 |
+
CONTROLNET_ERROR = short_error(exc)
|
| 1415 |
+
return None, CONTROLNET_ERROR
|
| 1416 |
+
|
| 1417 |
+
|
| 1418 |
def polish_diffusion_asset(image: Image.Image, spec: AssetSpec) -> bytes:
|
| 1419 |
image = image.convert("RGBA")
|
| 1420 |
if not is_background_spec(spec):
|
|
|
|
| 1507 |
return None, short_error(exc)
|
| 1508 |
|
| 1509 |
|
| 1510 |
+
def hf_structure_background_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None, str | None]:
|
| 1511 |
+
"""Refine the deterministic layout through a routed image-to-image provider."""
|
| 1512 |
+
if not USE_HF_STRUCTURE_PROVIDER:
|
| 1513 |
+
return None, None
|
| 1514 |
+
if not HF_TOKEN:
|
| 1515 |
+
return None, "HF_TOKEN is required for the routed structure-aware image API"
|
| 1516 |
+
if InferenceClient is None:
|
| 1517 |
+
return None, "huggingface_hub is not installed in this Space"
|
| 1518 |
+
|
| 1519 |
+
try:
|
| 1520 |
+
source_png = local_asset_png(spec, index, run_id)
|
| 1521 |
+
camera_instruction = camera_prompt_contract(spec.camera, True)
|
| 1522 |
+
edit_prompt = (
|
| 1523 |
+
"Polish this supplied game-background layout into production-quality 2D game art. "
|
| 1524 |
+
"Preserve its camera projection, walkable-space geometry, and obstacle placement. "
|
| 1525 |
+
f"{camera_instruction} Match this creative brief: {spec.prompt}"
|
| 1526 |
+
)
|
| 1527 |
+
client = InferenceClient(provider=HF_STRUCTURE_PROVIDER, api_key=HF_TOKEN)
|
| 1528 |
+
image = client.image_to_image(
|
| 1529 |
+
source_png,
|
| 1530 |
+
prompt=edit_prompt,
|
| 1531 |
+
model=HF_STRUCTURE_MODEL,
|
| 1532 |
+
)
|
| 1533 |
+
return polish_diffusion_asset(image, spec), None
|
| 1534 |
+
except Exception as exc:
|
| 1535 |
+
return None, short_error(exc)
|
| 1536 |
+
|
| 1537 |
+
|
| 1538 |
def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str, str | None, str]:
|
| 1539 |
+
is_background = is_background_spec(spec)
|
| 1540 |
+
if not is_background and not USE_DIFFUSION_FOR_SPRITES and not USE_HF_IMAGE_PROVIDER:
|
| 1541 |
png_content = local_asset_png(spec, index, run_id)
|
| 1542 |
return (
|
| 1543 |
png_bytes_to_data_uri(png_content),
|
|
|
|
| 1545 |
None,
|
| 1546 |
"shape-aware 2D sprite generator",
|
| 1547 |
)
|
| 1548 |
+
background_models_enabled = any(
|
| 1549 |
+
(
|
| 1550 |
+
USE_CONTROLNET_FOR_BACKGROUNDS,
|
| 1551 |
+
USE_HF_STRUCTURE_PROVIDER,
|
| 1552 |
+
USE_DIFFUSION_FOR_BACKGROUNDS,
|
| 1553 |
+
USE_HF_IMAGE_PROVIDER,
|
| 1554 |
+
)
|
| 1555 |
+
)
|
| 1556 |
+
if is_background and not background_models_enabled:
|
| 1557 |
png_content = local_asset_png(spec, index, run_id)
|
| 1558 |
return (
|
| 1559 |
png_bytes_to_data_uri(png_content),
|
|
|
|
| 1562 |
"scene-safe 2D background generator",
|
| 1563 |
)
|
| 1564 |
|
| 1565 |
+
png_content = None
|
| 1566 |
+
source = ""
|
| 1567 |
+
errors: list[str] = []
|
| 1568 |
+
|
| 1569 |
+
if is_background and USE_CONTROLNET_FOR_BACKGROUNDS:
|
| 1570 |
+
png_content, error = controlnet_background_png(spec, index, run_id)
|
| 1571 |
+
source = f"{CONTROLNET_BASE_MODEL} + {CONTROLNET_MODEL}"
|
| 1572 |
+
if error:
|
| 1573 |
+
errors.append(f"ControlNet: {error}")
|
| 1574 |
+
|
| 1575 |
+
if png_content is None and is_background and USE_HF_STRUCTURE_PROVIDER:
|
| 1576 |
+
png_content, error = hf_structure_background_png(spec, index, run_id)
|
| 1577 |
+
source = f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}"
|
| 1578 |
+
if error:
|
| 1579 |
+
errors.append(f"structure API: {error}")
|
| 1580 |
+
|
| 1581 |
+
use_free_diffusion = USE_DIFFUSION_FOR_BACKGROUNDS if is_background else USE_DIFFUSION_FOR_SPRITES
|
| 1582 |
+
if png_content is None and use_free_diffusion:
|
| 1583 |
+
png_content, error = free_diffusion_png(spec, index, run_id)
|
| 1584 |
+
source = FREE_IMAGE_MODEL
|
| 1585 |
+
if error:
|
| 1586 |
+
errors.append(f"local diffusion: {error}")
|
| 1587 |
+
|
| 1588 |
+
if png_content is None and USE_HF_IMAGE_PROVIDER:
|
| 1589 |
+
png_content, error = hf_image_png(spec, index, run_id)
|
| 1590 |
source = HF_IMAGE_MODEL
|
| 1591 |
+
if error:
|
| 1592 |
+
errors.append(f"text-to-image API: {error}")
|
| 1593 |
+
|
| 1594 |
if png_content is None:
|
| 1595 |
png_content = local_asset_png(spec, index, run_id)
|
| 1596 |
source = "local procedural fallback"
|
| 1597 |
return (
|
| 1598 |
png_bytes_to_data_uri(png_content),
|
| 1599 |
write_gallery_image(png_content, spec.role),
|
| 1600 |
+
"; ".join(errors) if source == "local procedural fallback" and errors else None,
|
| 1601 |
source,
|
| 1602 |
)
|
| 1603 |
|
|
|
|
| 1754 |
lowered = image_model.lower()
|
| 1755 |
if "shape-aware" in lowered or "scene-safe" in lowered or "procedural" in lowered:
|
| 1756 |
return "procedural/heuristic output—predictable structure, limited open-ended semantic fidelity"
|
| 1757 |
+
if "controlnet" in lowered:
|
| 1758 |
+
return "depth-controlled diffusion—stronger camera structure; still requires visual perspective review"
|
| 1759 |
+
if "via fal-ai" in lowered or "via replicate" in lowered:
|
| 1760 |
+
return "routed image-to-image refinement—preserves a layout guide but consumes account inference credit"
|
| 1761 |
return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
|
| 1762 |
|
| 1763 |
|
| 1764 |
def model_configuration_summary() -> str:
|
| 1765 |
prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
|
| 1766 |
+
sprite_chain = []
|
| 1767 |
+
if USE_DIFFUSION_FOR_SPRITES:
|
| 1768 |
+
sprite_chain.append(FREE_IMAGE_MODEL)
|
| 1769 |
+
if USE_HF_IMAGE_PROVIDER:
|
| 1770 |
+
sprite_chain.append(HF_IMAGE_MODEL)
|
| 1771 |
+
sprite_chain.append("shape-aware procedural fallback")
|
| 1772 |
+
sprite_source = " → ".join(sprite_chain)
|
| 1773 |
+
background_chain = []
|
| 1774 |
+
if USE_CONTROLNET_FOR_BACKGROUNDS:
|
| 1775 |
+
background_chain.append(f"{CONTROLNET_BASE_MODEL} + {CONTROLNET_MODEL}")
|
| 1776 |
+
if USE_HF_STRUCTURE_PROVIDER:
|
| 1777 |
+
background_chain.append(f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}")
|
| 1778 |
+
if USE_DIFFUSION_FOR_BACKGROUNDS:
|
| 1779 |
+
background_chain.append(FREE_IMAGE_MODEL)
|
| 1780 |
+
if USE_HF_IMAGE_PROVIDER:
|
| 1781 |
+
background_chain.append(HF_IMAGE_MODEL)
|
| 1782 |
+
background_chain.append("camera-safe procedural fallback")
|
| 1783 |
+
background_source = " → ".join(background_chain)
|
| 1784 |
+
remote_sources = []
|
| 1785 |
+
if USE_HF_STRUCTURE_PROVIDER:
|
| 1786 |
+
remote_sources.append(f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}")
|
| 1787 |
+
if USE_HF_IMAGE_PROVIDER:
|
| 1788 |
+
remote_sources.append(HF_IMAGE_MODEL)
|
| 1789 |
+
remote_fallback = " → ".join(remote_sources) if remote_sources else "disabled"
|
| 1790 |
return (
|
| 1791 |
"**Configured model pipeline:** "
|
| 1792 |
f"prompts: `{prompt_source}` · sprites: `{sprite_source}` · backgrounds: `{background_source}` · "
|
| 1793 |
f"remote image fallback: `{remote_fallback}`. "
|
| 1794 |
+
"Every background carries an explicit camera contract. The free procedural path enforces that contract "
|
| 1795 |
+
"deterministically; model-based refinements are more varied and still require visual perspective review."
|
| 1796 |
)
|
| 1797 |
|
| 1798 |
|