LeafCat79 commited on
Commit
909497f
·
verified ·
1 Parent(s): 09186f2

Enforce camera perspective in background generation

Browse files
Files changed (2) hide show
  1. README.md +41 -3
  2. app.py +441 -31
README.md CHANGED
@@ -31,10 +31,48 @@ If a requested role has no deterministic hook, assets are still generated for re
31
 
32
  Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
33
 
34
- The UI reports the configured prompt and image pipeline and labels every role with its actual source. Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
 
 
35
 
36
  The app combines the game type, perspective, theme, and pasted code into explicit per-role image prompts, then generates PNG assets and embeds contract-compatible assets as base64 data URIs.
37
 
38
- If the Space has an `HF_TOKEN` secret, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
40
- It then tries the image model in `HF_IMAGE_MODEL` for each role. The default image model is `black-forest-labs/FLUX.1-schnell`. Without `HF_TOKEN`, or if an API call fails, the app uses a local style-aware Pillow fallback that varies watercolor, vector/neon, claymation, oil-painting, anime, pixel-art looks, and common game sprite silhouettes.
 
31
 
32
  Generated sprites receive automated dimension, alpha-channel, and corner-transparency checks. These checks do not establish semantic or artistic correctness, so every role also has a manual approval control and per-role regeneration.
33
 
34
+ The UI reports the configured prompt and image pipeline and labels every role with its actual source. Every asset now carries a normalized camera contract (`top_down`, `side_view`, `isometric`, `first_person`, or `front_view`). Top-down background prompts explicitly forbid a sky, horizon, eye-level view, and front-facing facades. The free procedural renderer also routes top-down and isometric backgrounds through dedicated geometry, so camera compliance does not depend only on prompt wording.
35
+
36
+ Procedural generators provide predictable structure and real transparency but have limited open-ended semantic fidelity. Diffusion models provide more visual variety but are more likely to miss perspective, subject, background, or edge constraints; sprite outputs therefore receive alpha post-processing and still require visual approval. Background and sprite diffusion runs use different negative prompts so valid characters and vehicles are not accidentally excluded from sprite generation.
37
 
38
  The app combines the game type, perspective, theme, and pasted code into explicit per-role image prompts, then generates PNG assets and embeds contract-compatible assets as base64 data URIs.
39
 
40
+ If the Space has an `HF_TOKEN` secret and `USE_HF_PROMPT_PROVIDER=1`, it first uses the text model in `HF_PROMPT_MODEL` to interpret the code, roles, and theme into image prompts. The default prompt model is `Qwen/Qwen2.5-Coder-7B-Instruct`. The camera contract is appended after model interpretation, so the model cannot silently remove it.
41
+
42
+ ## Background generation tiers
43
+
44
+ The default tier is deliberately free and deterministic:
45
+
46
+ ```text
47
+ camera-aware procedural background
48
+ ```
49
+
50
+ It runs on CPU Basic and is the active production-safe fallback. It fixes camera geometry, but it cannot match the semantic richness of a large image model.
51
+
52
+ For a GPU Space, enable the production depth-control path:
53
+
54
+ ```text
55
+ USE_CONTROLNET_FOR_BACKGROUNDS=1
56
+ CONTROLNET_BASE_MODEL=stabilityai/stable-diffusion-xl-base-1.0
57
+ CONTROLNET_MODEL=diffusers/controlnet-depth-sdxl-1.0-small
58
+ CONTROLNET_STEPS=24
59
+ ```
60
+
61
+ This constructs a camera-specific depth-like control image and runs SDXL with Depth ControlNet before falling back. It intentionally refuses to run on CPU-only hardware: loading SDXL on CPU Basic is not a credible interactive production configuration. The small ControlNet adapter is experimental, so perspective still needs human review.
62
+
63
+ For an API-backed refinement that works from a CPU Space, enable:
64
+
65
+ ```text
66
+ USE_HF_STRUCTURE_PROVIDER=1
67
+ HF_STRUCTURE_MODEL=black-forest-labs/FLUX.2-klein-4B
68
+ HF_STRUCTURE_PROVIDER=fal-ai
69
+ HF_TOKEN=<Space secret with Inference Providers permission>
70
+ ```
71
+
72
+ This sends the deterministic background as an image-to-image layout reference, asks the model to preserve its projection and walkable geometry, and falls back safely if the request fails. `FLUX.2-klein-4B` is Apache-2.0 licensed and available through a Hugging Face Inference Provider. It is not unlimited-free infrastructure: a free Hugging Face user currently receives only a small monthly inference credit, after which purchased credits are required. Keep the flag disabled unless paid API usage and token handling are acceptable.
73
+
74
+ The older text-to-image fallback remains opt-in through `USE_HF_IMAGE_PROVIDER=1` and `HF_IMAGE_MODEL` (default `black-forest-labs/FLUX.1-schnell`). Unlike the structure-refinement tier, text-to-image receives no layout image and is therefore the weakest model option for strict camera compliance.
75
+
76
+ If every enabled model path fails, the app uses the local style-aware Pillow fallback, which varies watercolor, vector/neon, claymation, oil-painting, anime, pixel-art looks, and common game sprite silhouettes.
77
 
78
+ `black-forest-labs/FLUX.1-Depth-dev` is not configured here. Its gated 12B weights, non-commercial development license, and lack of a Hugging Face Inference Provider make it a poor fit for this Space and impossible to run credibly on CPU Basic. Black Forest Labs offers Depth Pro through its own API, which requires a separate provider account and billing rather than only the current free Hugging Face account.
app.py CHANGED
@@ -92,6 +92,7 @@ class AssetSpec:
92
  filename: str
93
  width: int
94
  height: int
 
95
 
96
 
97
  @dataclass
@@ -121,11 +122,20 @@ USE_HF_PROMPT_PROVIDER = os.environ.get("USE_HF_PROMPT_PROVIDER", "0") == "1"
121
  USE_HF_IMAGE_PROVIDER = os.environ.get("USE_HF_IMAGE_PROVIDER", "0") == "1"
122
  USE_DIFFUSION_FOR_SPRITES = os.environ.get("USE_DIFFUSION_FOR_SPRITES", "0") == "1"
123
  USE_DIFFUSION_FOR_BACKGROUNDS = os.environ.get("USE_DIFFUSION_FOR_BACKGROUNDS", "0") == "1"
 
 
 
 
 
 
 
124
  HF_IMAGE_MODEL = os.environ.get("HF_IMAGE_MODEL", "black-forest-labs/FLUX.1-schnell")
125
  HF_PROMPT_MODEL = os.environ.get("HF_PROMPT_MODEL", "Qwen/Qwen2.5-Coder-7B-Instruct:fastest")
126
  HF_PROMPT_ENDPOINT = os.environ.get("HF_PROMPT_ENDPOINT", "https://router.huggingface.co/v1/chat/completions")
127
  FREE_DIFFUSION_PIPE = None
128
  FREE_DIFFUSION_ERROR = None
 
 
129
 
130
 
131
  def slugify(value: str) -> str:
@@ -133,6 +143,44 @@ def slugify(value: str) -> str:
133
  return value or "asset"
134
 
135
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
  def interpret_style_hint(style_hint: str) -> StylePlan:
137
  text = (style_hint or "").lower()
138
  tags: list[str] = []
@@ -198,24 +246,20 @@ def interpret_style_hint(style_hint: str) -> StylePlan:
198
  lighting = "balanced game lighting"
199
  linework = "clear readable silhouette and controlled edges"
200
 
201
- if has("top-down", "top down", "overhead"):
202
- camera = "top-down readable game camera"
203
- elif has("platformer", "side-scroller", "side scroller"):
204
- camera = "side-view platformer camera"
205
- elif has("isometric", "3/4 view", "three-quarter"):
206
- camera = "consistent isometric three-quarter game camera"
207
- elif has("first-person", "first person"):
208
- camera = "first-person game camera"
209
- elif has("front-facing", "front facing"):
210
- camera = "front-facing game camera"
211
- else:
212
- camera = "game-ready camera angle matching the role"
213
 
214
  return StylePlan(medium, palette, texture, lighting, linework, camera, tuple(tags))
215
 
216
 
217
  def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
218
  plan = interpret_style_hint(style_hint)
 
219
  slug = slugify(role)
220
  is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
221
  if is_background:
@@ -234,6 +278,7 @@ def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
234
  f"{role} asset: {prompt}. Creative brief: {style_hint}. "
235
  f"Style interpretation: {plan.medium}; {plan.palette}; "
236
  f"{plan.texture}; {plan.lighting}; {plan.linework}; {plan.camera}. "
 
237
  f"{asset_instruction} "
238
  "Game asset, readable at small size, no text, no watermark."
239
  )
@@ -397,8 +442,25 @@ def parse_assets(raw_roles: str, style_hint: str, prompt_map: dict[str, str] | N
397
  is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
398
  width, height = (800, 450) if is_background else (128, 128)
399
  filename = f"sprite_{slug}.png"
400
- full_prompt = (prompt_map or {}).get(role) or build_asset_prompt(role, prompt, style_hint)
401
- specs.append(AssetSpec(role=role, prompt=full_prompt, filename=filename, width=width, height=height))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
402
  return specs
403
 
404
 
@@ -577,15 +639,170 @@ def apply_style_finish(content: bytes, spec: AssetSpec, rng: random.Random) -> b
577
  return out.getvalue()
578
 
579
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
580
  def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
581
  colors = palette_for(spec.prompt)
582
  image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
583
  draw = ImageDraw.Draw(image, "RGBA")
584
  prompt = spec.prompt.lower()
585
 
 
 
 
 
 
586
  if (
587
  any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "hazard", "grid", "shooter"))
588
- and not any(word in prompt for word in ("space", "starfield", "nebula", "galaxy", "cosmic"))
589
  ):
590
  image = Image.new("RGBA", (spec.width, spec.height), (18, 22, 31, 255))
591
  draw = ImageDraw.Draw(image, "RGBA")
@@ -620,7 +837,7 @@ def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
620
  image.save(out, format="PNG")
621
  return out.getvalue()
622
 
623
- if any(word in prompt for word in ("space", "star", "nebula", "galaxy", "cosmic")):
624
  image = Image.new("RGBA", (spec.width, spec.height), (3, 4, 18, 255))
625
  draw = ImageDraw.Draw(image, "RGBA")
626
  for y in range(spec.height):
@@ -1085,10 +1302,119 @@ def diffusion_negative_prompt(spec: AssetSpec) -> str:
1085
  "multiple objects, cropped subject, close-up surface, fabric swatch, text, watermark"
1086
  )
1087
  if is_background_spec(spec):
1088
- return common + ", person, character, player, hero, creature, monster, vehicle, mascot, foreground subject"
 
 
 
 
 
 
 
 
 
1089
  return common + ", landscape, environment scene, room, horizon, opaque background, drop shadow"
1090
 
1091
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1092
  def polish_diffusion_asset(image: Image.Image, spec: AssetSpec) -> bytes:
1093
  image = image.convert("RGBA")
1094
  if not is_background_spec(spec):
@@ -1181,8 +1507,37 @@ def hf_image_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None
1181
  return None, short_error(exc)
1182
 
1183
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1184
  def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str, str | None, str]:
1185
- if not is_background_spec(spec) and not USE_DIFFUSION_FOR_SPRITES:
 
1186
  png_content = local_asset_png(spec, index, run_id)
1187
  return (
1188
  png_bytes_to_data_uri(png_content),
@@ -1190,7 +1545,15 @@ def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str,
1190
  None,
1191
  "shape-aware 2D sprite generator",
1192
  )
1193
- if is_background_spec(spec) and not USE_DIFFUSION_FOR_BACKGROUNDS:
 
 
 
 
 
 
 
 
1194
  png_content = local_asset_png(spec, index, run_id)
1195
  return (
1196
  png_bytes_to_data_uri(png_content),
@@ -1199,20 +1562,42 @@ def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str,
1199
  "scene-safe 2D background generator",
1200
  )
1201
 
1202
- png_content, free_error = free_diffusion_png(spec, index, run_id)
1203
- source = FREE_IMAGE_MODEL
1204
- error = free_error
1205
- if png_content is None:
1206
- png_content, hf_error = hf_image_png(spec, index, run_id)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1207
  source = HF_IMAGE_MODEL
1208
- error = hf_error or free_error
 
 
1209
  if png_content is None:
1210
  png_content = local_asset_png(spec, index, run_id)
1211
  source = "local procedural fallback"
1212
  return (
1213
  png_bytes_to_data_uri(png_content),
1214
  write_gallery_image(png_content, spec.role),
1215
- error if source == "local procedural fallback" else None,
1216
  source,
1217
  )
1218
 
@@ -1369,20 +1754,45 @@ def model_source_note(image_model: str) -> str:
1369
  lowered = image_model.lower()
1370
  if "shape-aware" in lowered or "scene-safe" in lowered or "procedural" in lowered:
1371
  return "procedural/heuristic output—predictable structure, limited open-ended semantic fidelity"
 
 
 
 
1372
  return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
1373
 
1374
 
1375
  def model_configuration_summary() -> str:
1376
  prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
1377
- sprite_source = FREE_IMAGE_MODEL if USE_DIFFUSION_FOR_SPRITES else "shape-aware procedural generator"
1378
- background_source = FREE_IMAGE_MODEL if USE_DIFFUSION_FOR_BACKGROUNDS else "scene-safe procedural generator"
1379
- remote_fallback = HF_IMAGE_MODEL if USE_HF_IMAGE_PROVIDER else "disabled"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1380
  return (
1381
  "**Configured model pipeline:** "
1382
  f"prompts: `{prompt_source}` · sprites: `{sprite_source}` · backgrounds: `{background_source}` · "
1383
  f"remote image fallback: `{remote_fallback}`. "
1384
- "Procedural output is structurally predictable but semantically limited; diffusion output is more varied "
1385
- "and therefore requires stricter visual review."
1386
  )
1387
 
1388
 
 
92
  filename: str
93
  width: int
94
  height: int
95
+ camera: str = "auto"
96
 
97
 
98
  @dataclass
 
122
  USE_HF_IMAGE_PROVIDER = os.environ.get("USE_HF_IMAGE_PROVIDER", "0") == "1"
123
  USE_DIFFUSION_FOR_SPRITES = os.environ.get("USE_DIFFUSION_FOR_SPRITES", "0") == "1"
124
  USE_DIFFUSION_FOR_BACKGROUNDS = os.environ.get("USE_DIFFUSION_FOR_BACKGROUNDS", "0") == "1"
125
+ USE_CONTROLNET_FOR_BACKGROUNDS = os.environ.get("USE_CONTROLNET_FOR_BACKGROUNDS", "0") == "1"
126
+ CONTROLNET_BASE_MODEL = os.environ.get("CONTROLNET_BASE_MODEL", "stabilityai/stable-diffusion-xl-base-1.0")
127
+ CONTROLNET_MODEL = os.environ.get("CONTROLNET_MODEL", "diffusers/controlnet-depth-sdxl-1.0-small")
128
+ CONTROLNET_STEPS = int(os.environ.get("CONTROLNET_STEPS", "24"))
129
+ USE_HF_STRUCTURE_PROVIDER = os.environ.get("USE_HF_STRUCTURE_PROVIDER", "0") == "1"
130
+ HF_STRUCTURE_MODEL = os.environ.get("HF_STRUCTURE_MODEL", "black-forest-labs/FLUX.2-klein-4B")
131
+ HF_STRUCTURE_PROVIDER = os.environ.get("HF_STRUCTURE_PROVIDER", "fal-ai")
132
  HF_IMAGE_MODEL = os.environ.get("HF_IMAGE_MODEL", "black-forest-labs/FLUX.1-schnell")
133
  HF_PROMPT_MODEL = os.environ.get("HF_PROMPT_MODEL", "Qwen/Qwen2.5-Coder-7B-Instruct:fastest")
134
  HF_PROMPT_ENDPOINT = os.environ.get("HF_PROMPT_ENDPOINT", "https://router.huggingface.co/v1/chat/completions")
135
  FREE_DIFFUSION_PIPE = None
136
  FREE_DIFFUSION_ERROR = None
137
+ CONTROLNET_PIPE = None
138
+ CONTROLNET_ERROR = None
139
 
140
 
141
  def slugify(value: str) -> str:
 
143
  return value or "asset"
144
 
145
 
146
+ def normalize_camera(text: str) -> str:
147
+ """Convert UI and prompt camera wording into a stable generation contract."""
148
+ lowered = (text or "").lower()
149
+ explicit = re.search(r"required camera perspective:\s*([^.]*)", lowered)
150
+ camera_text = explicit.group(1) if explicit else lowered
151
+ if any(word in camera_text for word in ("top-down", "top down", "overhead", "bird's-eye", "birds-eye")):
152
+ return "top_down"
153
+ if any(word in camera_text for word in ("isometric", "3/4 view", "three-quarter")):
154
+ return "isometric"
155
+ if any(word in camera_text for word in ("side view", "side-view", "side-scroller", "side scroller", "platformer")):
156
+ return "side_view"
157
+ if any(word in camera_text for word in ("first-person", "first person")):
158
+ return "first_person"
159
+ if any(word in camera_text for word in ("front-facing", "front facing", "front view")):
160
+ return "front_view"
161
+ return "auto"
162
+
163
+
164
+ def camera_prompt_contract(camera: str, is_background: bool) -> str:
165
+ if camera == "top_down":
166
+ if is_background:
167
+ return (
168
+ "Strict orthographic overhead composition viewed straight down. Show roofs, ground, paths, and "
169
+ "objects from above. No visible sky, no horizon line, no eye-level view, no front-facing facade, "
170
+ "and no vanishing-point perspective. Keep traversal space readable across the whole frame."
171
+ )
172
+ return "Strict overhead view of the subject from directly above; not front-facing and not side-view."
173
+ if camera == "isometric":
174
+ return "Consistent isometric three-quarter projection with parallel axes and no eye-level horizon."
175
+ if camera == "side_view":
176
+ return "Strict side-view profile composition suitable for a 2D platformer."
177
+ if camera == "first_person":
178
+ return "First-person viewpoint with environment geometry aligned to the player's eye line."
179
+ if camera == "front_view":
180
+ return "Front-facing composition with the subject or environment oriented toward the camera."
181
+ return "Use one internally consistent game camera angle appropriate to the role."
182
+
183
+
184
  def interpret_style_hint(style_hint: str) -> StylePlan:
185
  text = (style_hint or "").lower()
186
  tags: list[str] = []
 
246
  lighting = "balanced game lighting"
247
  linework = "clear readable silhouette and controlled edges"
248
 
249
+ camera = {
250
+ "top_down": "top-down readable game camera",
251
+ "side_view": "side-view platformer camera",
252
+ "isometric": "consistent isometric three-quarter game camera",
253
+ "first_person": "first-person game camera",
254
+ "front_view": "front-facing game camera",
255
+ }.get(normalize_camera(style_hint), "game-ready camera angle matching the role")
 
 
 
 
 
256
 
257
  return StylePlan(medium, palette, texture, lighting, linework, camera, tuple(tags))
258
 
259
 
260
  def build_asset_prompt(role: str, prompt: str, style_hint: str) -> str:
261
  plan = interpret_style_hint(style_hint)
262
+ camera = normalize_camera(f"{style_hint} {prompt}")
263
  slug = slugify(role)
264
  is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
265
  if is_background:
 
278
  f"{role} asset: {prompt}. Creative brief: {style_hint}. "
279
  f"Style interpretation: {plan.medium}; {plan.palette}; "
280
  f"{plan.texture}; {plan.lighting}; {plan.linework}; {plan.camera}. "
281
+ f"Camera contract: {camera_prompt_contract(camera, is_background)} "
282
  f"{asset_instruction} "
283
  "Game asset, readable at small size, no text, no watermark."
284
  )
 
442
  is_background = any(word in slug for word in ("background", "backdrop", "scene", "map", "level"))
443
  width, height = (800, 450) if is_background else (128, 128)
444
  filename = f"sprite_{slug}.png"
445
+ camera = normalize_camera(f"{style_hint} {prompt}")
446
+ interpreted_prompt = (prompt_map or {}).get(role)
447
+ if interpreted_prompt:
448
+ full_prompt = (
449
+ f"{interpreted_prompt}. Camera contract: {camera_prompt_contract(camera, is_background)} "
450
+ "Game asset, no text, no watermark."
451
+ )
452
+ else:
453
+ full_prompt = build_asset_prompt(role, prompt, style_hint)
454
+ specs.append(
455
+ AssetSpec(
456
+ role=role,
457
+ prompt=full_prompt,
458
+ filename=filename,
459
+ width=width,
460
+ height=height,
461
+ camera=camera,
462
+ )
463
+ )
464
  return specs
465
 
466
 
 
639
  return out.getvalue()
640
 
641
 
642
+ def finish_png(image: Image.Image) -> bytes:
643
+ out = io.BytesIO()
644
+ image.save(out, format="PNG")
645
+ return out.getvalue()
646
+
647
+
648
+ def is_space_background_prompt(prompt: str) -> bool:
649
+ lowered = prompt.lower()
650
+ return any(
651
+ phrase in lowered
652
+ for phrase in ("outer space", "deep space", "starfield", "star field", "nebula", "galaxy", "cosmic")
653
+ )
654
+
655
+
656
+ def draw_top_down_background(spec: AssetSpec, rng: random.Random) -> bytes:
657
+ """Draw an explicitly horizon-free overhead map for the free default path."""
658
+ prompt = spec.prompt.lower()
659
+ colors = palette_for(spec.prompt)
660
+ width, height = spec.width, spec.height
661
+
662
+ if any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "space station", "shooter")):
663
+ image = Image.new("RGBA", (width, height), (18, 22, 31, 255))
664
+ draw = ImageDraw.Draw(image, "RGBA")
665
+ panel = 64
666
+ for y in range(0, height, panel):
667
+ for x in range(0, width, panel):
668
+ shade = 24 + ((x // panel + y // panel) % 2) * 12
669
+ draw.rectangle(
670
+ (x, y, x + panel - 2, y + panel - 2),
671
+ fill=(shade, shade + 4, shade + 14, 255),
672
+ outline=(58, 74, 92, 170),
673
+ )
674
+ for x in range(0, width, 96):
675
+ draw.line((x, 0, x, height), fill=(34, 215, 236, 95), width=2)
676
+ for y in range(0, height, 96):
677
+ draw.line((0, y, width, y), fill=(34, 215, 236, 95), width=2)
678
+ return finish_png(image)
679
+
680
+ if any(word in prompt for word in ("garden", "flower", "grass", "meadow", "forest", "jungle", "path")):
681
+ image = Image.new("RGBA", (width, height), (64, 132, 70, 255))
682
+ draw = ImageDraw.Draw(image, "RGBA")
683
+ for _ in range(85):
684
+ x, y = rng.randint(0, width), rng.randint(0, height)
685
+ rx, ry = rng.randint(18, 70), rng.randint(12, 48)
686
+ draw.ellipse((x - rx, y - ry, x + rx, y + ry), fill=(78, rng.randint(135, 169), 76, 75))
687
+
688
+ path_points = [
689
+ (-40, int(height * 0.76)),
690
+ (int(width * 0.20), int(height * 0.66)),
691
+ (int(width * 0.43), int(height * 0.52)),
692
+ (int(width * 0.63), int(height * 0.37)),
693
+ (int(width * 0.83), int(height * 0.22)),
694
+ (width + 40, int(height * 0.13)),
695
+ ]
696
+ draw.line(path_points, fill=(112, 91, 59, 155), width=126, joint="curve")
697
+ draw.line(path_points, fill=(198, 169, 112, 255), width=108, joint="curve")
698
+ for _ in range(31):
699
+ edge = rng.choice(("top", "bottom", "left", "right"))
700
+ if edge in ("top", "bottom"):
701
+ x = rng.randint(-24, width + 24)
702
+ y = rng.randint(-28, 94) if edge == "top" else rng.randint(height - 94, height + 28)
703
+ else:
704
+ x = rng.randint(-28, 86) if edge == "left" else rng.randint(width - 86, width + 28)
705
+ y = rng.randint(-24, height + 24)
706
+ radius = rng.randint(24, 53)
707
+ draw.ellipse(
708
+ (x - radius, y - radius, x + radius, y + radius),
709
+ fill=(28, rng.randint(82, 116), 48, 255),
710
+ outline=(18, 70, 37, 210),
711
+ width=3,
712
+ )
713
+ inner = int(radius * 0.66)
714
+ draw.ellipse((x - inner, y - inner, x + inner, y + inner), fill=(58, rng.randint(126, 166), 69, 245))
715
+ for _ in range(42):
716
+ x, y = rng.randint(0, width), rng.randint(0, height)
717
+ radius = rng.randint(2, 5)
718
+ draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=rng.choice(colors) + (220,))
719
+ return finish_png(image)
720
+
721
+ if any(word in prompt for word in ("dungeon", "castle", "stone", "crypt")):
722
+ image = Image.new("RGBA", (width, height), (48, 50, 62, 255))
723
+ draw = ImageDraw.Draw(image, "RGBA")
724
+ tile = 54
725
+ for y in range(0, height, tile):
726
+ for x in range(0, width, tile):
727
+ shade = rng.randint(48, 78)
728
+ draw.rectangle(
729
+ (x, y, x + tile - 2, y + tile - 2),
730
+ fill=(shade, shade + 2, shade + 8, 255),
731
+ outline=(27, 29, 38, 190),
732
+ )
733
+ return finish_png(image)
734
+
735
+ if any(word in prompt for word in ("lava", "fire", "volcano")):
736
+ image = Image.new("RGBA", (width, height), (52, 23, 18, 255))
737
+ draw = ImageDraw.Draw(image, "RGBA")
738
+ for _ in range(24):
739
+ x, y = rng.randint(-80, width), rng.randint(0, height)
740
+ points = [(x, y)]
741
+ for _ in range(rng.randint(2, 5)):
742
+ x += rng.randint(45, 125)
743
+ y += rng.randint(-45, 45)
744
+ points.append((x, y))
745
+ draw.line(points, fill=(255, 103, 35, 235), width=rng.randint(7, 15), joint="curve")
746
+ draw.line(points, fill=(255, 202, 64, 210), width=rng.randint(2, 5), joint="curve")
747
+ return finish_png(image)
748
+
749
+ if any(word in prompt for word in ("ocean", "sea", "water", "underwater")):
750
+ image = Image.new("RGBA", (width, height), (18, 114, 158, 255))
751
+ draw = ImageDraw.Draw(image, "RGBA")
752
+ for _ in range(42):
753
+ x, y = rng.randint(0, width), rng.randint(0, height)
754
+ rx, ry = rng.randint(18, 75), rng.randint(7, 25)
755
+ draw.ellipse((x - rx, y - ry, x + rx, y + ry), outline=(176, 236, 230, 95), width=3)
756
+ return finish_png(image)
757
+
758
+ image = Image.new("RGBA", (width, height), colors[2] + (255,))
759
+ draw = ImageDraw.Draw(image, "RGBA")
760
+ for _ in range(75):
761
+ x, y = rng.randint(0, width), rng.randint(0, height)
762
+ radius = rng.randint(8, 52)
763
+ draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=rng.choice(colors) + (rng.randint(45, 105),))
764
+ return finish_png(image)
765
+
766
+
767
+ def draw_isometric_background(spec: AssetSpec, rng: random.Random) -> bytes:
768
+ """Draw a stable diamond-grid projection instead of an accidental horizon view."""
769
+ colors = palette_for(spec.prompt)
770
+ image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
771
+ draw = ImageDraw.Draw(image, "RGBA")
772
+ tile_w, tile_h = 96, 48
773
+ for row in range(-2, spec.height // tile_h + 4):
774
+ for col in range(-2, spec.width // tile_w + 4):
775
+ cx = col * tile_w + (row % 2) * (tile_w // 2)
776
+ cy = row * (tile_h // 2)
777
+ diamond = [(cx, cy), (cx + tile_w // 2, cy + tile_h // 2), (cx, cy + tile_h), (cx - tile_w // 2, cy + tile_h // 2)]
778
+ shade = colors[(row + col) % len(colors)] + (115,)
779
+ draw.polygon(diamond, fill=shade, outline=colors[3] + (150,))
780
+ for _ in range(14):
781
+ cx, cy = rng.randint(40, spec.width - 40), rng.randint(45, spec.height - 55)
782
+ size = rng.randint(16, 35)
783
+ top = [(cx, cy - size), (cx + size, cy - size // 2), (cx, cy), (cx - size, cy - size // 2)]
784
+ left = [(cx - size, cy - size // 2), (cx, cy), (cx, cy + size), (cx - size, cy + size // 2)]
785
+ right = [(cx + size, cy - size // 2), (cx, cy), (cx, cy + size), (cx + size, cy + size // 2)]
786
+ draw.polygon(left, fill=colors[2] + (235,))
787
+ draw.polygon(right, fill=colors[0] + (235,))
788
+ draw.polygon(top, fill=colors[1] + (245,))
789
+ return finish_png(image)
790
+
791
+
792
  def draw_background(spec: AssetSpec, rng: random.Random) -> bytes:
793
  colors = palette_for(spec.prompt)
794
  image = Image.new("RGBA", (spec.width, spec.height), colors[2] + (255,))
795
  draw = ImageDraw.Draw(image, "RGBA")
796
  prompt = spec.prompt.lower()
797
 
798
+ if spec.camera == "top_down" and not is_space_background_prompt(prompt):
799
+ return draw_top_down_background(spec, rng)
800
+ if spec.camera == "isometric":
801
+ return draw_isometric_background(spec, rng)
802
+
803
  if (
804
  any(word in prompt for word in ("sci-fi", "scifi", "arena", "metal", "hazard", "grid", "shooter"))
805
+ and not is_space_background_prompt(prompt)
806
  ):
807
  image = Image.new("RGBA", (spec.width, spec.height), (18, 22, 31, 255))
808
  draw = ImageDraw.Draw(image, "RGBA")
 
837
  image.save(out, format="PNG")
838
  return out.getvalue()
839
 
840
+ if is_space_background_prompt(prompt):
841
  image = Image.new("RGBA", (spec.width, spec.height), (3, 4, 18, 255))
842
  draw = ImageDraw.Draw(image, "RGBA")
843
  for y in range(spec.height):
 
1302
  "multiple objects, cropped subject, close-up surface, fabric swatch, text, watermark"
1303
  )
1304
  if is_background_spec(spec):
1305
+ camera_negative = {
1306
+ "top_down": ", sky, horizon, eye-level view, front view, front-facing facade, vanishing point",
1307
+ "isometric": ", eye-level view, front view, inconsistent perspective, horizon",
1308
+ "side_view": ", overhead view, top-down view, isometric view",
1309
+ }.get(spec.camera, "")
1310
+ return (
1311
+ common
1312
+ + ", person, character, player, hero, creature, monster, vehicle, mascot, foreground subject"
1313
+ + camera_negative
1314
+ )
1315
  return common + ", landscape, environment scene, room, horizon, opaque background, drop shadow"
1316
 
1317
 
1318
+ def build_camera_control_image(spec: AssetSpec, width: int = 1024, height: int = 576) -> Image.Image:
1319
+ """Create a deterministic depth-like layout that encodes the requested camera."""
1320
+ image = Image.new("L", (width, height), 172)
1321
+ draw = ImageDraw.Draw(image)
1322
+ camera = spec.camera
1323
+
1324
+ if camera == "top_down":
1325
+ # Overhead maps use local depth islands and a constant-width traversal path.
1326
+ # There is deliberately no full-width horizontal boundary that could become a horizon.
1327
+ path = [
1328
+ (-60, int(height * 0.78)),
1329
+ (int(width * 0.18), int(height * 0.66)),
1330
+ (int(width * 0.43), int(height * 0.54)),
1331
+ (int(width * 0.68), int(height * 0.34)),
1332
+ (width + 60, int(height * 0.18)),
1333
+ ]
1334
+ draw.line(path, fill=205, width=150, joint="curve")
1335
+ seed = random.Random(f"control|{spec.role}|{spec.prompt}")
1336
+ for _ in range(24):
1337
+ x, y = seed.randint(0, width), seed.randint(0, height)
1338
+ radius = seed.randint(22, 62)
1339
+ draw.ellipse((x - radius, y - radius, x + radius, y + radius), fill=seed.randint(55, 125))
1340
+ elif camera == "isometric":
1341
+ tile_w, tile_h = 128, 64
1342
+ for row in range(-2, height // (tile_h // 2) + 3):
1343
+ for col in range(-2, width // tile_w + 3):
1344
+ cx = col * tile_w + (row % 2) * (tile_w // 2)
1345
+ cy = row * (tile_h // 2)
1346
+ value = 125 + ((row + col) % 3) * 30
1347
+ diamond = [(cx, cy), (cx + tile_w // 2, cy + tile_h // 2), (cx, cy + tile_h), (cx - tile_w // 2, cy + tile_h // 2)]
1348
+ draw.polygon(diamond, fill=value, outline=95)
1349
+ elif camera == "side_view":
1350
+ horizon = int(height * 0.55)
1351
+ for y in range(height):
1352
+ value = int(230 - 125 * (y / max(1, height - 1)))
1353
+ draw.line((0, y, width, y), fill=value)
1354
+ draw.rectangle((0, horizon, width, min(height, horizon + 18)), fill=70)
1355
+ for x in range(0, width, 160):
1356
+ draw.rectangle((x, horizon - 90, x + 75, horizon), fill=115)
1357
+ else:
1358
+ # Front/first-person guide: converging floor rays and a real horizon are intentional.
1359
+ horizon = int(height * 0.48)
1360
+ for y in range(height):
1361
+ value = 225 if y < horizon else int(180 - 95 * ((y - horizon) / max(1, height - horizon)))
1362
+ draw.line((0, y, width, y), fill=value)
1363
+ vanishing = (width // 2, horizon)
1364
+ for x in range(-width, width * 2, width // 6):
1365
+ draw.line((x, height, *vanishing), fill=70, width=5)
1366
+
1367
+ return image.convert("RGB")
1368
+
1369
+
1370
+ def controlnet_background_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None, str | None]:
1371
+ """Run SDXL Depth ControlNet when a CUDA Space explicitly enables it."""
1372
+ global CONTROLNET_PIPE, CONTROLNET_ERROR
1373
+ if CONTROLNET_ERROR:
1374
+ return None, CONTROLNET_ERROR
1375
+ try:
1376
+ import torch
1377
+ from diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline
1378
+
1379
+ if not torch.cuda.is_available():
1380
+ return None, "SDXL ControlNet requires CUDA; CPU Basic is not a production-viable runtime"
1381
+
1382
+ if CONTROLNET_PIPE is None:
1383
+ controlnet = ControlNetModel.from_pretrained(
1384
+ CONTROLNET_MODEL,
1385
+ variant="fp16",
1386
+ use_safetensors=True,
1387
+ torch_dtype=torch.float16,
1388
+ )
1389
+ CONTROLNET_PIPE = StableDiffusionXLControlNetPipeline.from_pretrained(
1390
+ CONTROLNET_BASE_MODEL,
1391
+ controlnet=controlnet,
1392
+ variant="fp16",
1393
+ use_safetensors=True,
1394
+ torch_dtype=torch.float16,
1395
+ )
1396
+ CONTROLNET_PIPE.enable_model_cpu_offload()
1397
+
1398
+ width, height = 1024, 576
1399
+ seed = abs(hash(f"controlnet|{spec.role}|{spec.prompt}|{index}|{run_id}")) % 2147483647
1400
+ generator = torch.Generator(device="cuda").manual_seed(seed)
1401
+ image = CONTROLNET_PIPE(
1402
+ prompt=spec.prompt,
1403
+ negative_prompt=diffusion_negative_prompt(spec),
1404
+ image=build_camera_control_image(spec, width, height),
1405
+ width=width,
1406
+ height=height,
1407
+ num_inference_steps=CONTROLNET_STEPS,
1408
+ guidance_scale=6.5,
1409
+ controlnet_conditioning_scale=0.65,
1410
+ generator=generator,
1411
+ ).images[0]
1412
+ return polish_diffusion_asset(image, spec), None
1413
+ except Exception as exc:
1414
+ CONTROLNET_ERROR = short_error(exc)
1415
+ return None, CONTROLNET_ERROR
1416
+
1417
+
1418
  def polish_diffusion_asset(image: Image.Image, spec: AssetSpec) -> bytes:
1419
  image = image.convert("RGBA")
1420
  if not is_background_spec(spec):
 
1507
  return None, short_error(exc)
1508
 
1509
 
1510
+ def hf_structure_background_png(spec: AssetSpec, index: int, run_id: int) -> tuple[bytes | None, str | None]:
1511
+ """Refine the deterministic layout through a routed image-to-image provider."""
1512
+ if not USE_HF_STRUCTURE_PROVIDER:
1513
+ return None, None
1514
+ if not HF_TOKEN:
1515
+ return None, "HF_TOKEN is required for the routed structure-aware image API"
1516
+ if InferenceClient is None:
1517
+ return None, "huggingface_hub is not installed in this Space"
1518
+
1519
+ try:
1520
+ source_png = local_asset_png(spec, index, run_id)
1521
+ camera_instruction = camera_prompt_contract(spec.camera, True)
1522
+ edit_prompt = (
1523
+ "Polish this supplied game-background layout into production-quality 2D game art. "
1524
+ "Preserve its camera projection, walkable-space geometry, and obstacle placement. "
1525
+ f"{camera_instruction} Match this creative brief: {spec.prompt}"
1526
+ )
1527
+ client = InferenceClient(provider=HF_STRUCTURE_PROVIDER, api_key=HF_TOKEN)
1528
+ image = client.image_to_image(
1529
+ source_png,
1530
+ prompt=edit_prompt,
1531
+ model=HF_STRUCTURE_MODEL,
1532
+ )
1533
+ return polish_diffusion_asset(image, spec), None
1534
+ except Exception as exc:
1535
+ return None, short_error(exc)
1536
+
1537
+
1538
  def generate_asset(spec: AssetSpec, index: int, run_id: int) -> tuple[str, str, str | None, str]:
1539
+ is_background = is_background_spec(spec)
1540
+ if not is_background and not USE_DIFFUSION_FOR_SPRITES and not USE_HF_IMAGE_PROVIDER:
1541
  png_content = local_asset_png(spec, index, run_id)
1542
  return (
1543
  png_bytes_to_data_uri(png_content),
 
1545
  None,
1546
  "shape-aware 2D sprite generator",
1547
  )
1548
+ background_models_enabled = any(
1549
+ (
1550
+ USE_CONTROLNET_FOR_BACKGROUNDS,
1551
+ USE_HF_STRUCTURE_PROVIDER,
1552
+ USE_DIFFUSION_FOR_BACKGROUNDS,
1553
+ USE_HF_IMAGE_PROVIDER,
1554
+ )
1555
+ )
1556
+ if is_background and not background_models_enabled:
1557
  png_content = local_asset_png(spec, index, run_id)
1558
  return (
1559
  png_bytes_to_data_uri(png_content),
 
1562
  "scene-safe 2D background generator",
1563
  )
1564
 
1565
+ png_content = None
1566
+ source = ""
1567
+ errors: list[str] = []
1568
+
1569
+ if is_background and USE_CONTROLNET_FOR_BACKGROUNDS:
1570
+ png_content, error = controlnet_background_png(spec, index, run_id)
1571
+ source = f"{CONTROLNET_BASE_MODEL} + {CONTROLNET_MODEL}"
1572
+ if error:
1573
+ errors.append(f"ControlNet: {error}")
1574
+
1575
+ if png_content is None and is_background and USE_HF_STRUCTURE_PROVIDER:
1576
+ png_content, error = hf_structure_background_png(spec, index, run_id)
1577
+ source = f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}"
1578
+ if error:
1579
+ errors.append(f"structure API: {error}")
1580
+
1581
+ use_free_diffusion = USE_DIFFUSION_FOR_BACKGROUNDS if is_background else USE_DIFFUSION_FOR_SPRITES
1582
+ if png_content is None and use_free_diffusion:
1583
+ png_content, error = free_diffusion_png(spec, index, run_id)
1584
+ source = FREE_IMAGE_MODEL
1585
+ if error:
1586
+ errors.append(f"local diffusion: {error}")
1587
+
1588
+ if png_content is None and USE_HF_IMAGE_PROVIDER:
1589
+ png_content, error = hf_image_png(spec, index, run_id)
1590
  source = HF_IMAGE_MODEL
1591
+ if error:
1592
+ errors.append(f"text-to-image API: {error}")
1593
+
1594
  if png_content is None:
1595
  png_content = local_asset_png(spec, index, run_id)
1596
  source = "local procedural fallback"
1597
  return (
1598
  png_bytes_to_data_uri(png_content),
1599
  write_gallery_image(png_content, spec.role),
1600
+ "; ".join(errors) if source == "local procedural fallback" and errors else None,
1601
  source,
1602
  )
1603
 
 
1754
  lowered = image_model.lower()
1755
  if "shape-aware" in lowered or "scene-safe" in lowered or "procedural" in lowered:
1756
  return "procedural/heuristic output—predictable structure, limited open-ended semantic fidelity"
1757
+ if "controlnet" in lowered:
1758
+ return "depth-controlled diffusion—stronger camera structure; still requires visual perspective review"
1759
+ if "via fal-ai" in lowered or "via replicate" in lowered:
1760
+ return "routed image-to-image refinement—preserves a layout guide but consumes account inference credit"
1761
  return "diffusion output—higher visual variety; verify subject, perspective, alpha edges, and prompt compliance"
1762
 
1763
 
1764
  def model_configuration_summary() -> str:
1765
  prompt_source = HF_PROMPT_MODEL if USE_HF_PROMPT_PROVIDER else "local prompt interpreter"
1766
+ sprite_chain = []
1767
+ if USE_DIFFUSION_FOR_SPRITES:
1768
+ sprite_chain.append(FREE_IMAGE_MODEL)
1769
+ if USE_HF_IMAGE_PROVIDER:
1770
+ sprite_chain.append(HF_IMAGE_MODEL)
1771
+ sprite_chain.append("shape-aware procedural fallback")
1772
+ sprite_source = " → ".join(sprite_chain)
1773
+ background_chain = []
1774
+ if USE_CONTROLNET_FOR_BACKGROUNDS:
1775
+ background_chain.append(f"{CONTROLNET_BASE_MODEL} + {CONTROLNET_MODEL}")
1776
+ if USE_HF_STRUCTURE_PROVIDER:
1777
+ background_chain.append(f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}")
1778
+ if USE_DIFFUSION_FOR_BACKGROUNDS:
1779
+ background_chain.append(FREE_IMAGE_MODEL)
1780
+ if USE_HF_IMAGE_PROVIDER:
1781
+ background_chain.append(HF_IMAGE_MODEL)
1782
+ background_chain.append("camera-safe procedural fallback")
1783
+ background_source = " → ".join(background_chain)
1784
+ remote_sources = []
1785
+ if USE_HF_STRUCTURE_PROVIDER:
1786
+ remote_sources.append(f"{HF_STRUCTURE_MODEL} via {HF_STRUCTURE_PROVIDER}")
1787
+ if USE_HF_IMAGE_PROVIDER:
1788
+ remote_sources.append(HF_IMAGE_MODEL)
1789
+ remote_fallback = " → ".join(remote_sources) if remote_sources else "disabled"
1790
  return (
1791
  "**Configured model pipeline:** "
1792
  f"prompts: `{prompt_source}` · sprites: `{sprite_source}` · backgrounds: `{background_source}` · "
1793
  f"remote image fallback: `{remote_fallback}`. "
1794
+ "Every background carries an explicit camera contract. The free procedural path enforces that contract "
1795
+ "deterministically; model-based refinements are more varied and still require visual perspective review."
1796
  )
1797
 
1798