App back to ZeroGPU (merged in-Space); Modal offline-only

README.md

@@ -46,9 +46,10 @@ schema from a minimal ask.
 - **Student (in the app):** [`Qwen3-4B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) + a published **Case Forge LoRA** adapter.
 - **Teacher (offline only):** a large dense model on Modal that generated the
   synthetic training corpus. It never ships in the app and is not counted below.
-- **Runtime:** the student is **served on Modal** (a deployed warm vLLM class,
-  `case-forge-serve`); the Space itself runs on CPU and calls it. Warm latency ≈ a
-  few seconds per case; the first call after idle pays a cold start (~2 min).
+- **Runtime:** the student runs **in-Space on ZeroGPU** (free) — base Qwen3-4B + the
+  published LoRA, merged at the first GPU call, generating via transformers. **Modal**
+  is used only for the **offline** pipeline (corpus generation, fine-tune, LoRA merge),
+  which is pay-per-use; keeping a GPU warm for serving was not cost-viable.
 - **Quality:** an Opus-4.8 content audit found the first model fabricated sources and
   made numeric errors; the corpus was regenerated with a numeric-auditor pass and the
   model retrained (**v3**) — fabricated sources and severe math errors went to **0/6**
@@ -84,9 +85,9 @@ when generating the corpus and when validating in the app.
 
 **Backyard AI** (a real instructor authoring cases he'll teach) · **Well-Tuned**
 (published fine-tune) · **Tiny Titan** (≤4B) · **Best Agent** (multi-stage
-teacher→audit→student pipeline) · **Modal** (corpus generation, fine-tune, **and**
-serving all run on Modal). (Off-Grid is ambiguous here — own weights, but on rented
-GPU — so not claimed.)
+teacher→audit→student pipeline) · **Modal** (corpus generation, fine-tune and LoRA
+merge run on Modal credits) · **Off-Grid** (app serves its own weights in-Space on
+ZeroGPU — no third-party model API).
 
 ## Run locally
 

core/infer.py

@@ -1,15 +1,17 @@
 """Inference for the fine-tuned student — short request → full case+note JSON.
 
-Runtime = **Modal** (no ZeroGPU for this project). The Gradio app calls a deployed
-warm Modal class (`case-forge-serve` / `CaseForge`, see `pipeline/serve_modal.py`)
-that holds base Qwen3-4B + the `qwen3-4b-v3` LoRA. Modal creds come from the Space
-secrets (MODAL_TOKEN_ID / MODAL_TOKEN_SECRET). With no creds / on failure it falls
-back to a real sample case so the UI is fully testable offline.
+Runtime = **ZeroGPU**, in-Space (free). On the first GPU call we load base
+Qwen3-4B + the published LoRA and `merge_and_unload()` it (folding the adapter into
+the weights removes PEFT overhead → faster decode), then generate via transformers
+under `@spaces.GPU`. `max_new_tokens` is capped so a full case fits ZeroGPU's ~120s
+window. Locally (no `spaces`/CUDA) it falls back to a real sample so the UI works
+offline. Heavy/offline work (corpus gen, training, merge) runs on Modal, not here.
 
 Config (env):
-  CASE_FORGE_MODAL_APP   deployed Modal app name (default case-forge-serve)
-  CASE_FORGE_MODAL_CLS   class name (default CaseForge)
-  CASE_FORGE_DEMO=1      force the demo sample (no Modal call)
+  CASE_FORGE_BASE       base model id
+  CASE_FORGE_ADAPTER    HF repo id of the published LoRA
+  CASE_FORGE_MAX_TOKENS generation cap (default 2800 — fits the ZeroGPU window)
+  CASE_FORGE_DEMO=1     force the demo sample (no model load)
 """
 
 from __future__ import annotations
@@ -19,6 +21,8 @@ import os
 import sys
 from pathlib import Path
 
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+
 _ROOT = Path(__file__).resolve().parent.parent          # case-forge/
 _MONOREPO = _ROOT.parent                                  # build-small-hackathon/
 for _p in (str(_ROOT), str(_MONOREPO)):
@@ -26,25 +30,78 @@ for _p in (str(_ROOT), str(_MONOREPO)):
         sys.path.insert(0, _p)
 
 from data.schema import validate_case                     # noqa: E402
-from pipeline.prompts import Seed                          # noqa: E402
+from pipeline.prompts import Seed, build_minimal_prompt    # noqa: E402
+from shared import gpu                                     # noqa: E402
 
-MODAL_APP = os.environ.get("CASE_FORGE_MODAL_APP", "case-forge-serve")
-MODAL_CLS = os.environ.get("CASE_FORGE_MODAL_CLS", "CaseForge")
+BASE_MODEL = os.environ.get("CASE_FORGE_BASE", "Qwen/Qwen3-4B-Instruct-2507")
+ADAPTER_REPO = os.environ.get(
+    "CASE_FORGE_ADAPTER", "build-small-hackathon/case-forge-qwen3-4b").strip()
+MAX_NEW_TOKENS = int(os.environ.get("CASE_FORGE_MAX_TOKENS", "2800"))
 FORCE_DEMO = os.environ.get("CASE_FORGE_DEMO", "").strip() in ("1", "true", "yes")
 
-_CLS = None  # cached Modal class handle
+_MODEL = None
+_TOK = None
 
 
-def _modal_ready() -> bool:
-    return bool(os.environ.get("MODAL_TOKEN_ID") and os.environ.get("MODAL_TOKEN_SECRET"))
+def _has_cuda() -> bool:
+    try:
+        import torch
+        return torch.cuda.is_available()
+    except Exception:
+        return False
+
+
+def _ensure_model() -> None:
+    """Lazy-load base + LoRA and merge — runs inside the GPU-allocated context."""
+    global _MODEL, _TOK
+    if _MODEL is not None:
+        return
+    import torch
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+
+    tok = AutoTokenizer.from_pretrained(BASE_MODEL, trust_remote_code=True)
+    model = AutoModelForCausalLM.from_pretrained(
+        BASE_MODEL, torch_dtype=torch.bfloat16, device_map="cuda",
+        trust_remote_code=True,
+    )
+    if ADAPTER_REPO:
+        from peft import PeftModel
+        model = PeftModel.from_pretrained(model, ADAPTER_REPO)
+        model = model.merge_and_unload()   # fold LoRA into base → faster generation
+    model.eval()
+    _MODEL, _TOK = model, tok
 
 
-def _cls():
-    global _CLS
-    if _CLS is None:
-        import modal
-        _CLS = modal.Cls.from_name(MODAL_APP, MODAL_CLS)
-    return _CLS
+@gpu.gpu(duration=120)
+def _generate_raw(messages: list[dict]) -> str:
+    """Run the model on ZeroGPU and return the raw decoded completion."""
+    import torch
+
+    _ensure_model()
+    try:
+        text = _TOK.apply_chat_template(
+            messages, tokenize=False, add_generation_prompt=True,
+            enable_thinking=False,
+        )
+    except TypeError:
+        text = _TOK.apply_chat_template(
+            messages, tokenize=False, add_generation_prompt=True,
+        )
+    inputs = _TOK(text, return_tensors="pt").to(_MODEL.device)
+    with torch.no_grad():
+        out = _MODEL.generate(
+            **inputs, max_new_tokens=MAX_NEW_TOKENS,
+            do_sample=True, temperature=0.7, top_p=0.95,
+            pad_token_id=_TOK.pad_token_id or _TOK.eos_token_id,
+        )
+    return _TOK.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
+
+
+def _parse(raw: str) -> dict | None:
+    try:
+        return json.loads(raw[raw.find("{"): raw.rfind("}") + 1])
+    except Exception:
+        return None
 
 
 # --- demo fallback -------------------------------------------------------
@@ -53,8 +110,7 @@ _DEMO_BANK: dict[str, dict] | None = None
 
 
 def _demo_bank() -> dict[str, dict]:
-    """One valid sample case per language from the local corpus — lets the whole
-    UI (render, badges, export) work without Modal (offline/screenshots)."""
+    """One valid sample case per language from the local corpus — UI works offline."""
     global _DEMO_BANK
     if _DEMO_BANK is not None:
         return _DEMO_BANK
@@ -98,7 +154,7 @@ def _demo_result(seed: Seed) -> dict:
 
 def generate(domain: str, topic: str, level: str = "MBA",
              language: str = "pt", theory: str = "") -> dict:
-    """Forge one case+note from a short request, via the Modal server.
+    """Forge one case+note from a short request, on ZeroGPU.
 
     Returns {obj, valid, errors, warnings, raw, demo}.
     """
@@ -110,18 +166,20 @@ def generate(domain: str, topic: str, level: str = "MBA",
         theory=[t.strip() for t in (theory or "").split(",") if t.strip()],
     )
 
-    if FORCE_DEMO or not _modal_ready():
+    if FORCE_DEMO or not (gpu._HAS_SPACES or _has_cuda()):
         return _demo_result(seed)
 
     try:
-        res = _cls()().generate.remote(seed.__dict__)
-    except Exception as exc:  # Modal unreachable / cold-start failure → don't crash UI
+        raw = _generate_raw(build_minimal_prompt(seed))
+    except Exception as exc:
         out = _demo_result(seed)
-        out["errors"] = [f"falha na geração (Modal): {exc}"] + out["errors"]
+        out["errors"] = [f"falha na geração: {exc}"] + out["errors"]
         return out
 
-    res["demo"] = False
-    return res
+    obj = _parse(raw)
+    ok, errs, warns = validate_case(obj) if obj else (False, ["parse falhou"], [])
+    return {"obj": obj, "valid": ok, "errors": errs, "warnings": warns,
+            "raw": raw, "demo": False}
 
 
 __all__ = ["generate", "Seed"]

requirements.txt

@@ -1,10 +1,13 @@
-# Case Forge — HF Space runtime (Gradio app).
+# Case Forge — HF Space runtime (Gradio app on ZeroGPU).
 #
-# Inference runs on MODAL (no ZeroGPU for this project): the app calls a deployed
-# warm Modal class (case-forge-serve / CaseForge, see pipeline/serve_modal.py),
-# so the Space itself needs NO GPU libs (torch/transformers/peft) — it just renders
-# and calls Modal. Set MODAL_TOKEN_ID / MODAL_TOKEN_SECRET as Space secrets.
+# App inference runs in-Space on ZeroGPU (free): base Qwen3-4B + the published LoRA,
+# merged at first GPU call, generating via transformers under @spaces.GPU.
+# Heavy/offline work (corpus generation, training, LoRA merge) runs on Modal — NOT here.
 
 gradio>=6
-modal                  # calls the deployed Modal inference server
+spaces                 # ZeroGPU (@spaces.GPU) — no-op locally
+transformers>=4.49
+peft>=0.13             # load + merge_and_unload the adapter
+accelerate>=1
+torch
 jsonschema>=4          # full structural validation of the output contract