final: sync completed submission

MODEL_SELECTION.md

@@ -106,6 +106,11 @@ Primary metrics:
 ## Implementation Notes
 
 - Keep temperature at `0.0-0.2` for explanation generation.
+- Thinking mode support is profile-gated. MiniCPM4.1 Balanced supports hybrid
+  reasoning via `enable_thinking` and uses `768` recommended max tokens when
+  enabled. The selected Qwen3 `*-Instruct-2507` Tiny and Quality profiles are
+  documented non-thinking variants, so the app disables Thinking for them rather
+  than sending ineffective `/think` hints.
 - Cap explanation context initially at `SFT_MAX_LENGTH=4096`; our RAG pipeline
   should retrieve better evidence, not dump the whole database into the prompt.
 - Enable `assistant_only_loss=True` only after verifying the selected model's chat

PLAN.md

@@ -21,7 +21,7 @@
 - Storage: raw API JSONL in `data/raw`, normalized Parquet in `data/parquet`, DuckDB at `data/dota2tuned.duckdb`, generated RAG docs in `data/rag`.
 - Tables: `dim_patch`, `dim_hero`, `dim_item`, `dim_ability`, `dim_league`, `fact_match`, `fact_player_match`, `fact_draft_pickban`, `fact_item_purchase`, `fact_hero_pair_stats`, `fact_hero_build_stats`, `fact_hero_skill_builds`, `doc_patch_change`, `doc_stratz_match`, `doc_stat_card`, `ingest_run`, `api_call_log`.
 - Data clients: Steam discovery/raw facts, OpenDota normalized stats, STRATZ rich enrichment, Valve patch JSON feed, dotaconstants constants.
-- Modal backend: deployed `ui`, `remote_smoke`, `train_sft`, and `generate_answer` functions. The HF Space remains available; Modal provides GPU training, adapter inference, and a verified alternate Gradio endpoint.
+- Modal backend: deployed `ui`, `remote_smoke`, `train_sft`, `train_sft_quality`, `generate_answer`, and `generate_answer_quality` functions. The HF Space remains available; Modal provides GPU training, profile-routed adapter inference, and a verified alternate Gradio endpoint.
 
 ## Interfaces
 
@@ -39,7 +39,7 @@
 - Recommendations: rank heroes by predicted win-probability delta plus empirical counter/synergy lift; builds come from observed item/skill/talent timing stats, not LLM invention.
 - RAG: index patch-change docs, hero stat cards, item constants, item timing stats, skill-build stats, and STRATZ match docs with patch/scope filters before retrieval.
 - Fine-tuning: default Tiny profile `Qwen/Qwen3-4B-Instruct-2507`; Balanced profile `openbmb/MiniCPM4.1-8B`; Quality profile `Qwen/Qwen3-30B-A3B-Instruct-2507`. Use explicit 4-bit QLoRA with TRL SFT on Modal `A100-80GB`, 1 epoch, structured answer examples, then push adapters to Hub.
-- Serving: HF Gradio Space loads compact predictor/RAG artifacts; Modal `ui` is a verified alternate Gradio endpoint, Modal `train_sft` handles GPU fine-tuning, and Modal `generate_answer` serves the fine-tuned adapter for explicit LLM responses.
+- Serving: HF Gradio Space loads compact predictor/RAG artifacts; Modal `ui` is a verified alternate Gradio endpoint, Modal `train_sft` handles Tiny/Balanced GPU fine-tuning, Modal `train_sft_quality` handles H200 Quality fine-tuning, and `generate_answer`/`generate_answer_quality` serve profile-routed adapter responses.
 
 ## Current Execution Process
 
@@ -57,7 +57,7 @@
    - `uv run python -c "from app import demo; print(type(demo).__name__, len(demo.blocks), len(demo.fns))"`
 5. Deploy Modal functions.
    - `uv run dota2tuned modal-deploy`
-   - Expected functions: `ui`, `remote_smoke`, `train_sft`, `generate_answer`.
+   - Expected functions: `ui`, `remote_smoke`, `train_sft`, `train_sft_quality`, `generate_answer`, `generate_answer_quality`.
    - Current Modal URL: `https://dracufeuer--dota2tuned-ui.modal.run`
 6. Verify Modal runtime artifacts.
    - `uv run dota2tuned modal-smoke`
@@ -77,6 +77,9 @@
    - Expected files: `adapter_model.safetensors`, `adapter_config.json`, tokenizer files, `training_args.bin`.
 8. Verify fine-tuned adapter inference.
    - `uv run dota2tuned modal-ask "Suggest one mid hero against Phantom Assassin and Witch Doctor. Include one caveat."`
+   - Quality profile routes to the H200-backed `generate_answer_quality` function.
+   - Balanced MiniCPM uses OpenBMB non-reasoning prompt mode and falls back to a grounded evidence response if the adapter returns malformed punctuation/empty text.
+   - Ask exposes a Thinking toggle and Max Tokens slider. Current support matrix: Balanced MiniCPM4.1 supports hybrid thinking and recommends `768` tokens; Tiny and Quality Qwen3 Instruct-2507 profiles are non-thinking variants, so the toggle is disabled for them.
    - Target behavior: concise grounded answer; caveat weak evidence; no invented heroes/items.
 9. Configure HF Space runtime variables/secrets for Modal-backed inference.
    - Variables: `MODAL_ENABLED=1`, `MODAL_APP_NAME=dota2tuned`.
@@ -212,6 +215,9 @@ All times are `America/Los_Angeles` / PDT unless noted.
 - 2026-06-15 16:39: Quality H200 progressed past memory and target-module setup, then failed because PEFT's `ParamWrapper` path rejects nonzero LoRA dropout for this model. Set Quality `lora_dropout=0.0` and prepared another retry.
 - 2026-06-15 16:45: Quality Qwen3 30B-A3B retry `fc-01KV6TCTX9CSYVHJ98S3P5PTMK` completed successfully and pushed to `build-small-hackathon/dota2tuned-qwen3-30b-a3b-2507-lora`.
 - 2026-06-15 16:47: added Draft-style `New here? How to use ...` guide cards to Ask, Meta, Builds, Predictor, and Data; kept Draft's existing guide; and documented the Build Small README validator/social-post requirement in `README.md` and `SUBMISSION.md`.
+- 2026-06-15 17:05: fixed adapter inference routing after live smoke showed Balanced MiniCPM returning malformed punctuation and Quality loading through the wrong A100-backed endpoint. Added profile-aware Modal inference selection, a dedicated H200 `generate_answer_quality` function, MiniCPM non-reasoning prompt mode, and a grounded malformed-output fallback for Balanced.
+- 2026-06-15 17:33: redeployed Modal and verified `modal-ask --profile minicpm4_1_8b` returns a grounded answer with `fallback=balanced_malformed_generation` instead of blank/garbled text; verified `modal-ask --profile qwen3_30b_a3b_2507` returns a real adapter answer from `build-small-hackathon/dota2tuned-qwen3-30b-a3b-2507-lora` through the H200 `generate_answer_quality` function.
+- 2026-06-15 17:52: researched reasoning support for the active bases. `openbmb/MiniCPM4.1-8B` supports hybrid thinking/non-thinking via `enable_thinking`; `Qwen/Qwen3-4B-Instruct-2507` and `Qwen/Qwen3-30B-A3B-Instruct-2507` are documented non-thinking Instruct variants. Added Ask-side Thinking and Max Tokens controls, enabling Thinking only for Balanced and auto-recommending `768` tokens when toggled on.
 
 ## Adapter Eval Notes
 

README.md

@@ -32,7 +32,7 @@ The implementation is designed around a simple rule: stats and predictors choose
 - Balanced fine-tuned adapter: https://huggingface.co/build-small-hackathon/dota2tuned-minicpm4-1-8b-lora
 - Quality fine-tuned adapter: https://huggingface.co/build-small-hackathon/dota2tuned-qwen3-30b-a3b-2507-lora
 - Dataset artifacts: https://huggingface.co/datasets/build-small-hackathon/dota2tuned-data
-- Demo video: TODO add final demo video URL before validation.
+- Demo video: https://drive.google.com/file/d/12l0sKN-rJJ3RwKDEE-TGbXfiRMepN_zz/view?usp=drive_link
 - Social post: TODO add final social post URL before validation.
 
 ## Hackathon Validation
@@ -93,11 +93,11 @@ The submitted Space includes compact serving artifacts under `data/parquet`, `da
 - `dota2tuned modal-smoke` validates the deployed Modal app can load artifacts.
 - `dota2tuned modal-train --profile qwen3_4b_2507` submits the Tiny Modal GPU QLoRA run. Other profiles include `minicpm4_1_8b` and `qwen3_30b_a3b_2507`; Quality routes to the H200-backed `train_sft_quality` function.
 - `uv run python scripts/watch_modal_training.py --call tiny=fc-... --interval 300` polls detached Modal training calls without blocking local development.
-- `dota2tuned modal-ask` calls the fine-tuned adapter through Modal GPU inference.
+- `dota2tuned modal-ask` calls the fine-tuned adapter through Modal GPU inference. Quality routes to H200-backed `generate_answer_quality`; Tiny and Balanced use `generate_answer`. Use `--thinking` only with the Balanced MiniCPM profile; Tiny and Quality use Qwen3 Instruct-2507 non-thinking bases, so the app disables the toggle for them.
 - `dota2tuned finetune --launch-job` remains available for Hugging Face Jobs if a token has `job.write`.
 - `dota2tuned serve` launches the Gradio app.
 
-The Gradio app includes searchable hero, item, role, scope, and ability dropdowns, alias-aware hero lookup (`PA`, `CM`, `AM`, `QOP`, `KOTL`, and curated initials), selected icon previews, and six sidebar modules: Ask, Draft, Meta, Builds, Predictor, and Data. Ask calls the Modal `generate_answer` function when Modal credentials are configured in the runtime environment; otherwise it degrades with a clear unavailable message.
+The Gradio app includes searchable hero, item, role, scope, and ability dropdowns, alias-aware hero lookup (`PA`, `CM`, `AM`, `QOP`, `KOTL`, and curated initials), selected icon previews, and six sidebar modules: Ask, Draft, Meta, Builds, Predictor, and Data. Ask calls the profile-routed Modal inference function when Modal credentials are configured in the runtime environment; otherwise it degrades with a clear unavailable message. Ask also exposes a Thinking toggle and Max Tokens slider; Thinking is enabled only for MiniCPM4.1 Balanced and recommends `768` tokens.
 
 See [PLAN.md](PLAN.md) for the full architecture and delivery plan.
 See [MODEL_SELECTION.md](MODEL_SELECTION.md) for the current LLM decision and eval protocol.

SUBMISSION.md

@@ -12,7 +12,7 @@ Last checked: 2026-06-15 16:17 PDT.
 - Dataset artifacts: https://huggingface.co/datasets/build-small-hackathon/dota2tuned-data
 - Modal alternate UI: https://dracufeuer--dota2tuned-ui.modal.run
 - README validator: https://build-small-hackathon-field-guide.hf.space/submit
-- Demo video: TODO add final demo video URL.
+- Demo video: https://drive.google.com/file/d/12l0sKN-rJJ3RwKDEE-TGbXfiRMepN_zz/view?usp=drive_link
 - Social post: TODO add final social post URL and mirror it in `README.md`.
 
 ## Hackathon Fit
@@ -45,7 +45,7 @@ Do not claim the Nemotron, Off the Grid, Llama Champion, or Sharing is Caring ba
 - [x] Repository license is declared as Apache-2.0.
 - [x] GitHub `main` is pushed and clean.
 - [x] README lists selected tracks, sponsor prizes, and eligible badges for the validator.
-- [ ] Record short demo video.
+- [x] Record short demo video.
 - [ ] Publish social post and replace the README `Social post` TODO with its URL.
 - [ ] Submit Space link, demo video link, and social post link by June 15, 2026.
 

src/dota2tuned/cli.py

@@ -19,6 +19,7 @@ from dota2tuned.finetune import (
     write_train_script,
 )
 from dota2tuned.ingest import IngestCoordinator
+from dota2tuned.modal_backend import modal_infer_function_name
 from dota2tuned.model_profiles import resolve_model_profile
 from dota2tuned.normalize import SCHEMAS, build_hero_build_stats, normalize_all
 from dota2tuned.rag import build_index
@@ -405,6 +406,10 @@ def modal_ask(
         str | None,
         typer.Option(help="Model profile key, e.g. qwen3_30b_a3b_2507."),
     ] = None,
+    thinking: Annotated[
+        bool,
+        typer.Option(help="Request reasoning mode when the selected profile supports it."),
+    ] = False,
 ) -> None:
     settings = get_settings()
     _require_modal(settings)
@@ -414,7 +419,20 @@ def modal_ask(
         typer.echo("Install Modal dependencies with `uv sync --extra modal`.", err=True)
         raise typer.Exit(1) from exc
 
-    generate_fn = modal.Function.from_name(settings.modal_app_name, "generate_answer")
+    selected_profile = resolve_model_profile(profile or settings.model_profile)
+    generate_fn = modal.Function.from_name(
+        settings.modal_app_name,
+        modal_infer_function_name(selected_profile.key),
+    )
     typer.echo(
-        json.dumps(generate_fn.remote(question, context, max_new_tokens, profile), indent=2)
+        json.dumps(
+            generate_fn.remote(
+                question,
+                context,
+                max_new_tokens,
+                selected_profile.key,
+                thinking,
+            ),
+            indent=2,
+        )
     )

src/dota2tuned/modal_backend.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import os
+import re
 import subprocess
 import sys
 from pathlib import Path
@@ -20,6 +21,13 @@ REMOTE_ROOT = Path("/app")
 REMOTE_DATA = REMOTE_ROOT / "data"
 REMOTE_CACHE = Path("/cache")
 REMOTE_OUTPUTS = Path("/outputs")
+BALANCED_PROFILE_KEY = "minicpm4_1_8b"
+QUALITY_PROFILE_KEY = "qwen3_30b_a3b_2507"
+QUALITY_INFER_FUNCTION = "generate_answer_quality"
+DEFAULT_INFER_FUNCTION = "generate_answer"
+THINKING_TOKEN_LIMIT = 1536
+NON_THINKING_TOKEN_LIMIT = 768
+THINK_RE = re.compile(r"<think>.*?</think>", flags=re.DOTALL | re.IGNORECASE)
 
 CORE_DEPS = [
     "duckdb>=1.5.3",
@@ -88,6 +96,117 @@ def _runtime_secret_dict() -> dict[str, str]:
     return {key: value for key, value in values.items() if value}
 
 
+def modal_infer_function_name(profile: str | None = None) -> str:
+    selected = resolve_model_profile(profile)
+    if selected.key == QUALITY_PROFILE_KEY:
+        return QUALITY_INFER_FUNCTION
+    return DEFAULT_INFER_FUNCTION
+
+
+def _looks_malformed_answer(answer: str) -> bool:
+    text = " ".join(str(answer or "").split())
+    if not text:
+        return True
+    if len(text) < 12:
+        return True
+    if text.count("\\") >= 6 or text.count("�") >= 2:
+        return True
+    nonspace = [char for char in text if not char.isspace()]
+    if not nonspace:
+        return True
+    alpha_numeric = sum(char.isalnum() for char in nonspace)
+    if alpha_numeric / len(nonspace) < 0.25:
+        return True
+    stripped_words = [
+        word.strip(".,;:!?*#`\"'()[]{}")
+        for word in text.split()
+    ]
+    word_like = [
+        word
+        for word in stripped_words
+        if len(word) >= 3 and any(char.isalpha() for char in word)
+    ]
+    normalized_words = [word.lower() for word in word_like]
+    if len(normalized_words) >= 8:
+        counts = {
+            word: normalized_words.count(word)
+            for word in set(normalized_words)
+        }
+        stems = [word[:5] for word in normalized_words]
+        stem_counts = {stem: stems.count(stem) for stem in set(stems)}
+        if max(counts.values()) / len(normalized_words) > 0.28:
+            return True
+        if max(stem_counts.values()) / len(stems) > 0.45:
+            return True
+    if len(text) < 120 and len(word_like) < 5:
+        return True
+    marker_count = text.count("##") + text.count("**") + text.count("\\")
+    if marker_count >= 3 and len(word_like) < 8:
+        return True
+    return len(set(nonspace)) <= 4 and len(nonspace) >= 24
+
+
+def _evidence_fallback_answer(question: str, context: str) -> str:
+    evidence_lines = [
+        " ".join(line.split())
+        for line in str(context or "").splitlines()
+        if line.strip()
+    ]
+    evidence = " ".join(evidence_lines)[:900].strip()
+    if not evidence:
+        evidence = "No retrieved evidence was supplied for this question."
+    return (
+        "Based on the retrieved evidence, use the most directly supported draft or "
+        "build choice rather than inventing outside context.\n\n"
+        f"Question: {question.strip()}\n\n"
+        f"Evidence summary: {evidence}\n\n"
+        "Recommendation: prefer the hero, item, or draft choice with the strongest "
+        "role fit and current-patch evidence in the retrieved context. Caveat: treat "
+        "this as evidence-grounded guidance, not a guaranteed match outcome."
+    )
+
+
+def _strip_reasoning_blocks(answer: str) -> str:
+    text = str(answer or "")
+    text = THINK_RE.sub("", text)
+    if "</think>" in text:
+        text = text.rsplit("</think>", 1)[-1]
+    return text.strip()
+
+
+def _install_peft_weight_converter_compat() -> bool:
+    try:
+        import inspect
+
+        from transformers.core_model_loading import WeightConverter
+    except Exception:
+        return False
+
+    if "distributed_operation" in inspect.signature(WeightConverter.__init__).parameters:
+        return False
+    if getattr(WeightConverter.__init__, "_dota2tuned_compat", False):
+        return True
+
+    original_init = WeightConverter.__init__
+
+    def _compat_init(
+        self,
+        source_patterns,
+        target_patterns,
+        operations,
+        distributed_operation=None,
+        quantization_operation=None,
+        **_kwargs,
+    ):
+        original_init(self, source_patterns, target_patterns, operations)
+        self.distributed_operation = distributed_operation
+        self.quantization_operation = quantization_operation
+
+    _compat_init._dota2tuned_compat = True
+    WeightConverter.__init__ = _compat_init
+    return True
+
+
 def _add_project_files(image: modal.Image) -> modal.Image:
     image = image.add_local_dir(
         PROJECT_ROOT / "src" / "dota2tuned",
@@ -114,7 +233,7 @@ if modal is not None and settings.modal_enabled:
         modal.Image.debian_slim(python_version="3.12").uv_pip_install(*TRAIN_DEPS).env(REMOTE_ENV)
     )
     _MODEL_CACHE: dict[str, object] = {}
-    quality_train_profile = resolve_model_profile("qwen3_30b_a3b_2507")
+    quality_train_profile = resolve_model_profile(QUALITY_PROFILE_KEY)
 
     @app.function(image=web_image, secrets=[runtime_secret], timeout=900)
     def remote_smoke() -> dict[str, object]:
@@ -242,19 +361,12 @@ if modal is not None and settings.modal_enabled:
     ) -> dict[str, object]:
         return _run_train_sft(dataset_source, profile or quality_train_profile.key)
 
-    @app.function(
-        image=train_image,
-        gpu=settings.modal_infer_gpu,
-        secrets=[runtime_secret],
-        volumes={REMOTE_CACHE: cache_volume},
-        timeout=settings.modal_infer_timeout,
-        max_containers=1,
-    )
-    def generate_answer(
+    def _run_generate_answer(
         question: str,
         context: str = "",
         max_new_tokens: int = 384,
         profile: str | None = None,
+        thinking: bool = False,
     ) -> dict[str, object]:
         import torch
         from peft import AutoPeftModelForCausalLM
@@ -262,6 +374,8 @@ if modal is not None and settings.modal_enabled:
 
         from dota2tuned.model_profiles import resolve_model_profile
 
+        _install_peft_weight_converter_compat()
+
         try:
             from transformers.utils import import_utils
 
@@ -283,6 +397,7 @@ if modal is not None and settings.modal_enabled:
             raise RuntimeError("question is required.")
 
         selected = resolve_model_profile(profile)
+        thinking_enabled = bool(thinking) and bool(selected.supports_thinking)
         model_id = selected.hf_model_repo_id
         cache_key = f"{selected.key}:{model_id}"
         if cache_key not in _MODEL_CACHE:
@@ -319,37 +434,117 @@ if modal is not None and settings.modal_enabled:
         user = question.strip()
         if context.strip():
             user = f"{user}\n\nEvidence:\n{context.strip()}"
-        messages = [
-            {"role": "system", "content": system},
-            {"role": "user", "content": user},
-        ]
+        if selected.key == BALANCED_PROFILE_KEY:
+            thinking_tag = "/think" if thinking_enabled else "/no_think"
+            messages = [
+                {
+                    "role": "user",
+                    "content": f"{system}\n\n{user}\n\n{thinking_tag}",
+                }
+            ]
+        else:
+            messages = [
+                {"role": "system", "content": system},
+                {"role": "user", "content": user},
+            ]
         try:
-            prompt = tokenizer.apply_chat_template(
-                messages,
-                tokenize=False,
-                add_generation_prompt=True,
-            )
+            template_kwargs = {
+                "tokenize": False,
+                "add_generation_prompt": True,
+            }
+            if selected.key == BALANCED_PROFILE_KEY:
+                template_kwargs["enable_thinking"] = thinking_enabled
+            prompt = tokenizer.apply_chat_template(messages, **template_kwargs)
         except Exception:
             prompt = f"{system}\n\nUser: {user}\nAssistant:"
+        token_limit = THINKING_TOKEN_LIMIT if thinking_enabled else NON_THINKING_TOKEN_LIMIT
         inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+        generation_kwargs = {
+            "max_new_tokens": max(32, min(max_new_tokens, token_limit)),
+            "use_cache": False,
+            "pad_token_id": tokenizer.pad_token_id,
+            "eos_token_id": tokenizer.eos_token_id,
+        }
+        if selected.key == BALANCED_PROFILE_KEY:
+            generation_kwargs.update(
+                {
+                    "do_sample": True,
+                    "temperature": 0.6,
+                    "top_p": 0.95,
+                    "repetition_penalty": 1.05,
+                }
+            )
+        else:
+            generation_kwargs["do_sample"] = False
         with torch.no_grad():
             outputs = model.generate(
                 **inputs,
-                max_new_tokens=max(32, min(max_new_tokens, 768)),
-                do_sample=False,
-                use_cache=False,
-                pad_token_id=tokenizer.pad_token_id,
-                eos_token_id=tokenizer.eos_token_id,
+                **generation_kwargs,
             )
         generated = outputs[0][inputs["input_ids"].shape[-1] :]
-        answer = tokenizer.decode(generated, skip_special_tokens=True).strip()
-        return {
+        raw_answer = tokenizer.decode(generated, skip_special_tokens=True).strip()
+        answer = _strip_reasoning_blocks(raw_answer)
+        fallback_reason = None
+        if selected.key == BALANCED_PROFILE_KEY and _looks_malformed_answer(answer):
+            answer = _evidence_fallback_answer(question, context)
+            fallback_reason = "balanced_malformed_generation"
+        response = {
             "status": "ok",
             "profile": selected.key,
             "model": model_id,
             "answer": answer,
             "tokens": int(generated.numel()),
+            "thinking_requested": bool(thinking),
+            "thinking_enabled": thinking_enabled,
+            "recommended_max_tokens": (
+                selected.thinking_recommended_max_tokens
+                if selected.supports_thinking
+                else NON_THINKING_TOKEN_LIMIT
+            ),
         }
+        if fallback_reason:
+            response["fallback"] = fallback_reason
+        return response
+
+    @app.function(
+        image=train_image,
+        gpu=settings.modal_infer_gpu,
+        secrets=[runtime_secret],
+        volumes={REMOTE_CACHE: cache_volume},
+        timeout=settings.modal_infer_timeout,
+        max_containers=1,
+    )
+    def generate_answer(
+        question: str,
+        context: str = "",
+        max_new_tokens: int = 384,
+        profile: str | None = None,
+        thinking: bool = False,
+    ) -> dict[str, object]:
+        return _run_generate_answer(question, context, max_new_tokens, profile, thinking)
+
+    @app.function(
+        image=train_image,
+        gpu=quality_train_profile.modal_infer_gpu,
+        secrets=[runtime_secret],
+        volumes={REMOTE_CACHE: cache_volume},
+        timeout=quality_train_profile.modal_infer_timeout,
+        max_containers=1,
+    )
+    def generate_answer_quality(
+        question: str,
+        context: str = "",
+        max_new_tokens: int = 384,
+        profile: str | None = None,
+        thinking: bool = False,
+    ) -> dict[str, object]:
+        return _run_generate_answer(
+            question,
+            context,
+            max_new_tokens,
+            profile or quality_train_profile.key,
+            thinking,
+        )
 else:
     app = None
 

src/dota2tuned/model_profiles.py

@@ -29,6 +29,8 @@ class ModelProfile:
     sft_grad_accum: int = 8
     model_load_in_4bit: bool = True
     model_torch_dtype: str = "bfloat16"
+    supports_thinking: bool = False
+    thinking_recommended_max_tokens: int = 768
 
 
 DEFAULT_MODEL_PROFILE = "qwen3_4b_2507"
@@ -69,6 +71,8 @@ DEFAULT_MODEL_PROFILES: dict[str, ModelProfile] = {
         lora_alpha=16,
         sft_learning_rate=1.5e-4,
         sft_grad_accum=8,
+        supports_thinking=True,
+        thinking_recommended_max_tokens=768,
     ),
 }
 
@@ -151,4 +155,13 @@ def apply_profile_env_overrides(profile: ModelProfile) -> ModelProfile:
             os.getenv("MODEL_LOAD_IN_4BIT"), profile.model_load_in_4bit
         ),
         model_torch_dtype=os.getenv("MODEL_TORCH_DTYPE", profile.model_torch_dtype),
+        supports_thinking=_bool(
+            os.getenv("MODEL_SUPPORTS_THINKING"), profile.supports_thinking
+        ),
+        thinking_recommended_max_tokens=int(
+            os.getenv(
+                "THINKING_RECOMMENDED_MAX_TOKENS",
+                str(profile.thinking_recommended_max_tokens),
+            )
+        ),
     )

src/dota2tuned/ui/gradio_app.py

@@ -10,7 +10,8 @@ import polars as pl
 from starlette.middleware import Middleware
 
 from dota2tuned.config import get_settings
-from dota2tuned.model_profiles import profile_choices
+from dota2tuned.modal_backend import modal_infer_function_name
+from dota2tuned.model_profiles import profile_choices, resolve_model_profile
 from dota2tuned.normalize import ROLE_ALL
 from dota2tuned.rag import Retriever
 from dota2tuned.recommend import DraftRecommender
@@ -976,6 +977,19 @@ __NAV_ICON_CSS__
   font-size: 15px !important;
   line-height: 22px !important;
 }
+.assistant-thinking-note-wrap,
+.assistant-thinking-note-wrap .block,
+.assistant-thinking-note-wrap .prose {
+  border: 0 !important;
+  background: transparent !important;
+  box-shadow: none !important;
+}
+.assistant-thinking-note {
+  color: rgba(239, 229, 187, 0.86);
+  font-size: 11.5px;
+  line-height: 17px;
+  padding: 0 2px 2px;
+}
 .assistant-actions {
   align-items: center;
 }
@@ -4536,6 +4550,7 @@ def _call_tuned_model(
     context: str,
     max_new_tokens: int = 384,
     profile: str | None = None,
+    thinking: bool = False,
 ) -> str:
     if not question.strip():
         return "Enter a question."
@@ -4551,8 +4566,18 @@ def _call_tuned_model(
     os.environ["MODAL_TOKEN_ID"] = settings.modal_token_id
     os.environ["MODAL_TOKEN_SECRET"] = settings.modal_token_secret
     try:
-        generate_fn = modal.Function.from_name(settings.modal_app_name, "generate_answer")
-        result = generate_fn.remote(question, context, max_new_tokens, profile)
+        selected_profile = resolve_model_profile(profile or settings.model_profile)
+        generate_fn = modal.Function.from_name(
+            settings.modal_app_name,
+            modal_infer_function_name(selected_profile.key),
+        )
+        result = generate_fn.remote(
+            question,
+            context,
+            max_new_tokens,
+            selected_profile.key,
+            thinking,
+        )
     except Exception as exc:
         return f"Tuned model call failed: {str(exc)[:500]}"
 
@@ -4562,7 +4587,14 @@ def _call_tuned_model(
     model = result.get("model") or settings.hf_model_repo_id
     profile_key = result.get("profile") or profile or settings.model_profile
     tokens = result.get("tokens")
-    return f"{answer}\n\n`profile: {profile_key}` `model: {model}` `tokens: {tokens}`"
+    thinking_label = "on" if result.get("thinking_enabled") else "off"
+    fallback = result.get("fallback")
+    fallback_text = f" `fallback: {fallback}`" if fallback else ""
+    return (
+        f"{answer}\n\n"
+        f"`profile: {profile_key}` `model: {model}` `tokens: {tokens}` "
+        f"`thinking: {thinking_label}`{fallback_text}"
+    )
 
 
 def build_app() -> gr.Blocks:
@@ -4684,10 +4716,72 @@ def build_app() -> gr.Blocks:
         )
         return _render_module_result_html(answer, docs, hero_metadata, item_metadata)
 
+    def thinking_note_html(model_profile: str | None, thinking: bool = False) -> str:
+        selected = resolve_model_profile(model_profile or settings.model_profile)
+        if selected.supports_thinking:
+            status = "enabled" if thinking else "available"
+            return (
+                "<div class='assistant-thinking-note d2-text-glitch'>"
+                f"Thinking {status} for {selected.label}. "
+                f"Recommended max tokens: {selected.thinking_recommended_max_tokens}."
+                "</div>"
+            )
+        return (
+            "<div class='assistant-thinking-note'>"
+            f"Thinking is disabled for {selected.label}; this profile uses a "
+            "non-thinking base model."
+            "</div>"
+        )
+
+    def thinking_controls(
+        model_profile: str | None,
+        thinking: bool | None = False,
+        max_tokens: int | float | None = 384,
+    ) -> tuple[dict, dict, str]:
+        selected = resolve_model_profile(model_profile or settings.model_profile)
+        if not selected.supports_thinking:
+            return (
+                gr.update(value=False, interactive=False),
+                gr.update(value=384, maximum=768),
+                thinking_note_html(selected.key, False),
+            )
+        enabled = bool(thinking)
+        recommended = int(selected.thinking_recommended_max_tokens)
+        current_tokens = int(max_tokens or 384)
+        token_value = recommended if enabled else min(current_tokens, 768)
+        return (
+            gr.update(value=enabled, interactive=True),
+            gr.update(value=token_value, maximum=1536),
+            thinking_note_html(selected.key, enabled),
+        )
+
+    def thinking_toggle_controls(
+        model_profile: str | None,
+        thinking: bool | None,
+        max_tokens: int | float | None = 384,
+    ) -> tuple[dict, str]:
+        selected = resolve_model_profile(model_profile or settings.model_profile)
+        if not selected.supports_thinking:
+            return gr.update(value=384, maximum=768), thinking_note_html(selected.key, False)
+        enabled = bool(thinking)
+        token_value = (
+            int(selected.thinking_recommended_max_tokens)
+            if enabled
+            else min(int(max_tokens or 384), 768)
+        )
+        return gr.update(value=token_value, maximum=1536), thinking_note_html(selected.key, enabled)
+
     def tuned_model(
-        question: str, model_profile: str | None, source_filter: str
+        question: str,
+        model_profile: str | None,
+        source_filter: str,
+        thinking: bool | None,
+        max_tokens: int | float | None,
     ) -> tuple[str, str]:
         question = (question or "").strip()
+        selected_profile = resolve_model_profile(model_profile or settings.model_profile)
+        thinking_enabled = bool(thinking) and bool(selected_profile.supports_thinking)
+        token_budget = int(max_tokens or 384)
         docs = retriever.search(question or "current meta", patch="current", limit=6)
         if source_filter and source_filter != "All sources":
             docs = [
@@ -4698,7 +4792,14 @@ def build_app() -> gr.Blocks:
         evidence = "\n\n".join(
             f"{doc['source']} score={doc['score']}\n{doc['text']}" for doc in docs
         )
-        answer = _call_tuned_model(settings, question, evidence, 384, model_profile)
+        answer = _call_tuned_model(
+            settings,
+            question,
+            evidence,
+            token_budget,
+            selected_profile.key,
+            thinking_enabled,
+        )
         html = _render_tuned_answer_html(answer, docs, hero_metadata, item_metadata)
         return html, json.dumps(docs, indent=2)
 
@@ -4927,6 +5028,26 @@ def build_app() -> gr.Blocks:
                                 label="Sources",
                                 elem_classes=["dota-dropdown"],
                             )
+                        selected_profile_supports_thinking = resolve_model_profile(
+                            selected_model_profile
+                        ).supports_thinking
+                        with gr.Row(elem_classes=["d2-selector-grid", "d2-selector-grid-2"]):
+                            tuned_thinking = gr.Checkbox(
+                                value=False,
+                                label="Thinking",
+                                interactive=selected_profile_supports_thinking,
+                            )
+                            tuned_max_tokens = gr.Slider(
+                                minimum=128,
+                                maximum=1536 if selected_profile_supports_thinking else 768,
+                                value=384,
+                                step=64,
+                                label="Max Tokens",
+                            )
+                        tuned_thinking_note = gr.HTML(
+                            thinking_note_html(selected_model_profile, False),
+                            elem_classes=["assistant-thinking-note-wrap"],
+                        )
                         with gr.Row(elem_classes=["assistant-actions"]):
                             tuned_button = gr.Button("Ask DOTA2Tuned", variant="primary")
                             tuned_clear = gr.Button("Clear")
@@ -4944,19 +5065,51 @@ def build_app() -> gr.Blocks:
                                     queue=False,
                                 )
                     tuned_evidence = gr.Code(label="Evidence", language="json", value="[]")
+                    tuned_profile.change(
+                        thinking_controls,
+                        inputs=[tuned_profile, tuned_thinking, tuned_max_tokens],
+                        outputs=[tuned_thinking, tuned_max_tokens, tuned_thinking_note],
+                        api_visibility="private",
+                        queue=False,
+                    )
+                    tuned_thinking.change(
+                        thinking_toggle_controls,
+                        inputs=[tuned_profile, tuned_thinking, tuned_max_tokens],
+                        outputs=[tuned_max_tokens, tuned_thinking_note],
+                        api_visibility="private",
+                        queue=False,
+                    )
                     tuned_button.click(
                         tuned_model,
-                        inputs=[tuned_question, tuned_profile, tuned_source_filter],
+                        inputs=[
+                            tuned_question,
+                            tuned_profile,
+                            tuned_source_filter,
+                            tuned_thinking,
+                            tuned_max_tokens,
+                        ],
                         outputs=[assistant_output, tuned_evidence],
                     )
                     tuned_question.submit(
                         tuned_model,
-                        inputs=[tuned_question, tuned_profile, tuned_source_filter],
+                        inputs=[
+                            tuned_question,
+                            tuned_profile,
+                            tuned_source_filter,
+                            tuned_thinking,
+                            tuned_max_tokens,
+                        ],
                         outputs=[assistant_output, tuned_evidence],
                     )
                     tuned_clear.click(
-                        lambda: ("", _assistant_empty_state_html(), "[]"),
-                        outputs=[tuned_question, assistant_output, tuned_evidence],
+                        lambda: ("", _assistant_empty_state_html(), "[]", False, 384),
+                        outputs=[
+                            tuned_question,
+                            assistant_output,
+                            tuned_evidence,
+                            tuned_thinking,
+                            tuned_max_tokens,
+                        ],
                         api_visibility="private",
                         queue=False,
                     )

tests/test_gradio_app.py

@@ -135,6 +135,14 @@ def test_tuned_model_is_first_nav_and_default_view():
     assert "width: 100%" in APP_CSS
     assert "justify-content: flex-start" in APP_CSS
     assert "padding: 0;" in APP_CSS
+    assert "tuned_thinking = gr.Checkbox" in source
+    assert 'label="Thinking"' in source
+    assert "tuned_max_tokens = gr.Slider" in source
+    assert 'label="Max Tokens"' in source
+    assert "thinking_controls" in source
+    assert "thinking_toggle_controls" in source
+    assert "selected_profile_supports_thinking" in source
+    assert "assistant-thinking-note" in APP_CSS
 
 
 def test_assistant_references_link_patch_and_hero_sources():

tests/test_model_profiles.py

@@ -1,3 +1,12 @@
+import inspect
+
+from dota2tuned.modal_backend import (
+    _evidence_fallback_answer,
+    _install_peft_weight_converter_compat,
+    _looks_malformed_answer,
+    _strip_reasoning_blocks,
+    modal_infer_function_name,
+)
 from dota2tuned.model_profiles import apply_profile_env_overrides, resolve_model_profile
 
 
@@ -10,8 +19,11 @@ def test_model_profiles_include_quality_and_sponsor_paths():
     assert quality.modal_train_gpu == "H200"
     assert quality.lora_dropout == 0.0
     assert "q_proj" in quality.lora_target_modules
+    assert quality.supports_thinking is False
     assert sponsor.base_model_id == "openbmb/MiniCPM4.1-8B"
     assert sponsor.hf_model_repo_id.endswith("minicpm4-1-8b-lora")
+    assert sponsor.supports_thinking is True
+    assert sponsor.thinking_recommended_max_tokens == 768
 
 
 def test_model_profile_env_overrides(monkeypatch):
@@ -24,3 +36,57 @@ def test_model_profile_env_overrides(monkeypatch):
     assert profile.base_model_id == "example/base"
     assert profile.hf_model_repo_id == "example/adapter"
     assert profile.lora_r == 12
+
+
+def test_model_profile_thinking_env_overrides(monkeypatch):
+    monkeypatch.setenv("MODEL_SUPPORTS_THINKING", "1")
+    monkeypatch.setenv("THINKING_RECOMMENDED_MAX_TOKENS", "1024")
+
+    profile = apply_profile_env_overrides(resolve_model_profile("qwen3_4b_2507"))
+
+    assert profile.supports_thinking is True
+    assert profile.thinking_recommended_max_tokens == 1024
+
+
+def test_modal_inference_routes_quality_to_h200_function():
+    assert modal_infer_function_name("qwen3_4b_2507") == "generate_answer"
+    assert modal_infer_function_name("minicpm4_1_8b") == "generate_answer"
+    assert modal_infer_function_name("qwen3_30b_a3b_2507") == "generate_answer_quality"
+
+
+def test_malformed_balanced_output_guard_returns_grounded_text():
+    assert _looks_malformed_answer('".  \\  \\  \\  \\  \\  \\  \\')
+    assert _looks_malformed_answer("c3 ×ontology**\n\ninter. ##")
+    assert _looks_malformed_answer(
+        "vestig vestig vestig vestib vestige vestig vestment vestig vestig"
+    )
+    assert not _looks_malformed_answer("Crystal Maiden is a support with control.")
+
+    fallback = _evidence_fallback_answer(
+        "Suggest one support against Phantom Assassin.",
+        "Crystal Maiden has control. Phantom Assassin is a carry.",
+    )
+
+    assert "Based on the retrieved evidence" in fallback
+    assert "Crystal Maiden has control" in fallback
+
+
+def test_reasoning_blocks_are_stripped_from_visible_answer():
+    assert _strip_reasoning_blocks("<think>private scratchpad</think>Final answer.") == (
+        "Final answer."
+    )
+    assert _strip_reasoning_blocks("private scratchpad</think>Final answer.") == (
+        "Final answer."
+    )
+
+
+def test_peft_weight_converter_compat_accepts_new_peft_kwargs():
+    installed = _install_peft_weight_converter_compat()
+    if not installed:
+        return
+
+    from transformers.core_model_loading import WeightConverter
+
+    signature = inspect.signature(WeightConverter.__init__)
+    assert "distributed_operation" in signature.parameters
+    assert "quantization_operation" in signature.parameters