feat: increase max_tokens to 64 and add model download instructions to README

README.md

@@ -34,9 +34,21 @@ Choose Direct, Calm, or Encouraging before pressing Enter to start a session.
 
 ## Full Local Model Mode (Production)
 
-To run the full LLM backend, place the required quantized GGUF files under `models/`:
-* `step-zero-nemotron-finetuned.gguf`
-* `minicpm-3-4b.gguf`
+To run the full LLM backend, you must first download the required quantized GGUF files into the `models/` directory.
+
+You can download them using a quick Python script (since `huggingface-hub` is installed):
+```bash
+mkdir -p models
+
+# Download the fine-tuned Nemotron model
+python3 -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='tc043/step-zero-nemotron', filename='step-zero-nemotron-finetuned.gguf', local_dir='models')"
+
+# Download the MiniCPM fallback model (Q4_K_M recommended for 4GB VRAM)
+python3 -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='mradermacher/MiniCPM3-4B-GGUF', filename='MiniCPM3-4B.Q4_K_M.gguf', local_dir='models')"
+
+# Rename fallback model to match path config
+mv models/MiniCPM3-4B.Q4_K_M.gguf models/minicpm-3-4b.gguf
+```
 
 Then run:
 

app.py

@@ -113,7 +113,7 @@ async def generate_atomic_task(goal: str, previous_failures: int, history: list
             fallback_res = await run_minicpm(
                 minicpm.create_chat_completion,
                 messages=messages,
-                max_tokens=20, 
+                max_tokens=64, 
                 temperature=0.1
             )
             content = fallback_res['choices'][0]['message']['content'].strip()
@@ -131,7 +131,7 @@ async def generate_atomic_task(goal: str, previous_failures: int, history: list
         response = await run_nemotron(
             nemotron,
             prompt,
-            max_tokens=20, 
+            max_tokens=64, 
             temperature=0.3,
             stop=["\n", "<extra_id_1>"],
             grammar=grammar # Enforcing grammar here!
@@ -161,7 +161,7 @@ async def generate_atomic_task(goal: str, previous_failures: int, history: list
             fallback_res = await run_minicpm(
                 minicpm.create_chat_completion,
                 messages=messages,
-                max_tokens=20, 
+                max_tokens=64, 
                 temperature=0.1
             )
             content = fallback_res['choices'][0]['message']['content'].strip()