field_notes.md

Field Notes: Building Step-Zero (Cognitive Pacemaker)

Step-Zero is a local-first "Cognitive Pacemaker" application designed to break down overwhelming goals into atomic physical actions to overcome executive dysfunction.

Running small language models (≤4B parameters) entirely on consumer hardware forces you to confront the "Alignment Tax" head-on. Here is the honest developer log of the design decisions, engineering failures, and breakthroughs we encountered.

---

1. System Architecture

Step-Zero uses a dual-model orchestrator executing locally on CPU/laptop hardware via the llama.cpp runtime.

                  ┌──────────────────────┐
                  │ Goal & UI Input      │
                  └──────────┬───────────┘
                             │
                             ▼
               ┌───────────────────────────┐
               │ Primary Model (Nemotron)  │
               └─────────────┬─────────────┘
                             │
                     [Output Valid?]
                     /             \
                   Yes              No (or "Too Hard")
                   /                 \
                  ▼                   ▼
      ┌──────────────────────┐   ┌───────────────────────┐
      │ Render Action in UI  │   │ Fallback (MiniCPM)    │
      └──────────────────────┘   └───────────────────────┘

1. Primary Model (Nemotron-Mini-4B): Fine-tuned using Unsloth SFT specifically on daily goal breakdown datasets (verbs.jsonl) to produce atomic, physical next-step instructions.
2. Fallback Model (MiniCPM-3-4B): An instruction-following model used for stylistic tone adjustments (Calm vs. Encouraging), semantic repetition loops, and simplifying tasks when a user signals "Too Hard".

---

2. Technical Alignments & The GBNF Stop-Token Deadlock

During testing under real model inference, we noticed that Nemotron-Mini-4B suffered from severe over-generation, outputting long paragraphs containing prompt template headers, formatting guidelines, and repeat sentences. Every single task triggered our validation filters and fell back to MiniCPM, causing a 25-second latency penalty.

Here are the two breakthroughs that solved this:

Breakthrough A: Resolving the GBNF Sampler Deadlock

We originally constrained the model using a GBNF grammar:

root ::= [a-zA-Z0-9 ,.!?'-]+

We also specified a stop token of \n in the inference call (stop=["\n"]) to terminate the action after the first sentence.

The Failure: The GBNF grammar restricted the allowed character set to letters, numbers, and basic punctuation. Crucially, it did not include the newline character \n. Because GBNF constraints are enforced at the sampler level, the model's probability of selecting \n was set to exactly zero.

As a result, the model was mathematically forbidden from generating the stop token. It kept generating sentence after sentence until it hit the hard limit of max_tokens=64, leading to severe paragraph bloating and prompt echoing.

The Fix: We updated the GBNF grammar to permit newline characters:

root ::= [a-zA-Z0-9 ,.!?\n'-]+

This allowed the model to output a newline character as soon as it finished the task, instantly triggering the stop=["\n"] check and terminating generation after the first sentence. The model now outputs exactly 6 to 8 words, saving massive CPU latency.

Breakthrough B: Token-Level Early Stopping for Prompt Echoing

Small models are highly susceptible to "prompt echoing" (repeating headers like Completed Tasks: or Goal: when prompted). Rather than waiting for the model to generate a full 64-token sequence of echoed prompt template text before filtering it out in Python, we added the headers directly to the llama.cpp stop-words:

stop=["\n", "<extra_id_1>", "Completed Tasks", "Goal:", "User:", "Assistant:"]

If the model begins echoing the prompt template, it hits a stop word within 1 or 2 tokens and halts generation immediately. This terminates failed inference loops in under 50ms, allowing an instantaneous fallback to MiniCPM.

---

3. Custom UI/UX: The "Fog of War"

To support the cognitive design of the pacemaker (which aims to reduce cognitive load), the app features a minimal, custom dark theme. By leveraging Gradio's CSS custom properties on :root and .dark, we overrode Gradio's standard elements to deliver a clean, borderless dark UI styled like a game interface:

• Fog of War: Completed history is styled with low opacity (opacity: 0.3) and blurred (filter: blur(2px)) to keep the user's attention anchored on the single next step.
• Start Button Theme: Overrode Gradio's default primary orange buttons to match the deep forest green theme of the interface.
• Timer and Recovery: Included a clean "Start Over" action flow to allow user recovery once the circuit breaker fires.

---

4. Key Takeaways & Small Model Constraints

1. Syntax Constraints are Sampler Constraints: Constraining text syntax with GBNF without matching stop criteria can lead to sampler deadlocks. Always align your grammar's allowed character space with your engine's stop sequences.
2. LoRA Fine-tuning is Highly Sensitive: A 4B parameter model is highly sensitive to format drift. Minor discrepancies between fine-tuning prompt structure and inference structure will cause formatting breakdowns.
3. Optimized CPU Execution: With optimized GGUF quantizations and proper early stopping, both models run under 50ms per token on consumer CPU laptop hardware, making fully offline cognitive assistance viable.