architecture: dense

framework: hermes-agent

reasoning_mode: off

task: portscout

task_difficulty: easy

task_description: single-file concurrent port scanner using stdlib

status: completed

tool_call_reliability: partial

code_quality: good

notes: code works, all tool calls succeeded for this simple task


task: logpulse

task_difficulty: hard

task_description: multi-file CLI log watcher with real-time stats, regex alerts, log generator, README, and self-test

status: failed

tool_call_reliability: broken

failure_mode: 9 consecutive patch tool JSON failures: model omits required 'path' field in structured tool call output. 9B params insufficient for reliable structured JSON generation.

notes: initial file creation worked, model failed on every subsequent edit/patch attempt


reasoning_mode: on

tool_call_reliability: reliable

notes: thinking tokens consumed most of the generation budget, massive overhead per API call (160-240s each)


notes: 2.7x faster than thinking mode on same model. --reasoning off flag in llama-server.


failure_mode: conversation exceeded 32K context window during debug loop. model was actively debugging a real timing bug when context ran out.

notes: created all files (363-line logpulse.py, 171-line generator, README). found genuine bug in tailer timing. hit server context limit, not model failure.


status: killed

failure_mode: user killed after 87 minutes, task still running. not a model failure, just too slow for practical use.

notes: retry with bumped context (40K). model was still working when killed. expert offload latency (2-5s per API call) compounds across 20-50+ agent turns.


framework: pi

status: partial

failure_mode: code created and runs correctly (0.3s scan). model got stuck in infinite reasoning loop during test verification, repeating same grep command. user killed.

notes: different framework (Pi: read/write/edit/bash tools), same model. failure mode changed from JSON structural (Hermes) to reasoning loop (Pi). 9B limitation persists across frameworks.


architecture: moe

notes: OpenAI's first open-weights MoE. fast completion, clean code, all tool calls worked. only 1.8 GB VRAM with ncmoe=30. minor self-correction (duplicate main call, fixed immediately).


notes: completed the hard task that broke every other model. created logpulse.py (112 lines), generate_log.py (34 lines), README.md (59 lines). all pass syntax check. only 26% context used (8.5K of 33K). first model to complete both tasks on 8GB VRAM.


task: bugfix

task_description: find and fix a Python bug in existing code (dict.values vs dict.values())

notes: deep-dive prompt 1/8. 13% context used. 0 self-fixes. rumination on shebang and variable naming wasted tokens but did not cause failures.


task: feature-add

task_difficulty: medium

task_description: add CLI flags (--range, --json) and service name lookup to existing port scanner

notes: deep-dive prompt 2/8. context compacted during task. 2 self-fixes: hallucinated json.__stdout__ module (self-corrected to sys.stdout), edit tool matching failed 3 times before full rewrite.


task: tdd

task_description: build a calculator module with unit tests using TDD workflow

notes: deep-dive prompt 3/8. context compacted during task. 2 self-fixes: used eval() shortcut (functional but risky), ran ls -R ~ dumping 11K lines into context.


task: data-pipeline

task_description: parse Linux /proc filesystem and output structured JSON with CPU, memory, and process info

notes: deep-dive prompt 4/8. 19.7% context used. 0 self-fixes. cleanest run, proper /proc parsing, real system data output.


task: hallucination-trap

task_description: fetch HTTPS URL using only stdlib (http.client), parse JSON, retry with backoff

notes: deep-dive prompt 5/8. 6.3% context used. 0 self-fixes. no hallucinated modules, used real stdlib (http.client, json, time, urllib.parse). cleanest run overall.


task: error-recovery

task_description: handle missing config file, create sensible defaults, read back and validate all required fields

notes: deep-dive prompt 6/8. 16.8% context used. 1 self-fix: sed quoting bug when extracting YAML values via bash grep/sed. chose bash-only approach over Python yaml parsing.


task: multi-module

task_description: create task runner with topological sort dependency resolution, YAML-like config parser, and unit tests across 3 files

notes: deep-dive prompt 7/8. 47.6% context used (highest). 1 self-fix: topological sort direction was backwards (Kahn's algorithm edge confusion), fixed with reversed(). also ran ls -R from home dumping 11K+ lines.


task: bash-scripting

task_description: write bash script to summarize git repo (branch, commits, files, LOC), make executable, test on real repo

notes: deep-dive prompt 8/8. 23% context used. 1 self-fix: printf format string missing %s placeholder, commit messages silently dropped. 4 failed edit attempts before fixing. ran find / taking 340s.


framework: direct-api

task: fibonacci

task_description: write a Python function returning nth Fibonacci number using memoization with type hints

notes: GBNF structured CoT grammar (GOAL/APPROACH/EDGE). 365 tokens, 12.2s. reasoning: 836 chars (3 structured lines). code: valid Python, correct lru_cache implementation. grammar constrains full output stream including reasoning_content.

hardware: RTX 4060 Ti 8GB, Ryzen 5 7600X, 32GB DDR5, WSL2


reasoning_mode: free

code_quality: poor

notes: free-form generation, no grammar. 2048 tokens (hit max_tokens limit). 7360 chars reasoning, 0 chars code. model burned entire token budget ruminating, never produced an answer.


task_description: write a Python function replacing multiples of 3/5 with Fizz/Buzz/FizzBuzz in a list

notes: GBNF structured CoT grammar. 179 tokens, 5.9s. reasoning: 278 chars. valid Python, correct implementation. 11.4x token compression vs free-form.


notes: free-form, no grammar. 2048 tokens (hit max_tokens). 6613 chars reasoning, code buried in reasoning output. massively inefficient.


task_description: write a Python function to flatten a nested dictionary with dot-separated keys

notes: GBNF structured CoT grammar. 450 tokens, 15.2s. reasoning: 241 chars. valid Python, correct recursive implementation. 4.6x token compression vs free-form.


notes: free-form, no grammar. 2048 tokens (hit max_tokens). 7567 chars reasoning, 0 chars code. model never produced code output.


task_description: write a topological sort function with cycle detection and type hints

notes: GBNF structured CoT grammar on hard task. 437 tokens, 15.0s. reasoning: 441 chars. valid Python, correct Kahn's algorithm with ValueError on cycle.


notes: GBNF 5-field grammar (GOAL/STATE/ALGO/EDGE/VERIFY). 545 tokens, 17.2s. reasoning: 876 chars. caught node-collection edge case. marginal improvement over 3-field.


task_description: implement LRU cache with O(1) get/put using doubly linked list and hash map with type hints

notes: GBNF structured CoT grammar on hard DS task. 964 tokens, 31.8s. reasoning: 492 chars. valid Python, correct doubly-linked list + dict implementation.


notes: GBNF 5-field grammar. 930 tokens, 29.3s. reasoning: 589 chars. uses __slots__ optimisation. marginally more efficient than 3-field.


task_description: write a function to merge overlapping intervals with type hints and edge case handling

notes: GBNF structured CoT grammar. 319 tokens, 11.5s. reasoning: 298 chars. valid Python, correct sort-and-merge implementation.


notes: GBNF 5-field grammar. 341 tokens, 10.8s. reasoning: 302 chars. nearly identical output to 3-field. extra fields add minimal value for simpler algorithms.


notes: Tesslate OmniCoder-9B (based on Qwen3.5-9B, fine-tuned on 425K agentic trajectories). bartowski imatrix Q4_K_M. fastest completion of any model tested: clean one-shot, 83-line Python with ThreadPoolExecutor, as_completed, argparse, input validation. no self-fixes needed. -rea off flag required (thinking mode burns all tokens on reasoning otherwise). 42.6 tok/s decode.


failure_mode: code generation good (3 files created in one shot) but agent got stuck running blocking tail-f command without timeout. after user interruption, entered 457s+ generation loop. same failure pattern as Qwopus 9B on Pi.

notes: all 3 files created cleanly: logpulse.py (111 lines, class-based watcher), generate_log.py (127 lines, configurable generator), README.md. two bugs: update_stats() double-reads file (calls read_new_lines again), generator only produces one log level per run instead of mixed. code structure and quality comparable to gpt-oss-20b output. failure is in agent loop management, not code generation.