Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF / PATCHCODE_TESTING_PROCESS.html
jackasda211233's picture
Upload PATCHCODE_TESTING_PROCESS.html with huggingface_hub
e286d83 verified
Raw
History Blame Contribute Delete
43.5 kB
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PatchCode (agentic-coder) β€” Testing Process β€” Qwen3.6 AEON RYS Docs</title>
<meta name="description" content="Historical public record for the original Qwen3.6 AEON RYS SignalLatch ckpt386 s0.10 fine-tune, testing process, runtime setup, and first-release decision.">
<style>
:root {
color-scheme: light;
--paper: #f7f5ee;
--paper-2: #ede9dc;
--ink: #111816;
--muted: #586560;
--line: #d2d7ce;
--panel: #ffffff;
--panel-2: #fbfaf5;
--code: #eef1ea;
--green: #0f7667;
--green-dark: #09564c;
--blue: #275f86;
--rust: #9a5a1c;
--red: #943b34;
--gold: #b48412;
--shadow: 0 18px 46px rgba(17, 24, 22, 0.10);
--soft-shadow: 0 10px 24px rgba(17, 24, 22, 0.07);
--radius: 8px;
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
body {
margin: 0;
color: var(--ink);
background:
linear-gradient(90deg, rgba(17, 24, 22, 0.035) 1px, transparent 1px) 0 0 / 38px 38px,
linear-gradient(180deg, var(--paper), #fbfaf5 34rem, var(--paper));
font: 16px/1.62 ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
letter-spacing: 0;
overflow-x: hidden;
}
a {
color: var(--green-dark);
text-decoration-thickness: 0.08em;
text-underline-offset: 0.18em;
}
code, pre {
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, "Liberation Mono", monospace;
font-size: 0.93em;
}
.wrap {
width: min(1180px, calc(100% - 36px));
margin: 0 auto;
}
header {
position: sticky;
top: 0;
z-index: 20;
background: rgba(247, 245, 238, 0.92);
border-bottom: 1px solid var(--line);
backdrop-filter: blur(14px);
}
nav {
display: flex;
align-items: center;
justify-content: space-between;
gap: 18px;
min-height: 58px;
font-size: 0.9rem;
}
nav .brand {
color: var(--ink);
font-weight: 900;
white-space: nowrap;
text-decoration: none;
}
nav .links {
display: flex;
align-items: center;
justify-content: flex-end;
gap: 12px 16px;
flex-wrap: wrap;
}
nav .links a {
color: var(--muted);
text-decoration: none;
font-weight: 700;
}
nav .links a:hover { color: var(--green-dark); }
.hero {
padding: 76px 0 46px;
border-bottom: 1px solid var(--line);
background:
linear-gradient(180deg, rgba(237, 233, 220, 0.9), rgba(247, 245, 238, 0.22)),
linear-gradient(90deg, rgba(15, 118, 103, 0.08), rgba(39, 95, 134, 0.08));
}
.eyebrow {
display: inline-flex;
align-items: center;
gap: 10px;
color: var(--rust);
font-size: 0.78rem;
font-weight: 900;
text-transform: uppercase;
letter-spacing: 0.08em;
margin-bottom: 18px;
}
.eyebrow::before {
content: "";
width: 34px;
height: 2px;
background: var(--rust);
}
h1 {
margin: 0;
max-width: 1080px;
font-size: clamp(2.35rem, 5.3vw, 5.6rem);
line-height: 0.99;
letter-spacing: 0;
overflow-wrap: anywhere;
}
.dek {
max-width: 920px;
margin: 24px 0 0;
color: var(--muted);
font-size: clamp(1.03rem, 1.7vw, 1.28rem);
}
.actions {
display: flex;
gap: 12px;
flex-wrap: wrap;
margin-top: 28px;
}
.button {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 42px;
padding: 0 15px;
border: 1px solid var(--line);
border-radius: 7px;
background: var(--panel);
color: var(--ink);
text-decoration: none;
font-weight: 800;
box-shadow: var(--soft-shadow);
}
.button.primary {
background: var(--green);
border-color: var(--green);
color: #ffffff;
}
.quick {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 12px;
margin-top: 36px;
}
.metric {
min-height: 126px;
background: var(--panel);
border: 1px solid var(--line);
border-radius: var(--radius);
padding: 16px;
box-shadow: var(--soft-shadow);
}
.metric strong {
display: block;
font-size: 1.72rem;
line-height: 1.08;
margin-bottom: 8px;
overflow-wrap: anywhere;
}
.metric span {
display: block;
color: var(--muted);
font-size: 0.92rem;
}
main { padding: 26px 0 74px; }
section {
padding: 42px 0;
border-top: 1px solid var(--line);
}
h2 {
margin: 0 0 16px;
font-size: clamp(1.7rem, 3vw, 2.65rem);
line-height: 1.12;
letter-spacing: 0;
}
h3 {
margin: 0 0 10px;
font-size: 1.12rem;
letter-spacing: 0;
}
p {
margin: 0 0 16px;
max-width: 900px;
}
.muted { color: var(--muted); }
.grid-2 {
display: grid;
grid-template-columns: minmax(0, 0.98fr) minmax(320px, 1.02fr);
gap: 32px;
align-items: start;
}
.cards {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 14px;
}
.card {
background: var(--panel);
border: 1px solid var(--line);
border-radius: var(--radius);
padding: 18px;
box-shadow: var(--shadow);
}
.card.flat {
background: var(--panel-2);
box-shadow: none;
}
.card.good { border-left: 5px solid var(--green); }
.card.warn { border-left: 5px solid var(--gold); }
.card.risk { border-left: 5px solid var(--red); }
.card.blue { border-left: 5px solid var(--blue); }
.card p:last-child { margin-bottom: 0; }
.callout {
border-left: 4px solid var(--green);
background: rgba(15, 118, 103, 0.08);
padding: 18px 20px;
border-radius: 0 var(--radius) var(--radius) 0;
margin: 18px 0;
max-width: 980px;
}
.callout.warn {
border-left-color: var(--gold);
background: rgba(180, 132, 18, 0.10);
}
.flow {
display: grid;
grid-template-columns: repeat(5, minmax(0, 1fr));
gap: 10px;
margin: 20px 0;
}
.flow .step {
background: var(--panel);
border: 1px solid var(--line);
border-radius: var(--radius);
padding: 14px;
min-height: 118px;
}
.step b {
display: block;
color: var(--green-dark);
margin-bottom: 8px;
}
.figure {
margin: 24px 0 0;
padding: 14px;
border: 1px solid var(--line);
border-radius: var(--radius);
background: var(--panel);
box-shadow: var(--soft-shadow);
}
.figure img {
display: block;
width: 100%;
height: auto;
border-radius: 6px;
background: #ffffff;
}
.caption {
margin-top: 10px;
color: var(--muted);
font-size: 0.9rem;
}
.table-wrap {
overflow-x: auto;
border: 1px solid var(--line);
border-radius: var(--radius);
background: var(--panel);
box-shadow: var(--soft-shadow);
margin: 18px 0 26px;
}
table {
width: 100%;
min-width: 740px;
border-collapse: collapse;
font-size: 0.92rem;
}
th, td {
padding: 11px 12px;
border-bottom: 1px solid var(--line);
text-align: left;
vertical-align: top;
}
th {
background: var(--paper-2);
color: var(--ink);
font-weight: 900;
}
tr:last-child td { border-bottom: 0; }
.num { text-align: right; white-space: nowrap; }
.pass { color: var(--green-dark); font-weight: 800; }
.fail { color: var(--red); font-weight: 800; }
.soft { color: var(--muted); }
pre {
overflow-x: auto;
background: var(--code);
border: 1px solid var(--line);
border-radius: var(--radius);
padding: 16px;
color: var(--ink);
white-space: pre-wrap;
word-break: break-word;
}
.split-list {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 16px;
margin-top: 18px;
}
ul {
padding-left: 20px;
margin: 0 0 16px;
}
li { margin: 6px 0; }
footer {
padding: 32px 0 52px;
border-top: 1px solid var(--line);
color: var(--muted);
font-size: 0.92rem;
}
@media (max-width: 920px) {
nav { align-items: flex-start; flex-direction: column; padding: 12px 0; }
nav .links { justify-content: flex-start; }
.hero { padding-top: 46px; }
.quick { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.grid-2 { grid-template-columns: 1fr; }
.flow { grid-template-columns: 1fr; }
.cards, .split-list { grid-template-columns: 1fr; }
}
@media (max-width: 560px) {
.wrap { width: min(100% - 24px, 1180px); }
.quick { grid-template-columns: 1fr; }
.metric { min-height: auto; }
table { min-width: 680px; }
}
</style>
<style>main img{max-width:100%;height:auto;display:block;margin:1.4em auto .5em;border-radius:6px}main table{display:block;overflow-x:auto}</style></head>
<body>
<header>
<div class="wrap">
<nav aria-label="Page navigation">
<a class="brand" href="../index.html">Qwen3.6 AEON RYS Docs</a>
<div class="links">
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/">Docs home</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-signallatch/index.html">SignalLatch</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/ckpt386-s010-testing-process/index.html">ckpt386 process</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/signallatch-v1-1-production-process/index.html">v1.1 production</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/signallatch-v1-1-all-results/index.html">v1.1 all results</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/rys-layer-duplication-guide/">RYS arch guide</a>
<a href="https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-15-20/index.html">AEON 15/20</a>
</div>
</nav>
</div>
</header>
<div class="hero">
<div class="wrap">
<div class="eyebrow">PatchCode fine-tune record</div>
<h1>How the PatchCode agentic-coder fine-tune was made, tested, and selected.</h1>
<p class="dek">This page is the self-contained public record for the PatchCode behaviour distil &mdash; an agentic-coder joint LoRA (checkpoint 3661, merged at &lambda;=0.5) on top of the SignalLatch release. It covers the IQ4_NL quant bake-off, the run-to-run noise analysis, and the exact dataset pipeline.</p>
<div class="actions">
<a class="button primary" href="https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF">PatchCode on Hugging Face</a>
<a class="button" href="https://noonr48.github.io/qwen36-aeon-ik-llama/ckpt386-s010-testing-process/index.html">SignalLatch record</a>
<a class="button" href="https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-signallatch/index.html">SignalLatch overview</a>
<a class="button" href="https://noonr48.github.io/qwen36-aeon-ik-llama/rys-layer-duplication-guide/">RYS arch guide</a>
<a class="button" href="https://github.com/noonr48/qwen36-aeon-ik-llama">Runtime fork</a>
</div>
<div class="quick" aria-label="Key release facts">
<div class="metric"><strong>ckpt-3661</strong><span>Final one-epoch agentic-coder joint LoRA checkpoint used for the merge.</span></div>
<div class="metric"><strong>&lambda; = 0.5</strong><span>Selected merge strength (effective alpha/r = 1.0) &mdash; the trained default was over-applied.</span></div>
<div class="metric"><strong>IQ4_NL</strong><span>Shipped 16.6&nbsp;GB GGUF &mdash; ties BF16 within noise at ~&#8531; the size.</span></div>
<div class="metric"><strong>~58.5k</strong><span>Agentic-coding behaviour examples in the training blend (synthetic backbone + curated style slice).</span></div>
</div>
</div>
</div>
<main class="wrap"><div class="wrap">
<p>This is the longer, more casual write-up for the PatchCode upload candidate (internal project name <code>merged_lam0.5</code>).</p>
<p>The clean model card stays short. This document is the full story: what we distilled, exactly how the dataset was built, how we tested it, why the early single-run scores fooled us, why we stopped trusting them, and why the upload candidate ended up being the plain <code>IQ4_NL</code> (reasoning-imatrix) merged GGUF rather than a heavier mixed-quant recipe.</p>
<p>Related public guides:
- runtime fork: <code>https://github.com/noonr48/qwen36-aeon-ik-llama</code>
- RYS layer-duplication / architecture guide: <code>https://github.com/noonr48/qwen36-aeon-ik-llama/tree/main/docs/rys-layer-duplication-guide</code>
- previous fine-tuned release (SignalLatch): <code>https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-SignalLatch-GGUF</code></p>
<p>Related release line:
- previous finetune: <code>Qwen3.6-27B-AEON-RYS-SignalLatch-ckpt386-s010-IQ4_NL.gguf</code>
- this upload candidate: <code>Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf</code></p>
<h2>Glossary</h2>
<ul>
<li><code>AEON</code>: the upstream/source model family this RYS line was built from (<code>AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored</code>).</li>
<li><code>SignalLatch</code> / <code>ckpt386-s010</code>: the previous finetune in this line β€” a behaviour LoRA (checkpoint 386) merged into the AEON RYS base at strength <code>0.10</code>. PatchCode is built on top of this.</li>
<li><code>PatchCode</code> / <code>merged_lam0.5</code>: the public name for this release. It is a second behaviour distil (an agentic-coder joint LoRA) merged onto SignalLatch at strength <code>0.5</code>.</li>
<li><code>IQ4_NL</code>: the quantized GGUF deployment format we actually upload and run.</li>
<li><code>imatrix</code>: importance-matrix-assisted quantization data. <code>reasoning-imatrix</code> = calibrated on reasoning/coding text (the kind that worked); <code>media-imatrix</code> = an earlier calibration kind that underperformed.</li>
<li><code>ik-llama</code>: the custom runtime fork. The <code>qwen3_5</code> hybrid architecture does not load on stock <code>llama.cpp</code> / <code>vLLM</code>.</li>
<li><code>KritaLite</code>: our hardened real-world discriminator build (a ~160k-token multi-file app, 15 binary verifier components). Single-shot coding gates saturate on this model family, so we stopped trusting them.</li>
<li><code>discipline</code> / <code>style_discipline</code>: a rubric measuring the distilled action-first style (no preamble, claim-requires-run, narrate→act→verify).</li>
</ul>
<h2>The short version</h2>
<p>We started from the SignalLatch finetune and distilled a second, agentic-coder behaviour LoRA on top of it. The goal was not a new general chat model. The goal was to make the model a better coding agent: action-first execution, claims backed by an actual run, systematic diagnose→fix loops, stable multi-turn tool use, and fewer stalled runs.</p>
<p>After a full 5-phase bake-off, the model that held up was:</p>
<pre><code class="language-text">Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf
</code></pre>
<p>That means:
- base: <code>Qwen3.6-27B-AEON-RYS-SignalLatch-ckpt386-s010</code>
- adapter: agentic-coder joint LoRA, checkpoint <code>3661</code>
- merge strength: <code>0.5</code> (effective alpha/r = 1.0)
- deploy format: plain <code>IQ4_NL</code> with reasoning-imatrix
- runtime: custom AEON ik-llama fork</p>
<p>The awkward part β€” and the reason this write-up is long β€” is that the eventual ship pick was <strong>not</strong> the candidate that looked best early. A mixed-quant recipe (<code>c76</code>) hit a perfect-looking build score on the first multi-seed pass and did not reproduce. A 5-seed, same-condition confirm reversed the read. The plain <code>IQ4_NL</code> ended up tied with everything else within noise, so the decision fell to non-noise axes (size, recipe safety), where plain <code>IQ4_NL</code> wins.</p>
<h2>What this was meant to upgrade</h2>
<p>PatchCode is an upgrade over the existing SignalLatch finetune:</p>
<pre><code class="language-text">Qwen3.6-27B-AEON-RYS-SignalLatch-ckpt386-s010-IQ4_NL.gguf
</code></pre>
<p>The new work was not another RYS architecture pass (the architecture is unchanged and is documented in the layer-duplication guide). The new work was a behaviour distil layered on top of SignalLatch, then merged and quantized into the same practical Q4-class deployment lane.</p>
<p>Public framing stays narrow:</p>
<blockquote>
<p>This is a practical coding-agent / tool-use-oriented fine-tuned IQ4_NL variant of the SignalLatch release.</p>
</blockquote>
<p>It should not be framed as:
- a universal upgrade over base in every format
- a general chat benchmark win
- a stock <code>llama.cpp</code> / <code>vLLM</code> model
- a live-LoRA deployment recipe</p>
<h2>The dataset β€” exact pipeline</h2>
<p>This is the part most people ask about, so it is written out in full. The training blend is <code>~58.5k</code> examples and is made of two pieces: a large <strong>synthetic coding-agent behaviour backbone</strong> and a smaller <strong>curated action-first style slice</strong>, blended together.</p>
<h3>Piece 1 β€” synthetic coding-agent behaviour backbone (~43k)</h3>
<p>A standalone synthetic generator produces multi-turn coding-agent traces. It is <strong>fully synthetic</strong> β€” no real user data, no scraped repos. The pipeline:</p>
<ol>
<li><strong>Behaviour-driven generation.</strong> A pool of parallel workers calls a coding-agent teacher model. Each call is shaped around a named <em>behaviour</em> from a fixed behaviour pool (~30 behaviours), for example:
- <code>survey_before_edit</code> β€” read/search the real context before touching code
- <code>hypothesis_driven_debugging</code> β€” form a hypothesis, then verify
- <code>tool_intent_first</code> β€” express tool intent before prose
- <code>weigh_alternatives_then_commit</code> β€” weigh β‰₯3 options, commit to one, verify
- <code>external_awareness</code> β€” check versions/docs before asserting
- <code>recall_first_habit</code> β€” recall prior context before re-deriving</li>
<li><strong>Tool-agnostic vocabulary (anti-lock-in).</strong> Tool calls use a behavioural-category vocabulary (e.g. <code>memory_search</code>, <code>repo_search</code>, <code>render_or_visual_proof</code>), not real tool names. This is deliberate: the model learns <em>when/why to use a tool</em>, not a specific vendor's API surface.</li>
<li><strong>Scenarios.</strong> A synthetic scenario bank provides repo-shaped task context (file trees, failing tests, stack traces) so the traces are grounded in realistic edit/verify loops.</li>
<li><strong>Quality gates (per sample).</strong> Traces that fail the gates are dropped, not emitted:
- <code>no-op-edit</code> guard (a claimed edit that changes nothing)
- <code>claim-without-verify</code> reject (the assistant claims done with no run/check)
- <code>reasoning-empty</code> / <code>incomplete-trace</code> / <code>lang-runner-mismatch</code> / <code>prompt-over-cap</code></li>
<li><strong>Deficit-resume scheduling.</strong> Generation runs continuously, tracks per-behaviour deficits, and resumes after interruption until target counts are met (~30 samples/sec).</li>
</ol>
<p><strong>Corpus assembly + filtering (exact counts):</strong>
- raw unified coding corpus: <code>71,776</code> samples
- filter drops <code>10,666</code> bad samples β†’ <code>61,110</code> kept
- top drop reasons: <code>prompt_over_cap</code> 3,946 Β· <code>lang_runner_mismatch</code> 3,645 Β· <code>reasoning_empty</code> 2,086 Β· <code>incomplete_trace</code> 861 Β· <code>claim_without_verify</code> 620
- coding training subset used for the blend: <code>43,075</code></p>
<p>The broader synthetic corpus spans five behaviour layers (media-behaviour 42,973 Β· tool-depth 15,242 Β· reliability 19,393 Β· self-correction 31,476 Β· coding 7,721 = <code>116,805</code> total before filtering); the blend draws the coding-oriented subset.</p>
<h3>Piece 2 β€” curated action-first style slice (~7k)</h3>
<p>A smaller slice of curated execution-style traces that model the exact discipline we wanted to amplify: terse narrate→act→verify, no preamble, claim-requires-run. Composition (<code>6,953</code> total):
- own multi-project execution sessions (<code>5,455</code>) β€” span many different projects on purpose, so the style generalises instead of locking to one domain
- a different-domain contributor (<code>1,130</code>) β€” explicitly included for cross-project transfer
- reasoning-chain exemplars (<code>368</code>) β€” weigh-alternatives deliberation seeds</p>
<p><strong>De-identification / anti-lock-in pass:</strong> real tool names, hostnames, absolute paths, and identifiers are abstracted to behavioural-category tokens / placeholders. The supervision is <strong>assistant-turn-only</strong> β€” system/user/tool turns (where real project content lives) are masked (<code>IGNORE_INDEX</code>), so the model learns a <em>behaviour policy conditioned on varied context</em>, not project facts as outputs.</p>
<h3>Piece 3 β€” the blend</h3>
<p>A small blender oversamples the style slice so it is not drowned by the larger coding backbone, then shuffles:</p>
<ul>
<li>coding backbone: <code>43,075</code></li>
<li>style slice oversampled ~2.2Γ—</li>
<li>blended training set: <code>58,576</code> β‰ˆ <strong>~74% coding backbone / ~26% action-first style</strong></li>
</ul>
<p>The oversample ratio was chosen so the style shows up without overfitting the smaller slice; a held-out task type was used to check it generalises rather than parrots.</p>
<h3>What the dataset is <em>not</em></h3>
<ul>
<li>It is not scraped real-user data or real private repos.</li>
<li>It is not a single-topic dataset β€” both pieces deliberately span many projects/domains.</li>
<li>It does not teach new domain <em>facts</em>; it teaches an execution <em>discipline</em>.</li>
</ul>
<h2>The training piece</h2>
<p>A single LoRA was joint-co-trained on the blended <code>58.5k</code> set (one adapter, not two-then-merge β€” a prior two-adapter Ξ»-merge plan was superseded because post-hoc merges can kill a fragile capability with no usable Ξ»).</p>
<p>Training config:
- PEFT type: <code>LORA</code>
- rank: <code>r=32</code>, alpha: <code>64</code> (alpha/r = 2.0)
- dropout: <code>0.05</code>
- target modules: <strong>all-linear</strong>, including the hybrid arch projections β€” <code>q/k/v/o_proj</code>, <code>gate/up/down_proj</code>, <code>out_proj</code>, and the linear-attn/SSM projections <code>in_proj_qkv / in_proj_a / in_proj_b / in_proj_z</code>
- supervision: completion-only (assistant turns only)
- optimiser: adamw, lr <code>5e-5</code> + warmup + cosine decay
- epochs: <code>1</code>
- backend: model-parallel <code>device_map</code> across a multi-GPU host (the max-quality path; the no-NVLink fleet ruled out DeepSpeed/FSDP here)</p>
<p>Completion:
- <code>global_step=3661</code> = <code>epoch 1.0</code> complete
- final <code>train_loss β‰ˆ 0.853</code>
- runtime ~91h (~89.5 s/it), grad-norm steady (no divergence)
- 37 checkpoints saved across the run β†’ full trajectory available for eval</p>
<p>The adapter was behaviour-focused and small. It was not trained to teach broad new knowledge.</p>
<h2>The merge β€” why Ξ»=0.5</h2>
<p>The trained default adapter strength (alpha/r = 2.0) was <strong>over-applied</strong>. A checkpoint Γ— strength eval showed half-strength beat full-strength on all three tested checkpoints:</p>
<table>
<thead>
<tr>
<th>checkpoint</th>
<th style="text-align: right;">Ξ»=0.3</th>
<th style="text-align: right;">Ξ»=0.5</th>
<th style="text-align: right;">Ξ»=0.7</th>
<th style="text-align: right;">Ξ»=1.0</th>
</tr>
</thead>
<tbody>
<tr>
<td>3661</td>
<td style="text-align: right;">0.522</td>
<td style="text-align: right;"><strong>0.617</strong></td>
<td style="text-align: right;">0.490</td>
<td style="text-align: right;">0.491</td>
</tr>
<tr>
<td>2600</td>
<td style="text-align: right;">0.567</td>
<td style="text-align: right;"><strong>0.573</strong></td>
<td style="text-align: right;">β€”</td>
<td style="text-align: right;">0.442</td>
</tr>
<tr>
<td>1800</td>
<td style="text-align: right;">0.540</td>
<td style="text-align: right;"><strong>0.564</strong></td>
<td style="text-align: right;">β€”</td>
<td style="text-align: right;">0.397</td>
</tr>
</tbody>
</table>
<p>At Ξ»=1.0 the adapter was net-neutral-to-harmful (one checkpoint fell <em>below</em> the un-adapted base). The mechanism: an over-loud LoRA delta pushes activations into regimes that hurt calibrated behaviour (preamble returns, over-claiming). Ξ»=0.5 (effective alpha/r = 1.0) keeps the style direction but respects base calibration. So the merge was done at <strong>Ξ»=0.5 onto SignalLatch (ckpt386-s010)</strong>, then exported to BF16 GGUF. (A future v2 could bake the good strength in by training at alpha=r=32, removing the inference-time knob.)</p>
<p><img alt="Merge strength sweep β€” Ξ»=0.5 wins on all three checkpoints; trained default Ξ»=1.0 is over-applied." src="https://noonr48.github.io/qwen36-aeon-ik-llama/patchcode-testing-process/assets/lambda_sweep.png" /></p>
<h2>Why the final testing moved to merged IQ4_NL</h2>
<p>The key question was not "best adapter in BF16" β€” it was "what we would actually deploy". The deploy target was a merged GGUF, <code>IQ4_NL</code>, imatrix-quantized, on the custom ik-llama runtime (Jinja + DeepSeek reasoning format + flash attention + graph split, temp <code>0.7</code>).</p>
<p>Live LoRA loading is not the production path for this release (the tested serving profile uses flash attention, which conflicts with live LoRA on this runtime). So the long-term path became: <strong>merge the adapter first, then export + quantize a full GGUF.</strong> That is why the upload is a merged GGUF, not an adapter.</p>
<p>The plain <code>IQ4_NL</code> uses the <strong>reasoning/coding imatrix</strong> (the kind that worked). An earlier build used a media-domain imatrix; it underperformed and was superseded.</p>
<h2>Complete test catalog β€” every run, at a glance</h2>
<p>Thirteen separate test runs fed this decision, plus the behaviour-rubric Ξ»-sweep. This is the full list β€” what each measured, on what, and what it said. The detail for each follows in <em>The testing ladder</em>.</p>
<table>
<thead>
<tr>
<th>#</th>
<th>test</th>
<th>measures</th>
<th>candidates</th>
<th>conditions</th>
<th>headline result</th>
<th>verdict</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Phase 1 β€” single-seed KritaLite</td>
<td>160k-token real-world build</td>
<td>IQ4_NL, c76, c373, BF16</td>
<td>1 seed</td>
<td>IQ4_NL 0.933 vs c76 0.867</td>
<td><strong>noise</strong> β€” did not reproduce</td>
</tr>
<tr>
<td>2</td>
<td>Phase 2 β€” 3-seed KritaLite</td>
<td>build</td>
<td>8 quants + BF16</td>
<td>3 seeds</td>
<td>c76/c404 0.933; IQ4_NL/BF16 0.867</td>
<td>reversed phase 1; mixed recipe led</td>
</tr>
<tr>
<td>3</td>
<td>Phase 3 β€” 40-recipe broad search</td>
<td>build</td>
<td>40 mixed recipes</td>
<td>3 seeds</td>
<td>all-zero</td>
<td><strong>harness bug</strong> (missing <code>config.json</code>) β€” void</td>
</tr>
<tr>
<td>4</td>
<td>Phase 4 β€” search re-gate</td>
<td>build</td>
<td>53 candidates</td>
<td>bug fixed</td>
<td>none beat the curated originals</td>
<td>broad search doesn't help this merge</td>
</tr>
<tr>
<td>5</td>
<td>Phase 5 β€” discipline rubric</td>
<td>action-first style</td>
<td>5 quants</td>
<td>3 seeds</td>
<td>BF16/IQ4_NL 0.931; c76 0.903</td>
<td>IQ4_NL &amp; BF16 lead discipline</td>
</tr>
<tr>
<td>6</td>
<td>Phase 5 β€” <code>agent_eval_http</code></td>
<td>7-task agentic pass-rate + turns</td>
<td>5 quants</td>
<td>1 pass</td>
<td>c76 27/7 turns; baseline 27/11; c373 31/17 (thrash)</td>
<td>c76 leads process-efficiency</td>
</tr>
<tr>
<td>7</td>
<td>Q5 confirm</td>
<td>build + long-context + discipline</td>
<td>Q5_K_M (uniform, 20 G)</td>
<td>3 seeds</td>
<td>0.867 / 0.988 / 0.806</td>
<td>doesn't clear "both" (build+disc β‰₯ 0.90)</td>
</tr>
<tr>
<td>8</td>
<td>Overnight 2 β€” precision Γ— promotion matrix</td>
<td>build + discipline</td>
<td>8 (q5/q6/q8 Γ— uniform/promoted)</td>
<td>3 seeds</td>
<td>none clear both; promotion kills discipline</td>
<td>precision is <strong>not</strong> the build lever</td>
</tr>
<tr>
<td>9</td>
<td><strong>Confirm β€” 5-seed head-to-head</strong></td>
<td>build + long-context + discipline</td>
<td>IQ4_NL vs c76</td>
<td><strong>5 seeds, same-condition</strong></td>
<td>IQ4_NL 0.920 vs c76 0.907 (Ξ” 0.013 β‰ͺ 0.067)</td>
<td><strong>TIED within noise β€” the decisive test</strong></td>
</tr>
<tr>
<td>10</td>
<td>Q8 confirm</td>
<td>build + long-context + discipline</td>
<td>Q8_0 vs IQ4_NL</td>
<td>5 seeds</td>
<td>Q8 0.867 vs IQ4_NL 0.920</td>
<td>no edge; near-lossless buys nothing</td>
</tr>
<tr>
<td>11</td>
<td>Agentic-loop</td>
<td>40 held-out mini-projects; pytest-verified convergence + turns + recovery + stall</td>
<td>IQ4_NL, c76, Q8</td>
<td>40 tasks</td>
<td><strong>all 100 % convergence, ~6.6–7.2 turns, 0 % stall</strong></td>
<td><strong>did not discriminate</strong> β€” every quant (incl. base) converges; a family property, not a PatchCode distinction</td>
</tr>
<tr>
<td>12</td>
<td>SignalLatch 4-gate suite</td>
<td>coding/habits + hard-reasoning + long-context (exact + rubric)</td>
<td>IQ4_NL vs BF16</td>
<td>n=12 hard, n=4 longctx</td>
<td>IQ4_NL 0.887 vs BF16 0.846 (hard); 0.979 vs 0.941 (long); 0 errors</td>
<td>IQ4_NL tracks/edges BF16 within noise</td>
</tr>
<tr>
<td>13</td>
<td>Behaviour rubric β€” Ξ»-sweep</td>
<td>action-first style + coding discipline + held-out generalization</td>
<td>base (SignalLatch) vs PatchCode @ Ξ»{0.3,0.5,0.7,1.0,1.3}, ckpt{3661,2600,1800}</td>
<td>15 cases Γ— strengths</td>
<td>base 0.486 β†’ PatchCode Ξ»0.5 0.617 (~β…“ the tokens)</td>
<td>PatchCode beats base; Ξ»0.5 is the sweet spot</td>
</tr>
</tbody>
</table>
<p><strong>The only test that discriminated was #9</strong> (the 5-seed confirm) β€” and it discriminated by showing everything is <em>tied within noise</em>, which pushed the decision onto non-noise axes (size + plain-quant recipe), where IQ4_NL wins. Tests #1 and #3 were void (noise / harness bug). Tests #5–#8, #10 and #11 all failed to separate the finalists. #12 confirms IQ4_NL is not a quality cliff below BF16. #13 is the one place PatchCode clearly beats its SignalLatch base.</p>
<h2>The testing ladder (5 phases + confirms)</h2>
<p>Single-shot and hard-suite gates <strong>saturate</strong> on this model family (every quant scores ~the same, including BF16). The discrimination that actually changed the decision came from a 160k-token real-world build (KritaLite) run multi-seed, plus a discipline rubric, plus an agentic-process efficiency probe. The phases:</p>
<p><strong>Phase 1 β€” single-seed real-world build.</strong> Made the plain <code>IQ4_NL</code> look like the winner (0.933 vs c76's 0.867). This was <strong>noise</strong> β€” it did not reproduce.</p>
<p><strong>Phase 2 β€” multi-seed KritaLite (3 seeds).</strong> Reversed phase 1: <code>c76</code>/<code>c404</code>/<code>c373</code> hit 0.933; plain <code>IQ4_NL</code> dropped to 0.867. Now a mixed-quant recipe looked like the winner.</p>
<p><strong>Phase 3 β€” 40-recipe broad search.</strong> Returned all-zero. Root cause was a <strong>harness bug</strong> (the eval script imports a <code>config.json</code> that was not copied into the eval root), not real scores.</p>
<p><strong>Phase 4 β€” search re-gate (bug fixed).</strong> Re-scored all 53 candidates correctly. No new recipe beat the curated originals; the broad search does not help this merge.</p>
<p><strong>Phase 5 β€” discipline + agentic process.</strong> Plain <code>IQ4_NL</code> and BF16 led the action-first <em>discipline</em> rubric (0.931); <code>c76</code> led <em>process efficiency</em> (fewest turns/tools/errors).</p>
<p><strong>Overnight 2 β€” base-precision Γ— attention-promotion matrix (3 seeds).</strong> Decomposed the build/discipline tradeoff. No candidate clears "both" (build β‰₯ 0.90 <strong>and</strong> discipline β‰₯ 0.90):
- promotion destroys discipline regardless of base precision
- uniform higher precision does <strong>not</strong> fix build (build is not precision-limited)</p>
<p><strong>Confirm β€” 5-seed, same-condition, baseline vs c76 head-to-head.</strong> The decisive run:</p>
<table>
<thead>
<tr>
<th>candidate</th>
<th style="text-align: right;">build (5-seed)</th>
<th style="text-align: right;">long-context</th>
<th style="text-align: right;">discipline (5-seed)</th>
<th style="text-align: right;">size</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>plain IQ4_NL (reasoning imx)</strong></td>
<td style="text-align: right;">0.920 (Β±0.067)</td>
<td style="text-align: right;">0.975</td>
<td style="text-align: right;">0.842 (Β±0.333)</td>
<td style="text-align: right;">16.6 G</td>
</tr>
<tr>
<td>c76 (promoted attn)</td>
<td style="text-align: right;">0.907 (Β±0.067)</td>
<td style="text-align: right;">0.935</td>
<td style="text-align: right;">0.867 (Β±0.292)</td>
<td style="text-align: right;">20 G</td>
</tr>
</tbody>
</table>
<p>build gap <code>0.013</code> β‰ͺ <code>0.067</code> noise floor β†’ <strong>not discriminating</strong>. c76's earlier "0.933 build win" did not reproduce (it scored 0.933 β†’ 0.867 β†’ 0.907 across passes β€” pure run-to-run variance).</p>
<p><strong>Q8 confirm β€” 5-seed, near-lossless Q8 vs plain IQ4_NL.</strong> Q8 shows no edge on any axis and is ~2Γ— the size β†’ ruled out. Near-lossless precision buys nothing measurable here.</p>
<p><strong>Behaviour rubric β€” PatchCode vs the base it was distilled from.</strong> A 15-case rubric (action-first style + coding discipline + held-out generalization) was run across merge strengths, with the adapter disabled as the "strength 0" anchor β€” i.e. the SignalLatch base PatchCode was built on. PatchCode at the chosen Ξ»=0.5 beat the base on score while emitting far fewer tokens:</p>
<table>
<thead>
<tr>
<th>variant (15-case rubric)</th>
<th style="text-align: right;">score</th>
<th style="text-align: right;">avg output tokens</th>
<th style="text-align: right;">avg time/case</th>
</tr>
</thead>
<tbody>
<tr>
<td>base (adapter off = SignalLatch)</td>
<td style="text-align: right;"><code>0.486</code></td>
<td style="text-align: right;"><code>311</code></td>
<td style="text-align: right;"><code>34s</code></td>
</tr>
<tr>
<td>PatchCode (ckpt-3661 @ Ξ»=0.5)</td>
<td style="text-align: right;"><code>0.617</code></td>
<td style="text-align: right;"><code>91</code></td>
<td style="text-align: right;"><code>13s</code></td>
</tr>
</tbody>
</table>
<p>The base tended to ramble (~311 tokens of hedging preamble β€” e.g. it scored 0.20 on the coding-discipline case with "I might overwrite the user's changes…"); PatchCode was terse and on-target (~91 tokens) and scored higher. That is the distil's intended effect: more disciplined execution, less wasted output. Caveats: this is a behaviour rubric, not a multi-turn agent turn-count; Ξ»=0.5 is the sweet spot β€” higher strengths (0.7 / 1.0 / 1.3) also got terse (~60 tokens) but fell <em>below</em> the base (0.39–0.49), so terseness alone is not the win; single-temperature, small per-category N.</p>
<p><strong>Q5_K_M confirm β€” uniform Q5 (3 seeds).</strong> Does a uniform higher precision (no selective promotion) clear "both"? <code>Q5_K_M</code> (20 G, imatrix-calibrated): build <code>0.867</code> (Β±0.133), long-context <code>0.988</code>, discipline <code>0.806</code> (Β±0.292) β†’ build and discipline both below 0.90. Uniform-precision does not fix build and erodes discipline. Ruled out.</p>
<p><strong>Agentic-loop β€” autonomous convergence (40 held-out mini-projects).</strong> Each quant ran 40 held-out mini-projects (a README plus a <em>failing</em> pytest suite) fully autonomously: reason β†’ read β†’ implement β†’ run tests β†’ fix β†’ converge. Convergence is <strong>objective pytest pass, not self-claimed.</strong></p>
<table>
<thead>
<tr>
<th>quant</th>
<th>n</th>
<th>convergence</th>
<th>mean turns (converged)</th>
<th>recovery (mean)</th>
<th>stall</th>
</tr>
</thead>
<tbody>
<tr>
<td>c76</td>
<td>40</td>
<td><code>100%</code></td>
<td>6.6</td>
<td>0.4</td>
<td><code>0%</code></td>
</tr>
<tr>
<td>Q8_0</td>
<td>40</td>
<td><code>100%</code></td>
<td>7.0</td>
<td>0.5</td>
<td><code>0%</code></td>
</tr>
<tr>
<td>IQ4_NL</td>
<td>40</td>
<td><code>100%</code></td>
<td>7.2</td>
<td>0.4</td>
<td><code>0%</code></td>
</tr>
</tbody>
</table>
<p>This axis <strong>did not discriminate</strong> β€” every quant (including the un-adapted base behaviour) converged on all 40 tasks, so autonomous convergence is a property of the model <em>family</em> on these tasks, not a PatchCode distinction. It does not favour any ship candidate, and the decision falls to size + recipe methodology. (Per-task: 8 tasks Γ— 5 reps each, all 5/5 for every quant β€” <code>calc</code>, <code>debug_stack</code>, <code>graph</code>, <code>lru</code>, <code>mdlist</code>, <code>minijson</code>, <code>taskq</code>, <code>tracker</code>.)</p>
<p><strong>SignalLatch gate suite β€” IQ4_NL vs BF16.</strong> The established four-type gate set (coding/habits, hard-reasoning, hard-project, long-context) run on the PatchCode merge in both formats. Both clear every gate with <strong>zero errors</strong>; IQ4_NL tracks or nominally edges BF16. The ~0.04 gaps sit inside the build noise floor, so this reads as <em>tied</em>, not an IQ4_NL win.</p>
<table>
<thead>
<tr>
<th>gate (cases)</th>
<th>PatchCode IQ4_NL</th>
<th>BF16 (control)</th>
</tr>
</thead>
<tbody>
<tr>
<td>coding / habits</td>
<td><code>0.958</code></td>
<td><code>0.917</code></td>
</tr>
<tr>
<td>hard-reasoning</td>
<td><code>0.789</code></td>
<td><code>0.751</code></td>
</tr>
<tr>
<td>long-context (4)</td>
<td><code>0.979</code></td>
<td><code>0.941</code></td>
</tr>
<tr>
<td><strong>weighted overall</strong></td>
<td><strong><code>0.887</code></strong></td>
<td><code>0.846</code></td>
</tr>
</tbody>
</table>
<h2>The noise lesson (critical β€” reuse for every future bake-off)</h2>
<p>The SignalLatch-style suite is <strong>noisier than it looked</strong>:
- KritaLite build: Β±0.067–0.13 <strong>run-to-run</strong> variance (beyond seed). c76 scored 0.933 β†’ 0.867 β†’ 0.907 on the same gguf.
- discipline: Β±0.3 spread.
- build is <strong>ceiling-limited</strong> (max 0.933 = 14/15) β†’ zero headroom to discriminate two good quants.</p>
<p><strong>Rule:</strong> 3-seed differences &lt;0.13 on this suite are meaningless. Use <strong>5+ seeds, same-condition head-to-head</strong> before any ship call. Only non-noise axes (size, recipe methodology/safety, long-context at ceiling) reliably tiebreak. HumanEval was rejected β€” it saturates on Qwen and is the wrong mode for an agent.</p>
<p>This is exactly how a 3-seed pass almost shipped the <em>weaker</em> model.</p>
<h2>The ship decision</h2>
<p>With build, discipline, and long-context all <strong>tied within noise</strong>, the decision fell to non-noise axes, where plain <code>IQ4_NL</code> wins all three:</p>
<p><img alt="No candidate clears BOTH build and discipline (β‰₯0.90) β€” promotion destroys discipline; precision does not fix build." src="https://noonr48.github.io/qwen36-aeon-ik-llama/patchcode-testing-process/assets/bothquest.png" /></p>
<p><img alt="Ship scoreboard (5-seed): IQ4_NL ties the field within noise on build/long-context/discipline, and wins on size." src="https://noonr48.github.io/qwen36-aeon-ik-llama/patchcode-testing-process/assets/ship_scoreboard.png" /></p>
<ul>
<li><strong>smaller</strong> (16.6 G vs 20–29 G)</li>
<li><strong>marginal long-context</strong> edge (0.975 vs 0.935–0.969)</li>
<li><strong>plain-quant recipe</strong> β€” the fleet's proven pattern; promotion/mixed recipes carry evidence-harmful risk (discipline collapse) for zero measured benefit</li>
</ul>
<p>Ship: <strong>plain <code>IQ4_NL</code> (reasoning-imatrix)</strong>. The mixed-recipe <code>c76</code> is retained on disk as the build-heavy fallback if a future, harder build-gate ever discriminates beyond the noise floor (use 5+ seeds).</p>
<h2>What the testing says and does not say</h2>
<p><strong>Does say:</strong>
- PatchCode's distilled action-first discipline is preserved through <code>IQ4_NL</code> (tied with BF16 across build / long-context / discipline).
- Near-lossless precision (Q8) and attention promotion buy no measurable edge on this suite.
- Plain <code>IQ4_NL</code> is the defensible default on size + recipe safety.</p>
<p><strong>Does not say:</strong>
- It does not prove PatchCode is better for all tasks.
- It does not prove plain <code>IQ4_NL</code> is globally optimal.
- It does not make this a stock <code>llama.cpp</code> / <code>vLLM</code> release.
- It does not make live LoRA loading the recommended serving setup.</p>
<p>The most accurate public sentence:</p>
<blockquote>
<p>On a 5-seed, same-condition practical coding-agent bake-off, PatchCode plain <code>IQ4_NL</code> tied BF16 within noise on build, long-context, and discipline, and was the selected default on size and recipe safety.</p>
</blockquote>
<h2>Selected artifact</h2>
<pre><code class="language-text">Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf (16.6 GB β€” recommended)
Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.BF16.gguf (57.6 GB β€” source-quality reference)
</code></pre>
<p>Recommended runtime: <code>https://github.com/noonr48/qwen36-aeon-ik-llama</code></p>
<pre><code class="language-bash">./build/bin/llama-server \
-m /path/to/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf \
-c 65536 -ngl 999 -np 1 -fa on -sm none \
--temp 0.7 --jinja --reasoning-format deepseek --reasoning-budget 0
</code></pre>
<p>(<code>&lt;think&gt;</code> is emitted as a separate <code>reasoning_content</code> field β€” use <code>--reasoning-format deepseek</code> or fold it back so tool-action parsing sees the action.)</p>
<h2>Final read</h2>
<p>This was not a clean leaderboard. It was a real engineering pass: distil the style, build a hardened discriminator because the easy gates saturated, get fooled by a one-run perfect build score, repeat the finalists same-condition, discover the build is ceiling-limited and noisy, and ship the smallest plain-quant that ties everything within noise.</p>
<pre><code class="language-text">PatchCode IQ4_NL is a practical agentic-coder upgrade over the SignalLatch release.
It is the selected default among the tested quants, tied with BF16 within noise β€”
not a universal final answer.
</code></pre></div></main>
<footer>
<div class="wrap">
Last updated: 2026-06-29. This page documents the PatchCode (agentic-coder) fine-tune and quant bake-off for the Qwen3.6 AEON RYS / SignalLatch line.</div>
</footer>
</body>
</html>