| <title>stcoder-qwen35-9b — Mikrodev STCoder for PLC Structured Text</title> |
| <style> |
| :root{--bg:#e9edf1;--paper:#fff;--paper2:#f4f7f9;--rule:#d3dae1;--ink:#131a21;--mid:#5c6b78;--dim:#8b98a5; |
| --teal:#0e7361;--teal-soft:#d6eef1;--teal-line:#93cfd7;--amber:#9a6512;--amber-soft:#f2e4c9;--amber-line:#d8bb84; |
| --red:#ac3527;--red-soft:#f6dcd7;--code-bg:#f6f8fa;--code-ink:#1e2831; |
| --shadow:0 1px 1px rgba(19,26,33,.04),0 8px 26px rgba(19,26,33,.06);} |
| @media (prefers-color-scheme:dark){:root{--bg:#0b1014;--paper:#131b22;--paper2:#18222a;--rule:#25313b; |
| --ink:#d5dee6;--mid:#8c9aa7;--dim:#5f6d79;--teal:#37b898;--teal-soft:#10353a;--teal-line:#1d5b65; |
| --amber:#d09a3f;--amber-soft:#382c12;--amber-line:#6b5322;--red:#e07963;--red-soft:#3d1d17; |
| --code-bg:#0e161c;--code-ink:#c2ccd6;--shadow:0 1px 1px rgba(0,0,0,.4),0 10px 30px rgba(0,0,0,.34);}} |
| :root[data-theme=light]{--bg:#e9edf1;--paper:#fff;--paper2:#f4f7f9;--rule:#d3dae1;--ink:#131a21;--mid:#5c6b78; |
| --dim:#8b98a5;--teal:#0e7361;--teal-soft:#d6eef1;--teal-line:#93cfd7;--amber:#9a6512;--amber-soft:#f2e4c9; |
| --amber-line:#d8bb84;--red:#ac3527;--red-soft:#f6dcd7;--code-bg:#f6f8fa;--code-ink:#1e2831; |
| --shadow:0 1px 1px rgba(19,26,33,.04),0 8px 26px rgba(19,26,33,.06);} |
| :root[data-theme=dark]{--bg:#0b1014;--paper:#131b22;--paper2:#18222a;--rule:#25313b;--ink:#d5dee6;--mid:#8c9aa7; |
| --dim:#5f6d79;--teal:#37b898;--teal-soft:#10353a;--teal-line:#1d5b65;--amber:#d09a3f;--amber-soft:#382c12; |
| --amber-line:#6b5322;--red:#e07963;--red-soft:#3d1d17;--code-bg:#0e161c;--code-ink:#c2ccd6; |
| --shadow:0 1px 1px rgba(0,0,0,.4),0 10px 30px rgba(0,0,0,.34);} |
| *{box-sizing:border-box} |
| body{margin:0;background:var(--bg);color:var(--ink);line-height:1.6;-webkit-font-smoothing:antialiased; |
| font-family:"Segoe UI",system-ui,-apple-system,Roboto,Helvetica,Arial,sans-serif} |
| .sheet{max-width:1080px;margin:0 auto;padding:clamp(18px,3.5vw,50px) clamp(14px,3.5vw,42px) 90px} |
| code,pre,.mono{font-family:"Cascadia Code","JetBrains Mono",Consolas,ui-monospace,SFMono-Regular,Menlo,monospace} |
| .num,table,.bar b{font-variant-numeric:tabular-nums} |
| .mast{border-top:3px solid var(--ink);padding-top:14px} |
| .kick{display:flex;flex-wrap:wrap;gap:13px;align-items:baseline;font-family:"Cascadia Code",Consolas,monospace; |
| font-size:11px;letter-spacing:.16em;text-transform:uppercase;color:var(--mid)} |
| .kick .id{color:var(--teal);font-weight:700} |
| h1{font-size:clamp(28px,5vw,47px);line-height:1.03;letter-spacing:-.025em;margin:.3em 0 .16em;font-weight:800; |
| text-wrap:balance;max-width:24ch} |
| .sf{font-size:clamp(16px,2vw,19px);color:var(--mid);max-width:66ch;margin:0} |
| .sf b{color:var(--ink);font-weight:650} |
| .spec{display:grid;grid-template-columns:repeat(auto-fit,minmax(128px,1fr));gap:1px;background:var(--rule); |
| border:1px solid var(--rule);border-radius:10px;overflow:hidden;margin:28px 0 0;box-shadow:var(--shadow)} |
| .spec div{background:var(--paper);padding:12px 14px} |
| .spec .k{font-family:"Cascadia Code",Consolas,monospace;font-size:10px;letter-spacing:.13em; |
| text-transform:uppercase;color:var(--dim)} |
| .spec .v{font-size:19px;font-weight:750;letter-spacing:-.01em;margin-top:3px;word-break:break-word} |
| .spec .v small{font-size:11.5px;font-weight:500;color:var(--mid);letter-spacing:0;display:block} |
| section{margin-top:clamp(42px,6vw,70px)} |
| .sh{display:flex;gap:13px;align-items:baseline;border-bottom:1px solid var(--rule);padding-bottom:9px;margin-bottom:18px} |
| .sh .sn{font-family:"Cascadia Code",Consolas,monospace;font-size:12px;font-weight:700;color:var(--teal);letter-spacing:.08em} |
| h2{font-size:clamp(19px,2.6vw,26px);margin:0;letter-spacing:-.015em;font-weight:750;text-wrap:balance} |
| h3{font-size:16px;margin:24px 0 7px;font-weight:700} |
| p{max-width:70ch}p.lead{color:var(--mid)} |
| ul,ol{max-width:72ch;padding-left:20px}li{margin:.3em 0} |
| .card{background:var(--paper);border:1px solid var(--rule);border-radius:12px;padding:18px 20px; |
| margin:18px 0;box-shadow:var(--shadow)} |
| .card.hi{border-left:3px solid var(--teal)} |
| .card.warn{border-left:3px solid var(--red)} |
| .card h3{margin-top:0} |
| .tw{overflow-x:auto;border:1px solid var(--rule);border-radius:11px;box-shadow:var(--shadow);margin:18px 0} |
| table{border-collapse:collapse;width:100%;background:var(--paper);font-size:13.2px;min-width:600px} |
| th,td{padding:10px 13px;text-align:right;border-bottom:1px solid var(--rule);vertical-align:top} |
| th:first-child,td:first-child{text-align:left} |
| thead th{background:var(--paper2);font-size:10.4px;letter-spacing:.1em;text-transform:uppercase;color:var(--dim);font-weight:700} |
| tbody tr:last-child td{border-bottom:none} |
| td.ft{color:var(--teal);font-weight:700}td.bs{color:var(--amber);font-weight:650} |
| .pick{display:grid;gap:1px;background:var(--rule);border:1px solid var(--rule);border-radius:11px; |
| overflow:hidden;box-shadow:var(--shadow)} |
| .pick .row{background:var(--paper);padding:14px 17px;display:grid;grid-template-columns:120px 150px 1fr;gap:14px;align-items:start} |
| @media(max-width:700px){.pick .row{grid-template-columns:1fr;gap:5px}} |
| .pick .vram{font-weight:750;font-size:15px} |
| .pick .q{font-family:"Cascadia Code",Consolas,monospace;font-size:12.6px;font-weight:700;color:var(--teal)} |
| .pick .q small{display:block;color:var(--mid);font-weight:400;font-size:11px} |
| .pick .why{color:var(--mid);font-size:13.5px} |
| .pick .why em{color:var(--amber);font-style:normal;font-weight:650} |
| .steps{counter-reset:s;list-style:none;padding:0} |
| .steps li{counter-increment:s;position:relative;padding-left:38px;margin-bottom:16px} |
| .steps li::before{content:counter(s);position:absolute;left:0;top:0;width:25px;height:25px;border-radius:50%; |
| background:var(--teal-soft);color:var(--teal);font-family:"Cascadia Code",Consolas,monospace;font-size:12px; |
| font-weight:700;display:flex;align-items:center;justify-content:center} |
| .steps b{display:block;font-size:14.5px} |
| .steps p{margin:.2em 0;color:var(--mid);font-size:13.5px} |
| pre{margin:9px 0 0;padding:12px 14px;overflow-x:auto;background:var(--code-bg);color:var(--code-ink); |
| font-size:12.3px;line-height:1.55;border-radius:8px;border:1px solid var(--rule)} |
| pre.plain{border-radius:0;border:none;margin:0} |
| pre .bad{background:var(--red-soft);color:var(--red);font-weight:700;border-radius:3px;padding:0 2px} |
| pre .good{background:var(--teal-soft);color:var(--teal);font-weight:700;border-radius:3px;padding:0 2px} |
| .pgrid{display:grid;grid-template-columns:repeat(auto-fit,minmax(292px,1fr));gap:13px} |
| .pc{background:var(--paper);border:1px solid var(--rule);border-radius:11px;padding:14px 16px;box-shadow:var(--shadow)} |
| .pc .q{font-size:14px;font-weight:650;margin-bottom:6px} |
| .pc .q::before{content:"\201C"}.pc .q::after{content:"\201D"} |
| .pc .e{font-size:13px;color:var(--mid)} |
| .pc .n{font-size:12.2px;color:var(--dim);margin-top:6px;font-style:italic} |
| .fig{background:var(--paper);border:1px solid var(--rule);border-radius:12px;padding:19px 21px 15px; |
| margin:20px 0;box-shadow:var(--shadow)} |
| .fc{font-family:"Cascadia Code",Consolas,monospace;font-size:10.5px;letter-spacing:.14em;text-transform:uppercase; |
| color:var(--dim);margin-bottom:4px} |
| .ftl{font-weight:700;font-size:15px;margin-bottom:13px} |
| .legend{display:flex;flex-wrap:wrap;gap:15px;font-size:12.3px;color:var(--mid);margin:0 0 14px} |
| .legend span{display:inline-flex;align-items:center;gap:6px} |
| .sw{width:11px;height:11px;border-radius:2px}.sw.b{background:var(--amber)}.sw.f{background:var(--teal)} |
| .sw.r{background:var(--red)} |
| .frow{display:grid;grid-template-columns:180px 1fr;gap:13px;align-items:center;margin-bottom:14px} |
| @media(max-width:640px){.frow{grid-template-columns:1fr;gap:5px}} |
| .flab{font-size:13px;font-weight:650} |
| .flab small{display:block;font-weight:400;color:var(--mid);font-size:10.6px;font-family:"Cascadia Code",Consolas,monospace} |
| .fbars{display:flex;flex-direction:column;gap:5px} |
| .bar{position:relative;height:23px;background:var(--paper2);border:1px solid var(--rule);border-radius:5px;overflow:hidden} |
| .bar>i{position:absolute;inset:0 auto 0 0;display:block} |
| .bar.base>i{background:var(--amber)}.bar.base2>i{background:var(--amber);opacity:.6} |
| .bar.ft>i{background:var(--teal)}.bar.ft2>i{background:var(--teal);opacity:.6} |
| .bar>b{position:absolute;right:8px;top:0;height:23px;display:flex;align-items:center;font-size:11.4px; |
| font-weight:700;color:var(--ink)} |
| .strip{display:flex;gap:2px;height:26px} |
| .strip i{flex:1 1 0;border-radius:1.5px;min-width:3px} |
| .strip i.ok{background:var(--teal)}.strip i.vi{background:var(--amber)}.strip i.nc{background:var(--red)} |
| .diff{display:grid;grid-template-columns:1fr 1fr;gap:14px} |
| @media(max-width:820px){.diff{grid-template-columns:1fr}} |
| .panel{background:var(--paper);border:1px solid var(--rule);border-radius:11px;overflow:hidden;box-shadow:var(--shadow)} |
| .panel.base{border-top:3px solid var(--amber-line)}.panel.ft{border-top:3px solid var(--teal-line)} |
| .ph{display:flex;justify-content:space-between;align-items:center;gap:8px;padding:9px 13px;border-bottom:1px solid var(--rule)} |
| .tag{font-family:"Cascadia Code",Consolas,monospace;font-size:10.3px;letter-spacing:.12em;text-transform:uppercase;font-weight:700} |
| .tag.base{color:var(--amber)}.tag.ft{color:var(--teal)} |
| .badge{font-family:"Cascadia Code",Consolas,monospace;font-size:10.7px;font-weight:700;padding:3px 8px; |
| border-radius:99px;white-space:nowrap} |
| .badge.ok{background:var(--teal-soft);color:var(--teal)}.badge.warn{background:var(--amber-soft);color:var(--amber)} |
| .pv{display:flex;flex-wrap:wrap;gap:5px;padding:9px 13px;border-top:1px solid var(--rule)} |
| .pv code{font-size:10.5px;padding:2px 7px;border-radius:99px} |
| .pv code.v{background:var(--red-soft);color:var(--red)}.pv code.c{background:var(--teal-soft);color:var(--teal)} |
| .note{font-size:13.2px;color:var(--mid);border-left:2px solid var(--rule);padding-left:13px;margin:12px 0 0;max-width:82ch} |
| .note b{color:var(--ink);font-weight:650} |
| .chip{display:inline-block;font-family:"Cascadia Code",Consolas,monospace;font-size:10.3px;font-weight:700; |
| padding:2px 7px;border-radius:99px} |
| .chip.ok{background:var(--teal-soft);color:var(--teal)}.chip.vi{background:var(--amber-soft);color:var(--amber)} |
| .chip.nc{background:var(--red-soft);color:var(--red)} |
| .sib{display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));gap:11px;margin-top:14px} |
| .sib a{display:block;background:var(--paper);border:1px solid var(--rule);border-radius:10px;padding:12px 14px; |
| text-decoration:none;color:inherit;box-shadow:var(--shadow)} |
| .sib a.self{border-color:var(--teal);background:var(--teal-soft)} |
| .sib .sn2{font-family:"Cascadia Code",Consolas,monospace;font-size:11.6px;font-weight:700;color:var(--teal)} |
| .sib .sd{font-size:12.6px;color:var(--mid);margin-top:2px} |
| footer{margin-top:60px;padding-top:19px;border-top:1px solid var(--rule);font-size:12.4px;color:var(--dim);max-width:82ch} |
| footer code{color:var(--mid);font-size:11.6px} |
| </style> |
| <div class="sheet"> |
|
|
| <div class="mast"> |
| <div class="kick"><span class="id">Mikrodev/stcoder-qwen35-9b-gguf</span><span>Fine-tuned from Qwen/Qwen3.5-9B</span> |
| <span>apache-2.0</span><span>GGUF · Ollama / llama.cpp</span></div> |
| <h1>stcoder-qwen35-9b</h1> |
| <p class="sf">stcoder-qwen35-9b is a chat-only fine-tune of Qwen3.5-9B that writes Mikrodev LogicStudio Structured Text; it is the only member of the line built on a reasoning base, so it thinks before it answers — which buys clean, reference-shaped code but costs roughly four times the tokens of its siblings and cost it one non-delivery in fifteen.</p> |
| <p class="sf" style="font-size:15px;margin-top:10px">This is a fine-tune for the Structured Text coding rules of Advance Logic Builder (ALB), the IEC 61131-3 engineering environment developed by Mikrodev and shipped as Mikrodev LogicStudio. It follows Mikrodev's ST dialect, not generic IEC 61131-3 and not another vendor's conventions.</p> |
| </div> |
|
|
| <div class="spec"> |
| <div><div class="k">Base model</div><div class="v" style="font-size:14px">Qwen3.5-9B<small>Qwen/Qwen3.5-9B</small></div></div> |
| <div><div class="k">Builds</div><div class="v">3<small>Q4_K_M / Q6_K / Q8_0</small></div></div> |
| <div><div class="k">Smallest</div><div class="v">5.4 GiB<small>Q4_K_M</small></div></div> |
| <div><div class="k">Recommended</div><div class="v" style="color:var(--teal)">Q8_0<small>9.11 GiB</small></div></div> |
| <div><div class="k">Dialect-clean</div><div class="v" style="color:var(--teal)">93.3%<small>base: 6.7%</small></div></div> |
| <div><div class="k">Tool calling</div><div class="v" style="color:var(--red);font-size:15px">Not supported<small>chat only</small></div></div> |
| </div> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">1</span><h2>Which build should you download?</h2></div> |
| <p class="lead">Start from the graphics memory you have. Every build below is the same fine-tune; |
| they differ only in how heavily the weights are compressed.</p> |
| <div class="pick"><div class="row"><div class="vram">11.1 GiB free VRAM</div><div class="q">Q8_0<small>qwen35_9b-tc.q8_0.gguf · 9.11 GiB</small></div><div class="why">recommended — the build this study ran</div></div><div class="row"><div class="vram">9.0 GiB free VRAM</div><div class="q">Q6_K<small>qwen35_9b-tc.q6_k.gguf · 7.04 GiB</small></div><div class="why">fine</div></div><div class="row"><div class="vram">7.4 GiB free VRAM</div><div class="q">Q4_K_M<small>qwen35_9b-tc.q4_k_m.gguf · 5.38 GiB</small></div><div class="why">not recommended</div></div></div> |
| <div class="card hi"><h3>Short answer</h3> |
| <p>Download <b>Q8_0</b> (<code>qwen35_9b-tc.q8_0.gguf</code>, 9.11 GiB). |
| Q8_0 is the smallest build with no practically measurable loss against the trained weights - it is the precision we recommend for real work. Free VRAM needed: about |
| 11.1 GiB — the file plus the 8192-token context and runtime.</p></div> |
| <div class="card warn"><h3>Why not Q4_K_M</h3> |
| <p>Q4_K_M halves the bits per weight again. The only capability we measured across quantisations degraded sharply - tool-call composite 0.816 at Q8_0 against 0.642 at Q4_K_M, measured on the Qwen3.5-9B family, the only one of the four where tool-calling worked at all - and low-bit quantisation is a known source of drift and hallucination on long or unusual requests. Our 15-prompt code study never ran at Q4, so we have no measured code-quality figure for it: treat this as a precaution, not a measured code-quality gap.</p></div> |
| <p class="note"><b>Also published for this model.</b> The LoRA adapter it was trained as |
| (<code>lora_adapter/</code> in the same repository) so you can merge it at any |
| precision or continue training from it, and the f16 merged weights |
| (f16 is the merged fine-tune at full precision: the reference build, for evaluation or for producing your own quantisations.). Ready-to-run Ollama <code>Modelfile</code>s ship next to every build: |
| correct chat template, stop tokens, <code>num_ctx 8192</code> — the sequence length this |
| model was fine-tuned at — and the system prompt.</p> |
| <div class="tw"><table> |
| <thead><tr><th>File</th><th>Quantisation</th><th>Size</th><th>Fit on 16 GB</th><th style="text-align:left">Note</th></tr></thead> |
| <tbody><tr><td><code>qwen35_9b-tc.q4_k_m.gguf</code></td><td class="num">Q4_K_M</td><td class="num">5.38 GiB</td><td class="num">solid</td><td></td></tr><tr><td><code>qwen35_9b-tc.q6_k.gguf</code></td><td class="num">Q6_K</td><td class="num">7.04 GiB</td><td class="num">solid</td><td></td></tr><tr><td><code>qwen35_9b-tc.q8_0.gguf</code></td><td class="num">Q8_0</td><td class="num">9.11 GiB</td><td class="num">solid</td><td>tested in this study</td></tr></tbody> |
| </table></div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">2</span><h2>Is this the right model for you?</h2></div> |
| <p>Read this first: tool calling was trained and evaluated for this line and then dropped as a product decision. This build is published and supported as a chat-only model. This family is in fact the one that did learn the skill (tool-call composite 0.816 at Q8_0, 0.805 at Q6_K, collapsing to 0.642 at Q4_K_M) while stcoder-qwen25-7b and stcoder-qwen25-14b scored 0.02-0.06 and stcoder-gemma4-12b scored 0.00. Because it was not usable across the line it is not part of this release and is not supported here. |
|
|
| If you have not already decided, take stcoder-qwen25-7b at Q6_K (5.82 GiB). That is the line default: 5.8 s per answer, the smallest files, 100% delivery, and the best ChrF at full delivery in this study (38.5 over 15 of 15), with composite within noise of everything else. |
|
|
| Pick stcoder-qwen35-9b instead only if you specifically want the reasoning family and can pay for it in latency. On the minimal-prompt condition it scored R24 100.0 with an empty violation table on every delivered reply, 93.3% clean, and composite 0.845 over its 14 delivered replies (0.835 if the missing reply is counted as zero, which puts it behind the 7B's 0.844 and the 14B's 0.842). Its ChrF is 40.9 over those 14 delivered replies — the highest delivered-only figure in the set — but 38.2 over all 15, below the 7B's 38.5; the lead exists only after excluding its own empty reply, so quote the denominator whenever you quote the number. Every reply carries a hidden thinking block (100% of replies in all four arms), averaging 1227 output tokens and 17.2 s per answer at the minimal prompt against 340 tokens and 5.8 s for the 7B. It is the slowest of the four per answer; the 14B is the slowest per token (about 50 tok/s). Accept also that it was the only fine-tune in the study that ever returned no code at all (14 of 15 at the minimal prompt, 15 of 15 with the full rule book).</p> |
| <div class="sib"><a class="" href="https://huggingface.co/Mikrodev/stcoder-qwen25-7b-gguf"><div class="sn2">Mikrodev/stcoder-qwen25-7b-gguf</div><div class="sd">7B · default pick, fastest per answer</div></a><a class="" href="https://huggingface.co/Mikrodev/stcoder-qwen25-14b-gguf"><div class="sn2">Mikrodev/stcoder-qwen25-14b-gguf</div><div class="sd">14B · only one 100% clean in both conditions</div></a><a class="self" href="https://huggingface.co/Mikrodev/stcoder-qwen35-9b-gguf"><div class="sn2">Mikrodev/stcoder-qwen35-9b-gguf</div><div class="sd">9B · reasoning family, highest ChrF when it answers · you are here</div></a><a class="" href="https://huggingface.co/Mikrodev/stcoder-gemma4-12b-gguf"><div class="sn2">Mikrodev/stcoder-gemma4-12b-gguf</div><div class="sd">12B · non-Qwen lineage, weakest fine-tune case</div></a></div> |
|
|
| <div class="fig"> |
| <div class="fc">Table — the four models on identical measurements</div> |
| <div class="ftl">Same 15 requests, realistic prompt, greedy decoding. ChrF is averaged over |
| <i>delivered</i> replies so the denominator is the same for every model.</div> |
| <div class="tw"><table> |
| <thead><tr><th>Model</th><th>Clean</th><th>Delivered</th><th>ChrF</th><th>Composite</th> |
| <th>Out tok</th><th>Per reply</th><th>Recommended build</th></tr></thead> |
| <tbody><tr><td><code>stcoder-qwen25-7b</code><br><span style="color:var(--dim);font-size:11.5px">7B · default pick, fastest per answer</span></td><td class="num">93.3%</td><td class="num">15/15</td><td class="num">38.5</td><td class="num">0.844</td><td class="num">340</td><td class="num">5.8s</td><td class="num">Q8_0 · 7.54 GiB</td></tr><tr><td><code>stcoder-qwen25-14b</code><br><span style="color:var(--dim);font-size:11.5px">14B · only one 100% clean in both conditions</span></td><td class="num">100.0%</td><td class="num">15/15</td><td class="num">34.9</td><td class="num">0.842</td><td class="num">288</td><td class="num">8.6s</td><td class="num">Q8_0 · 14.62 GiB</td></tr><tr><td><code>stcoder-qwen35-9b</code> (this model)<br><span style="color:var(--dim);font-size:11.5px">9B · reasoning family, highest ChrF when it answers</span></td><td class="num ft">93.3%</td><td class="num">14/15</td><td class="num">40.9</td><td class="num">0.845</td><td class="num">1227</td><td class="num">17.2s</td><td class="num">Q8_0 · 9.11 GiB</td></tr><tr><td><code>stcoder-gemma4-12b</code><br><span style="color:var(--dim);font-size:11.5px">12B · non-Qwen lineage, weakest fine-tune case</span></td><td class="num">86.7%</td><td class="num">15/15</td><td class="num">35.9</td><td class="num">0.83</td><td class="num">401</td><td class="num">8.5s</td><td class="num">Q8_0 · 11.80 GiB</td></tr></tbody> |
| </table></div> |
| <p class="note"><b>How to read this.</b> The 14B has the highest clean rate (100%). The 9B has |
| the highest ChrF among delivered replies (40.9) but delivered on only 14 of 15, so at full |
| delivery the 7B leads (38.5). Composites for the 9B, 7B and 14B (0.845 / 0.844 / 0.842) are |
| within noise of each other. Sizes are <b>GiB</b> throughout. |
| <b>If you are undecided, take stcoder-qwen25-7b at Q8_0.</b></p> |
| </div> |
|
|
| <div class="card warn"><h3>Read this before you start</h3> |
| <ul> |
| <li><b>Chat only — no tool calling.</b> You describe what you need and the model writes |
| Structured Text. Tool calling was trained and evaluated for this line: the Qwen3.5-9B family |
| did learn it (tool-call composite 0.816 at Q8_0, falling to 0.642 at Q4_K_M) while the |
| code-focused families scored 0.02–0.06 and the Gemma family exactly 0.00. Because it was |
| not usable across the line, <b>tool calling was dropped as a product decision</b> and these |
| builds are published and supported as chat models. Do not build an agent on them.</li> |
| <li><b>Always compile before deployment.</b> The model produces dialect-correct code, which is |
| not the same as correct control logic. Compile it in LogicStudio and review the logic.</li> |
| <li><b>It is a small local model.</b> Expect to iterate on a prompt, not to receive a finished |
| project.</li> |
| </ul> |
| </div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">3</span><h2>Set-up, step by step</h2></div> |
| <ol class="steps"><li><b>Install Ollama</b><p>Any recent release. Verify it responds before continuing.</p><pre>ollama --version</pre></li><li><b>Download one GGUF file</b><p>Q6_K is the recommendation for 12 GB and up; Q4_K_M for 8 GB; Q8_0 is the quant the published study numbers were measured on. You need one file only. The "-tc" in the filename is a build-pipeline tag, not a tool-calling build - these are chat-only.</p><pre>hf download Mikrodev/stcoder-qwen35-9b-gguf qwen35_9b-tc.q6_k.gguf --local-dir .</pre></li><li><b>Write a Modelfile next to the GGUF</b><p>Context window matters more for this model than for its siblings because it reasons before it answers. 8192 is the budget used in the study; 16384 removes the empty-reply risk. Greedy decoding (temperature 0) is what was measured.</p><pre>cat > Modelfile <<'EOF' |
| FROM ./qwen35_9b-tc.q6_k.gguf |
| PARAMETER num_ctx 8192 |
| # the published numbers were measured with num_ctx 16384 / num_predict 8192 |
| # this family reasons before answering and the thinking is drawn from the same budget, so do not set num_predict lower |
| PARAMETER num_predict 8192 |
| PARAMETER temperature 0 |
| PARAMETER top_p 1 |
| SYSTEM """You are a PLC programming assistant for the Mikrodev LogicStudio IDE. You write IEC 61131-3 Structured Text in the Mikrodev dialect: no POU wrappers, no TIME literals (use INT milliseconds), positional function-block calls only, no OOP, no pointers, no global or retained variables, no STRING types, no hex literals, no cast or selection functions. Declare variables in VAR / VAR_INPUT / VAR_OUTPUT / VAR CONSTANT blocks and use (* *) or // comments.""" |
| EOF</pre></li><li><b>Create the model in Ollama</b><p>Takes a few seconds; the GGUF is copied into the Ollama store.</p><pre>ollama create stcoder-qwen35-9b:q6 -f Modelfile</pre></li><li><b>Ask it for something small</b><p>The first reply includes a hidden reasoning pass, so expect a pause of several seconds before code appears. That is normal for this model and is why it averaged 17.2 s per answer in the study.</p><pre>ollama run stcoder-qwen35-9b:q6 "Chiller circulation pump has to keep running 45 seconds after the compressor stops. E-stop drops both at once. Give me the ST."</pre></li><li><b>Reproduce the published numbers, if you want to</b><p>The 240-generation head-to-head used these exact tags. The base arms ran at the Ollama library default, whose quantisation was not recorded in the run data.</p><pre># fine-tunes: stcoder_qwen35-9b:q8 stcoder_qwen25-7b:q8 stcoder_qwen25-14b:q6 stcoder_gemma4-12b:q6 |
| # bases: qwen3.5:9b qwen2.5-coder:7b qwen2.5-coder:14b gemma4:12b</pre></li><li><b>Paste into LogicStudio and compile</b><p>Always compile before deployment. The scoring behind this release checks dialect compliance, not whether the logic is correct - and this model's replies also need a check for doubled comment delimiters, which the scorer does not catch.</p></li></ol> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">4</span><h2>What to ask it first</h2></div> |
| <p class="lead">These are ordinary single-purpose requests, the kind this model handles well. |
| Type them as they are.</p> |
| <div class="pgrid"><div class="pc"><div class="q">Car-wash sump pump. Start pushbutton on DI1, NC stop on DI2, pump overload aux on DI3, contactor on RO1. Write me a start/stop seal-in. The start button is bouncy so it has to be held half a second before the pump picks up, and an overload trip has to latch the pump out until someone resets it.</div><div class="e">A body-only block with VAR_INPUT / VAR_OUTPUT, a TON called positionally as tStartHold(bStartBtn, START_HOLD_MS) with START_HOLD_MS : INT := 500 in a VAR CONSTANT block, an RS latch for the contactor and a separate lockout flag.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Seal-in logic was one of the model's stronger categories in the study, and it reaches for RS and R_TRIG without being told to. Check the reply for doubled comment delimiters - the study's seal-in row is one of the two minimal-prompt replies that had them.</div></div><div class="pc"><div class="q">Chiller circulation pump has to keep running 45 seconds after the compressor stops so the loop clears. E-stop cuts compressor and pump straight away. Give me a purge-in-progress flag for the HMI too.</div><div class="e">A timer with PURGE_TIME_MS : INT := 45000, the pump driven from bCompressorRun OR the timer output, and an E-stop override at the end that forces both outputs off.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Off-delay behaviour scored clean in the study, but check the retrigger condition yourself: on the equivalent study row the model latched on the compressor's falling edge rather than using a TOF-style pattern.</div></div><div class="pc"><div class="q">Ultrasonic flow meter on a 4-20 mA input. Raw word is an integer, 0 counts = 0 m3/h and 32767 counts = 250 m3/h. Scale it, clamp the result to 0-250, and raise a wire-break flag below 1000 counts. There is an enable bit; when it is off the output reads zero.</div><div class="e">INT counts into REAL maths with named constants, no INT_TO_REAL call, no MIN/MAX, and the clamp written out as IF/ELSIF.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Scaling was one of the model's cleanest categories (the study's scaling row scored ChrF 42.6 with zero violations). The base model reached for a cast function and MIN/MAX on that row; the fine-tune did not.</div></div><div class="pc"><div class="q">Operators type a brine chiller setpoint on the HMI. Pull anything below -8.0 or above 2.0 degC back to the nearest limit and raise an out-of-range warning. The setpoint going to the controller must not move more than 0.5 degC per scan, so ramp it.</div><div class="e">MIN_TEMP / MAX_TEMP / RAMP_STEP as VAR CONSTANT REALs, a clamp, and a three-branch ramp toward the clamped target.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Clamp-and-ramp was the model's highest-scoring study row (ChrF 51.2). Read the acknowledge handling before you keep it - the warning latch was the weak part of that answer.</div></div><div class="pc"><div class="q">A photo eye pulses once per carton leaving the label applicator. I need a lifetime total for maintenance records and a shift total the supervisor clears with bShiftReset. Count on the rising edge.</div><div class="e">R_TRIG on the photo eye, DINT for the lifetime total, INT for the shift total, and a reset that only zeroes the shift counter.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Expect compact code and no mention of persistence: RETAIN is forbidden in this dialect, so the model will not solve power-cycle survival for you and will not say so either. The study's counter row was among its lower ChrF results (33.4).</div></div><div class="pc"><div class="q">We poll a 16-bit soft-starter status word over Modbus. Bit 0 ready, bit 2 ramping, bit 4 fault, bit 6 bypassed, bits 12-15 are a 4-bit trip code. Break it out into booleans, give me the trip code as a plain integer, and one starter-healthy flag. If the poll is invalid do not trust any of it.</div><div class="e">Decimal mask constants rather than 16#0001, bitwise AND comparisons against 0, a shift for the trip nibble, and an ELSE branch that forces safe values when the poll is invalid.</div><div class="n">Fresh scenario, not one of the 15 study prompts. Bit-masking was one of the model's best categories (the study's status-word row scored ChrF 49.8, zero violations), but check the mask values and shift amount against your drive manual - on that row the nibble mask it chose did not match the bit range asked for.</div></div><div class="pc"><div class="q">AGV crossing strobe. While the crossing-active flag is set the strobe should flash continuously, lit for 300 ms then dark for 1.2 seconds, over and over. When the flag clears the strobe stays off. No hardware flasher relay in this panel.</div><div class="e">Two timers or one timer plus a phase flag, ON_MS and OFF_MS as INT constants, and an output that toggles - not an output tied straight to a single timer's Q.</div><div class="n">Fresh scenario, not one of the 15 study prompts, and the one to try if you want to see the model's main failure mode. On the study's blink row it declared FLASH_OFF_MS : INT := 1500, never used it, and drove the lamp from a single TON output, so the lamp would latch on instead of blinking - R24 100.0, logic wrong. Verify the off phase exists before you keep the block.</div></div><div class="pc"><div class="q">In the block you just wrote, why did you use an RS latch instead of a plain IF assignment, and what happens if start and stop are true in the same scan?</div><div class="e">A short prose explanation of reset dominance and the scan-order consequence, with no code unless you ask for it.</div><div class="n">Fresh scenario, not one of the 15 study prompts. This model keeps a reasoning pass on every turn and handles follow-ups in the same session well. Its study code replies carried no surrounding essay (the prose-opening flag fired on 0% of fine-tune replies), so ask for explanation explicitly when you want it.</div></div></div> |
| |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">5</span><h2>What fine-tuning changed, measured</h2></div> |
| <p class="lead">The same 15 requests were sent to the stock base model |
| (<code>qwen3.5:9b</code>) and to this fine-tune, twice: once with a short realistic |
| system prompt that lists <b>no dialect rules</b>, and once with the entire rule book pasted in. |
| Greedy decoding, fixed seed, identical prompts.</p> |
|
|
| <div class="fig"> |
| <div class="fc">Figure 1 — dialect-clean replies out of 15</div> |
| <div class="ftl">Clean means: code was delivered and it broke no dialect rule</div> |
| <div class="legend"><span><i class="sw b"></i>base</span><span><i class="sw f"></i>this fine-tune</span></div> |
| <div class="frow"><div class="flab">No rules in prompt<small>realistic use</small></div> |
| <div class="fbars"><div class="bar base"><i style="width:6.7%"></i><b>6.7%</b></div><div class="bar ft"><i style="width:93.3%"></i><b>93.3%</b></div></div></div> |
| <div class="frow"><div class="flab">Full rule book in prompt<small>charitable to base</small></div> |
| <div class="fbars"><div class="bar base2"><i style="width:26.7%"></i><b>26.7%</b></div><div class="bar ft2"><i style="width:100.0%"></i><b>100.0%</b></div></div></div> |
| </div> |
|
|
| <div class="fig"> |
| <div class="fc">Figure 2 — per-prompt outcome, no rules in prompt</div> |
| <div class="ftl">One tick per test prompt, same order in both rows</div> |
| <div class="legend"><span><i class="sw f"></i>clean</span><span><i class="sw b"></i>rule violation</span> |
| <span><i class="sw r"></i>no code delivered</span></div> |
| <div class="frow"><div class="flab">Base</div><div class="strip"><i class="vi"></i><i class="vi"></i><i class="nc"></i><i class="vi"></i><i class="vi"></i><i class="ok"></i><i class="nc"></i><i class="nc"></i><i class="vi"></i><i class="vi"></i><i class="vi"></i><i class="vi"></i><i class="vi"></i><i class="vi"></i><i class="vi"></i></div></div> |
| <div class="frow"><div class="flab">Fine-tune</div><div class="strip"><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="nc"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i><i class="ok"></i></div></div> |
| </div> |
|
|
| <div class="tw"><table> |
| <thead><tr><th>Measure</th><th>Base</th><th>This fine-tune</th><th>Base + full rules</th><th>Fine-tune + full rules</th></tr></thead> |
| <tbody> |
| <tr><td>Dialect-clean replies</td><td class="num bs">6.7%</td><td class="num ft">93.3%</td><td class="num">26.7%</td><td class="num">100.0%</td></tr> |
| <tr><td>Delivered code at all</td><td class="num">80.0%</td><td class="num">93.3%</td><td class="num">80.0%</td><td class="num">100.0%</td></tr> |
| <tr><td>R24 rule compliance</td><td class="num">93.02</td><td class="num">100.0</td><td class="num">96.57</td><td class="num">100.0</td></tr> |
| <tr><td>ChrF vs reference code</td><td class="num">28.9</td><td class="num">40.9</td><td class="num">34.1</td><td class="num">37.6</td></tr> |
| <tr><td>Output tokens (mean · median)</td><td class="num">6345 · 6952</td><td class="num">1227 · 793</td><td class="num">2902 · 1521</td><td class="num">554 · 488</td></tr> |
| <tr><td>Time per reply</td><td class="num">54.5s</td><td class="num">17.2s</td><td class="num">26.7s</td><td class="num">9.0s</td></tr> |
| <tr><td>Throughput</td><td class="num">123.2 t/s</td><td class="num">86.3 t/s</td><td class="num">120.5 t/s</td><td class="num">84.0 t/s</td></tr> |
| <tr><td>Opened with prose, not code</td><td class="num">80.0%</td><td class="num">0.0%</td><td class="num">66.7%</td><td class="num">0.0%</td></tr> |
| </tbody> |
| </table></div> |
| <p class="note"><b>On throughput:</b> the fine-tune is served here at higher precision than the |
| stock base model (Ollama's default build is more heavily compressed), so it generates more slowly |
| per token. Quantisation was not matched in this comparison — see the limitations.</p> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">6</span><h2>Which dialect rules the base model breaks</h2></div> |
| <p class="lead">Counted over replies that actually contained code. The upper bar is the realistic |
| condition; the lower, lighter bar is with the full rule book in the system prompt — the |
| difference shows which habits an instruction can fix and which it cannot.</p> |
| <div class="fig"> |
| <div class="fc">Figure 3 — base-model rule violations, qwen3.5:9b — |
| all 15 rules it broke</div> |
| <div class="legend" style="margin-bottom:10px"><span style="color:var(--dim)">bar length = share |
| of delivered replies that broke the rule, full width = 100%</span></div> |
| <div class="frow"><div class="flab"><code>no_VAR_GLOBAL</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:41.7%"></i><b>5/12</b></div><div class="bar base2"><i style="width:16.7%"></i><b>2/12</b></div></div></div><div class="frow"><div class="flab"><code>no_named_param</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:16.7%"></i><b>2/12</b></div><div class="bar base2"><i style="width:33.3%"></i><b>4/12</b></div></div></div><div class="frow"><div class="flab"><code>no_pou_fb</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:33.3%"></i><b>4/12</b></div><div class="bar base2"><i style="width:8.3%"></i><b>1/12</b></div></div></div><div class="frow"><div class="flab"><code>no_pou_program</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:33.3%"></i><b>4/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_single_quote</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:16.7%"></i><b>2/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_c_comment</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:16.7%"></i><b>2/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_lowercase_kw</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:8.3%"></i><b>1/12</b></div></div></div><div class="frow"><div class="flab"><code>no_cast_function</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:8.3%"></i><b>1/12</b></div></div></div><div class="frow"><div class="flab"><code>no_double_quote</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:16.7%"></i><b>2/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_var_retain</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_select_fn</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_TIME_literal</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div><div class="frow"><div class="flab"><code>no_direct_address</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:0.7%"></i><b>0/12</b></div><div class="bar base2"><i style="width:8.3%"></i><b>1/12</b></div></div></div><div class="frow"><div class="flab"><code>no_based_literal</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:0.7%"></i><b>0/12</b></div><div class="bar base2"><i style="width:8.3%"></i><b>1/12</b></div></div></div><div class="frow"><div class="flab"><code>no_STRING_type</code><small>minimal · full rules</small></div><div class="fbars"><div class="bar base"><i style="width:8.3%"></i><b>1/12</b></div><div class="bar base2"><i style="width:0.7%"></i><b>0/12</b></div></div></div> |
| </div> |
| <p class="note">For comparison, this fine-tune's violations across its 14 delivered |
| replies in the same condition: none.</p> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">7</span><h2>The same request, side by side</h2></div> |
| <p class="lead">Highlighted tokens are the ones that decide whether the code is accepted by the |
| IDE. Both columns are verbatim model output.</p> |
| |
| <div style="margin-top:26px"> |
| <h3>Three packaging machines in one array (minimal prompt): R24 86.76 to 100.0, 5671 tokens to 876</h3> |
| <p class="pmeta" style="margin-bottom:8px">machine_data_grouping · data_memory |
| · no dialect rules in the system prompt</p> |
| <p class="note" style="margin-bottom:12px"><b>The request:</b> I have three identical packaging machines in one project. For each machine I track a temperature setpoint, the actual temperature, the belt speed, a cycle counter and a state code. I would like each machine's values kept together so I can run the same monitoring logic over all three in a loop instead of copy-pasting the same block three times. Logic per machine: if it is in the running state and t</p> |
| <div class="diff"> |
| <div class="panel base"><div class="ph"><span class="tag base">Base · qwen3.5:9b</span> |
| <span class="badge warn">R24 86.76 · 5671 tok</span></div> |
| <pre class="plain">Here is the Structured Text solution designed for Mikrodev LogicStudio (IEC 61131-3). |
|
|
| This approach uses a **Type Definition** to encapsulate machine data, an **Array of Structures** to handle all three machines simultaneously in loops, and global variables to manage state persistence across scan cycles. |
|
|
| ```st |
| (* |
| Project: Packaging Machine Monitoring System |
| Description: Monitors 3 identical packaging machines using arrays for scalability. |
| Handles Temperature deviation, Belt Speed, Cycle Counting, and EStop logic. |
| *) |
|
|
| TYPE T_MachineData : |
| (* Configuration *) |
| SetpointTemp : REAL; // Target temperature (e.g., in Celsius) |
| ActualTemp : REAL; // Current sensor reading |
| |
| (* Process Parameters *) |
| BeltSpeed : REAL; // Speed of the belt (e.g., m/s or RPM) |
| |
| (* Status & Control *) |
| IsRunning : BOOL; // True if machine is in active running state |
| CycleCounter : INT := 0; // Counts completed cycles |
| |
| (* Faulting State *) |
| MachineFaulted : BOOL; // Indicates current fault status for this specific machine |
| END_TYPE |
|
|
| (* Global Variables - Persist across scan cycles *) |
| <span class="bad">VAR_GLOBAL</span> |
| Machines : ARRAY[1..3] OF T_MachineData; |
| PrevCycleDone : ARRAY[1..3] OF BOOL := (FALSE, FALSE, FALSE); // For rising edge detection |
| |
| (* External Inputs/Outputs *) |
| EStopInput : BOOL; // Emergency Stop Input from hardware/controller |
| TotalFaultCount </pre> |
| <div class="pv"><code class="v">no_VAR_GLOBAL</code><code class="v">no_double_quote</code><code class="v">no_lowercase_kw</code><code class="v">no_single_quote</code></div></div> |
| <div class="panel ft"><div class="ph"><span class="tag ft">STCoder fine-tune</span> |
| <span class="badge ok">R24 100.0 · 876 tok</span></div> |
| <pre class="plain">```iecst |
| (* Machine monitoring for 3 identical packaging machines *) |
| (* Each machine has the same structure and logic applied via array *) |
|
|
| TYPE MACHINE : |
| nState : INT := 0; (* 0=STOPPED, 1=RUNNING, 2=FAULTED *) |
| rTempSetpoint : REAL; |
| rTempActual : REAL; |
| rBeltSpeed : REAL; |
| nCycleCount : DINT; |
| bEStop : BOOL; |
| bFaulted : BOOL; |
| END_TYPE |
|
|
| VAR_INPUT |
| arrTempSetpoint : ARRAY[0..2] OF REAL; |
| arrTempActual : ARRAY[0..2] OF REAL; |
| arrBeltSpeed : ARRAY[0..2] OF REAL; |
| arrCycleDone : ARRAY[0..2] OF BOOL; |
| bEStop : BOOL; |
| END_VAR |
|
|
| VAR_OUTPUT |
| arrMachine : ARRAY[0..2] OF MACHINE; |
| nFaultedCount : INT; |
| END_VAR |
|
|
| VAR |
| i : INT; |
| rCycleDoneEdge : R_TRIG; |
| END_VAR |
|
|
| (* Initialize machines *) |
| FOR i := 0 TO 2 DO |
| arrMachine[i].nState := 0; |
| arrMachine[i].nCycleCount := 0; |
| arrMachine[i].bFaulted := FALSE; |
| END_FOR; |
|
|
| (* Process each machine with identical logic *) |
| FOR i := 0 TO 2 DO |
| arrMachine[i].rTempSetpoint := arrTempSetpoint[i]; |
| arrMachine[i].rTempActual := arrTempActual[i]; |
| arrMachine[i].rBeltSpeed := arrBeltSpeed[i]; |
| |
| (* Temperature deviation fault *) |
| IF arrMachine[i].nState = 1 THEN |
| IF (arrMachine[i].rTempActual - arrMachine[i].rTempSetpoint) > 5.0 |
| OR (arrMachine[i].rTempSetpoint - arrMachine[i].rTempActual) > 5.0 THEN |
| arrMachine[i].bFaulted := TRUE; |
| arrMachine[i].nState := 2; |
| END_IF; |
| END_IF; |
| |
| (* Belt speed fault *) |
| IF arrMachine[i].nState = 1 AND arrMachine[i].rBeltSpeed < 10.0 THEN |
| arrMachine[i].bFaulted := TR</pre> |
| <div class="pv"><code class="c">no violations</code></div></div> |
| </div> |
| <p class="note">The base answer opens with two paragraphs of prose, then builds exactly the structure the Mikrodev runtime rejects. It declares a user type with TYPE T_MachineData :, parks the machine array in a VAR_GLOBAL block so it "persists across scan cycles", annotates the fields with C-style // Current sensor reading comments, and wraps the logic in a BEGIN block - a Pascal-ism that is not ST at all. R24 put it at 86.76 after 5671 tokens, flagging no_VAR_GLOBAL, no_double_quote, no_single_quote and no_lowercase_kw; worth knowing that the lowercase-keyword hits landed on English words inside the reply's comment prose rather than on executable statements, while the VAR_GLOBAL, user-type and BEGIN problems are real. The fine-tune spent 876 tokens and scored 100.0. It exposes the per-machine values as plain 1-D arrays in VAR_INPUT, keeps the state code as an INT with an inline (* 0=STOPPED, 1=RUNNING, 2=FAULTED *) comment instead of an enum, and detects the cycle pulse with a declared R_TRIG. One honest caveat, visible in the marks: the fine-tune still emitted a TYPE MACHINE : ... END_TYPE block of its own for the output array. R24 did not flag it and the row counted as clean, but the dialect has no user-defined types, so that declaration has to go before the block will compile.</p> |
| <p class="note" style="color:var(--dim);font-size:12.3px">Red spans in the base panel are the |
| exact substrings the rule checker matched. The fine-tune panel is shown unmarked because it |
| broke no rule on this prompt — read it against the base, line for line.</p> |
| </div> |
| <div style="margin-top:26px"> |
| <h3>Reusable pump block (minimal prompt): R24 88.24 to 100.0, and the base hit the 8192-token wall</h3> |
| <p class="pmeta" style="margin-bottom:8px">reusable_pump_block · program_structure |
| · no dialect rules in the system prompt</p> |
| <p class="note" style="margin-bottom:12px"><b>The request:</b> We have three identical booster pumps on a skid and I am tired of copy-pasting the same logic three times. Give me one reusable pump control block that I can instantiate once per pump. Per instance: run request, motor thermal overload contact, dry-run (low suction) switch, fault reset. Outputs: motor run command, fault flag, and a fault code for the HMI so the operator can see what tripped. Behavi</p> |
| <div class="diff"> |
| <div class="panel base"><div class="ph"><span class="tag base">Base · qwen3.5:9b</span> |
| <span class="badge warn">R24 88.24 · 8192 tok</span></div> |
| <pre class="plain">Here is a reusable **Pump Control Function Block** written in Structured Text (ST), compatible with the Mikrodev LogicStudio IDE environment following IEC 61131-3 standards. |
|
|
| ### PumpControl FB Definition |
|
|
| This block encapsulates all logic, timers, and fault handling into one instance per pump. It handles start delays, dry-run protection windows, overload latching, restart lockouts, and HMI reporting. |
|
|
| ```st |
| <span class="bad">FUNCTION_BLOCK</span> PumpControl : |
| VAR_INPUT |
| RunRequest : BOOL; // Operator request to run the pump (e.g., from level switch) |
| OverloadIn : BOOL; // TRUE = Thermal Overload Relay Tripped / Fault Active |
| DryRunSwitch : BOOL; // TRUE = Low Suction Pressure Detected (Dry-run condition) |
| ResetButton : BOOL; // Momentary or Level signal to clear fault latch and reset timers |
| END_VAR |
|
|
| VAR_OUTPUT |
| MotorCmd : BOOL; // Output command for the motor starter/drive |
| FaultFlag : BOOL; // General alarm flag for HMI (TRUE if any active fault) |
| FaultCode : INT; // Specific code: 0=OK, 1=Overload, 2=Dry Run |
| END_VAR |
|
|
| VAR |
| StartDelayTimer : TON; // Handles the 2s start delay after request |
| SpinUpTimer : TON; // Waits for motor to stabilize (5s) before checking dry-run |
| CooldownTimer : TON; // Enforces 10s restart lockout after pump stops |
| |
| MotorRunning : BOOL := FALSE; // Internal flag to track if we are currently running |
| END_VAR |
|
|
| // --- Logic Implementation --- |
|
|
| // 1. Reset Fault L</pre> |
| <div class="pv"><code class="v">no_TIME_literal</code><code class="v">no_named_param</code><code class="v">no_pou_fb</code></div></div> |
| <div class="panel ft"><div class="ph"><span class="tag ft">STCoder fine-tune</span> |
| <span class="badge ok">R24 100.0 · 1139 tok</span></div> |
| <pre class="plain">```iecst |
| (* (* |
| Reusable booster pump control block. |
| Instantiate once per pump to avoid copy-paste. |
| *) *) |
| VAR_INPUT |
| bRunReq : BOOL; (* (* Operator run request *) *) |
| bOverload : BOOL; (* (* Motor thermal overload contact *) *) |
| bDryRun : BOOL; (* (* Low suction / dry-run switch *) *) |
| bFaultReset : BOOL; (* (* Fault reset button *) *) |
| bStartPulse : BOOL; (* (* Start pulse for 2s delay *) *) |
| bStopPulse : BOOL; (* (* Stop pulse for 10s off time *) *) |
| END_VAR |
| VAR_OUTPUT |
| bMotorRun : BOOL; (* (* Motor run command *) *) |
| bFault : BOOL; (* (* Latched fault flag *) *) |
| nFaultCode : INT; (* (* Fault code for HMI *) *) |
| END_VAR |
| VAR |
| tStartDelay : TON; (* (* 2s start delay *) *) |
| tDryRunDelay : TON; (* (* 5s dry-run evaluation delay *) *) |
| tOffDelay : TON; (* (* 10s minimum off time *) *) |
| bStartDone : BOOL; (* (* Start delay elapsed *) *) |
| bDryRunEval : BOOL; (* (* Dry-run evaluation ready *) *) |
| bFaultLatched : BOOL; (* (* Latched fault *) *) |
| END_VAR |
| VAR CONSTANT |
| START_DELAY_MS : INT := 2000; |
| DRYRUN_DELAY_MS: INT := 5000; |
| OFF_DELAY_MS : INT := 10000; |
| END_VAR |
|
|
| (* (* Start delay *) *) |
| tStartDelay(bStartPulse, START_DELAY_MS); |
| bStartDone := tStartDelay.Q; |
|
|
| (* (* Dry-run evaluation delay *) *) |
| tDryRunDelay(bMotorRun, DRYRUN_DELAY_MS); |
| bDryRunEval := tDryRunDelay.Q; |
|
|
| (* (* Off delay *) *) |
| tOffDelay(NOT bMotorRun, OFF_DELAY_MS); |
|
|
| (* (* Fault logic *) </pre> |
| <div class="pv"><code class="c">no violations</code></div></div> |
| </div> |
| <p class="note">This is the clearest case of the base fighting the dialect. It opens with FUNCTION_BLOCK PumpControl : - the IDE owns that shell, code is body-only - declares its timers with C-style // Handles the 2s start delay after request comments, and then, further down the reply, calls every one of them the CODESYS way: the scorer recorded StartDelayTimer(IN:=FALSE, PT := T#2S), CooldownTimer(IN:=TRUE) and SpinUpTimer(IN:=TRUE) under no_named_param and no_TIME_literal, where Mikrodev wants positional arguments and INT milliseconds. It also tried to assign to its own input with OverloadIn := FALSE;, argued with itself in a comment about whether that was legal, and finally ran out at the full 8192-token budget mid-thought. Score 88.24. The fine-tune produced the whole block in 1139 tokens at 100.0: START_DELAY_MS : INT := 2000 in a VAR CONSTANT block, tStartDelay(bStartPulse, START_DELAY_MS) called positionally, tOffDelay(NOT bMotorRun, OFF_DELAY_MS) for the restart lockout, and the three instances shown separately as asked. The flaw to fix by hand is cosmetic but real, and it is in the marks: this reply doubled its comment delimiters throughout, (* (* 2s start delay *) *), which is not valid ST and which the scorer does not catch.</p> |
| <p class="note" style="color:var(--dim);font-size:12.3px">Red spans in the base panel are the |
| exact substrings the rule checker matched. The fine-tune panel is shown unmarked because it |
| broke no rule on this prompt — read it against the base, line for line.</p> |
| </div> |
| <div style="margin-top:26px"> |
| <h3>Drilling station sequence with the full rule book in the prompt: R24 95.59 to 100.0</h3> |
| <p class="pmeta" style="margin-bottom:8px">machine_cycle_full_program · program_structure |
| · entire rule book in the system prompt</p> |
| <p class="note" style="margin-bottom:12px"><b>The request:</b> Can you write the complete program for our drilling station cycle? One part per cycle: when the guard is closed, a part is present and there are no alarms, pressing Start closes the clamp, waits 1.5 s for clamp pressure to build, then the drill motor starts and the head feeds down until the down limit switch, dwells 3 s at depth, retracts until the up limit, opens the clamp after 0.8 s, and the st</p> |
| <div class="diff"> |
| <div class="panel base"><div class="ph"><span class="tag base">Base · qwen3.5:9b</span> |
| <span class="badge warn">R24 95.59 · 3392 tok</span></div> |
| <pre class="plain">(* Drilling Station Cycle Logic *) |
| VAR |
| (* Inputs *) |
| bGuardClosed : BOOL; |
| bPartPresent : BOOL; |
| btnStart : BOOL; |
| swDownLimit : BOOL; |
| swUpLimit : BOOL; |
| btnEStop : BOOL; |
| alrDrillOverload : BOOL; |
|
|
| (* Outputs *) |
| clmClampClosed : BOOL; |
| mtrFeedMotor : BOOL; |
| mtrRetractMotor : BOOL; |
| vlvOpenClamp : BOOL; |
| |
| (* Timers - clamp pressure build, dwell at depth, open clamp delay *) |
| tClampPressure : TON; |
| tDwellAtDepth : TON; |
| tOpenClampDelay : TON; |
|
|
| (* State machine for step sequencing *) |
| eState : (IDLE, CLAMPING, DRILLING, DWELL, RETRACTING, OPEN_CLAMP); |
| |
| (* Fault state flag *) |
| bFaultActive : BOOL; |
| END_VAR |
|
|
| (* Safety check: E-stop or overload drops all outputs and sets fault *) |
| IF btnEStop OR alrDrillOverload THEN |
| clmClampClosed := FALSE; |
| mtrFeedMotor := FALSE; |
| mtrRetractMotor := FALSE; |
| vlvOpenClamp := TRUE; (* Open clamp for safety *) |
| bFaultActive := TRUE; |
| END_IF |
|
|
| (* Reset fault when reset button pressed and no active faults *) |
| IF btnReset AND NOT (btnEStop OR alrDrillOverload) THEN |
| bFaultActive := FALSE; |
| clmClampClosed := FALSE; (* Ensure clamp is open during reset *) |
| mtrFeedMotor := FALSE; |
| mtrRetractMotor := FALSE; |
| END_IF |
|
|
| (* Only allow cycle to start if guard closed, part present, no fault, and not already running *) |
| IF NOT bFaultActive AND btnStart THEN |
| CASE eState OF |
| IDLE: |
| (* Start clamp closing sequence *) |
| tClampPressure(IN :</pre> |
| <div class="pv"><code class="v">no_named_param</code></div></div> |
| <div class="panel ft"><div class="ph"><span class="tag ft">STCoder fine-tune</span> |
| <span class="badge ok">R24 100.0 · 809 tok</span></div> |
| <pre class="plain">```iecst |
| (* Drilling station cycle with clamp, drill, feed, dwell, retract, and clamp-open sequence *) |
| (* Safety: E-stop or overload forces fault state; reset button clears fault *) |
|
|
| VAR_INPUT |
| bGuardClosed : BOOL; |
| bPartPresent : BOOL; |
| bStartCycle : BOOL; |
| bEStop : BOOL; |
| bResetBtn : BOOL; |
| bDrillOverload : BOOL; |
| bClampClosed : BOOL; |
| bDownLimit : BOOL; |
| bUpLimit : BOOL; |
| END_VAR |
| VAR_OUTPUT |
| bClampMotor : BOOL; |
| bDrillMotor : BOOL; |
| bFeedMotor : BOOL; |
| bClampOpen : BOOL; |
| bStationFault : BOOL; |
| nPartsDone : INT; |
| END_VAR |
| VAR |
| nState : INT := 0; |
| tClamp : TON; |
| tDwell : TON; |
| bFault : BOOL; |
| END_VAR |
| VAR CONSTANT |
| CLAMP_BUILD_MS : INT := 1500; |
| DWELL_MS : INT := 3000; |
| CLAMP_OPEN_MS : INT := 800; |
| END_VAR |
|
|
| (* Safety interlocks *) |
| IF bEStop OR bDrillOverload THEN |
| bStationFault := TRUE; |
| bClampMotor := FALSE; |
| bDrillMotor := FALSE; |
| bFeedMotor := FALSE; |
| bClampOpen := FALSE; |
| nState := 0; |
| RETURN; |
| END_IF; |
|
|
| CASE nState OF |
| 0: (* Idle *) |
| bClampMotor := FALSE; |
| bDrillMotor := FALSE; |
| bFeedMotor := FALSE; |
| bClampOpen := FALSE; |
| IF bGuardClosed AND bPartPresent AND bStartCycle THEN |
| nState := 1; |
| END_IF; |
|
|
| 1: (* Clamp close *) |
| bClampMotor := TRUE; |
| tClamp(TRUE, CLAMP_BUILD_MS); |
| IF tClamp.Q THEN |
| bClampMotor := FALSE; |
| nState := 2; |
| END_IF; |
|
|
| 2: (* Dri</pre> |
| <div class="pv"><code class="c">no violations</code></div></div> |
| </div> |
| <p class="note">Here the base model had the entire rule book in its system prompt and still could not stop naming timer parameters. It dropped the POU wrapper and avoided TIME literals - prompting bought that much - but wrote tClampPressure(IN := TRUE); and five more calls in the same style, six flagged named-parameter tokens in total, which is the single rule it broke: no_named_param, 95.59, 3392 tokens. It never gave those timers a period either, so the 1.5 s, 3 s and 0.8 s values from the request are simply absent, and it referenced btnReset in the fault-reset branch without ever declaring it. It also typed its step variable as an anonymous enumeration, eState : (IDLE, CLAMPING, DRILLING, DWELL, RETRACTING, OPEN_CLAMP);, which the scorer did not flag but the dialect does not support. The fine-tune needed 809 tokens for 100.0: CLAMP_BUILD_MS : INT := 1500 and DWELL_MS : INT := 3000 as constants, tClamp(TRUE, CLAMP_BUILD_MS) and tDwell(TRUE, DWELL_MS) positional, a numeric CASE nState OF step chain, and an E-stop branch at the top that drops every output and RETURNs before the sequencer can run. The step logic still deserves review - the dwell is nested inside the down-limit branch and the retract step drives the feed motor in the same direction as the drill step - but nothing in it violates the dialect.</p> |
| <p class="note" style="color:var(--dim);font-size:12.3px">Red spans in the base panel are the |
| exact substrings the rule checker matched. The fine-tune panel is shown unmarked because it |
| broke no rule on this prompt — read it against the base, line for line.</p> |
| </div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">8</span><h2>Every test prompt and its result</h2></div> |
| <p class="lead">All 15 prompts, realistic condition (no rules in the system prompt). Prompt text |
| is in the appendix below.</p> |
| <div class="tw"><table> |
| <thead><tr><th>Prompt</th><th>Base</th><th>R24</th><th>ChrF</th><th>tok</th> |
| <th>Fine-tune</th><th>R24</th><th>ChrF</th><th>tok</th><th style="text-align:left">Rules the base broke</th></tr></thead> |
| <tbody><tr><td><code>analog_scale_raw_to_bar</code><br><span style="color:var(--dim);font-size:11.5px">analog_math</span></td><td><span class="chip vi">violation</span></td><td class="num bs">91.18</td><td class="num">32.82</td><td class="num">7197</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">42.58</td><td class="num">995</td><td style="text-align:left;font-size:11.6px"><code>no_cast_function</code>, <code>no_pou_fb</code>, <code>no_select_fn</code></td></tr><tr><td><code>setpoint_clamp_and_ramp</code><br><span style="color:var(--dim);font-size:11.5px">analog_math</span></td><td><span class="chip vi">violation</span></td><td class="num bs">97.06</td><td class="num">27.55</td><td class="num">6389</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">51.15</td><td class="num">524</td><td style="text-align:left;font-size:11.6px"><code>no_pou_program</code></td></tr><tr><td><code>heater_power_avg_squared</code><br><span style="color:var(--dim);font-size:11.5px">analog_math</span></td><td><span class="chip nc">no code</span></td><td class="num bs">100.0</td><td class="num">0.0</td><td class="num">8192</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">45.36</td><td class="num">520</td><td style="text-align:left;font-size:11.6px">—</td></tr><tr><td><code>timer_start_delay</code><br><span style="color:var(--dim);font-size:11.5px">timer_time_based</span></td><td><span class="chip vi">violation</span></td><td class="num bs">91.18</td><td class="num">28.21</td><td class="num">8060</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">41.15</td><td class="num">673</td><td style="text-align:left;font-size:11.6px"><code>no_VAR_GLOBAL</code>, <code>no_c_comment</code>, <code>no_pou_program</code></td></tr><tr><td><code>timer_blink_oscillator</code><br><span style="color:var(--dim);font-size:11.5px">timer_time_based</span></td><td><span class="chip vi">violation</span></td><td class="num bs">98.53</td><td class="num">22.08</td><td class="num">1739</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">30.25</td><td class="num">793</td><td style="text-align:left;font-size:11.6px"><code>no_double_quote</code></td></tr><tr><td><code>timer_offdelay_coastdown</code><br><span style="color:var(--dim);font-size:11.5px">timer_time_based</span></td><td><span class="chip ok">clean</span></td><td class="num bs">100.0</td><td class="num">5.65</td><td class="num">8192</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">41.73</td><td class="num">460</td><td style="text-align:left;font-size:11.6px">—</td></tr><tr><td><code>io_terminal_seal_in</code><br><span style="color:var(--dim);font-size:11.5px">io_misc</span></td><td><span class="chip nc">no code</span></td><td class="num bs">100.0</td><td class="num">0.0</td><td class="num">8192</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">46.95</td><td class="num">545</td><td style="text-align:left;font-size:11.6px">—</td></tr><tr><td><code>status_word_bit_mask</code><br><span style="color:var(--dim);font-size:11.5px">io_misc</span></td><td><span class="chip nc">no code</span></td><td class="num bs">100.0</td><td class="num">0.0</td><td class="num">8192</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">49.83</td><td class="num">509</td><td style="text-align:left;font-size:11.6px">—</td></tr><tr><td><code>first_out_fault_annunciator</code><br><span style="color:var(--dim);font-size:11.5px">io_misc</span></td><td><span class="chip vi">violation</span></td><td class="num bs">94.12</td><td class="num">34.8</td><td class="num">6621</td><td><span class="chip nc">no code</span></td><td class="num ft">100.0</td><td class="num">0.0</td><td class="num">8192</td><td style="text-align:left;font-size:11.6px"><code>no_VAR_GLOBAL</code>, <code>no_pou_program</code></td></tr><tr><td><code>machine_cycle_full_program</code><br><span style="color:var(--dim);font-size:11.5px">program_structure</span></td><td><span class="chip vi">violation</span></td><td class="num bs">92.65</td><td class="num">30.93</td><td class="num">5122</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">46.5</td><td class="num">989</td><td style="text-align:left;font-size:11.6px"><code>no_named_param</code>, <code>no_pou_program</code></td></tr><tr><td><code>reusable_pump_block</code><br><span style="color:var(--dim);font-size:11.5px">program_structure</span></td><td><span class="chip vi">violation</span></td><td class="num bs">88.24</td><td class="num">28.56</td><td class="num">8192</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">41.88</td><td class="num">1139</td><td style="text-align:left;font-size:11.6px"><code>no_TIME_literal</code>, <code>no_named_param</code>, <code>no_pou_fb</code></td></tr><tr><td><code>analog_scale_helper</code><br><span style="color:var(--dim);font-size:11.5px">program_structure</span></td><td><span class="chip vi">violation</span></td><td class="num bs">97.06</td><td class="num">30.24</td><td class="num">6952</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">23.58</td><td class="num">907</td><td style="text-align:left;font-size:11.6px"><code>no_pou_fb</code></td></tr><tr><td><code>retain_counter_power_cycle</code><br><span style="color:var(--dim);font-size:11.5px">data_memory</span></td><td><span class="chip vi">violation</span></td><td class="num bs">85.29</td><td class="num">30.49</td><td class="num">1304</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">33.4</td><td class="num">836</td><td style="text-align:left;font-size:11.6px"><code>no_STRING_type</code>, <code>no_VAR_GLOBAL</code>, <code>no_c_comment</code>, <code>no_single_quote</code>, <code>no_var_retain</code></td></tr><tr><td><code>shared_project_value</code><br><span style="color:var(--dim);font-size:11.5px">data_memory</span></td><td><span class="chip vi">violation</span></td><td class="num bs">94.12</td><td class="num">39.82</td><td class="num">5166</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">38.49</td><td class="num">450</td><td style="text-align:left;font-size:11.6px"><code>no_VAR_GLOBAL</code>, <code>no_pou_fb</code></td></tr><tr><td><code>machine_data_grouping</code><br><span style="color:var(--dim);font-size:11.5px">data_memory</span></td><td><span class="chip vi">violation</span></td><td class="num bs">86.76</td><td class="num">35.64</td><td class="num">5671</td><td><span class="chip ok">clean</span></td><td class="num ft">100.0</td><td class="num">40.42</td><td class="num">876</td><td style="text-align:left;font-size:11.6px"><code>no_VAR_GLOBAL</code>, <code>no_double_quote</code>, <code>no_lowercase_kw</code>, <code>no_single_quote</code></td></tr></tbody> |
| </table></div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">9</span><h2>Q4, Q6 and Q8 — what actually differs</h2></div> |
| <p>Quantisation compresses the weights: Q8_0 keeps roughly 8 bits per weight, Q6_K about 6, Q4_K_M about 4. Smaller file, less VRAM, slightly noisier arithmetic. All sizes here are GiB as reported by the build tooling: 9.11 GiB (Q8_0), 7.04 GiB (Q6_K), 5.38 GiB (Q4_K_M). |
|
|
| A note on the filenames: the "-tc" in qwen35_9b-tc.*.gguf is a build-pipeline tag left over from the tool-call training stage. It does not mark a tool-calling build. These files are published and supported as chat-only, as described above. |
|
|
| On the code task the quant spread is small. An earlier internal 2000-prompt evaluation of these same fine-tunes measured code composite 0.836 (Q8_0), 0.832 (Q6_K) and 0.828 (Q4_K_M), with ChrF 47.4 / 46.0 / 44.5 - roughly 0.008 of composite across the whole range, far less than the difference between prompting styles. That evaluation's code holdout overlapped the training distribution by about 84%, so those absolute code scores are inflated and weakly discriminative between models. They are quoted here only to compare quantisations within this one model, and must not be used to rank the four models against each other. |
|
|
| The large Q4 penalty reported earlier for this family was measured on the tool-calling task, where low precision damaged structured argument emission (0.816 at Q8_0 down to 0.642 at Q4_K_M). Tool calling has been dropped from this line as a product decision, so for a chat-only deployment the quantisation choice is a memory decision, not an accuracy decision. |
|
|
| Methodology, stated the same way in all four cards. The R24 and clean figures here are computed over the whole reply. Base replies were prose-wrapped 66.7-100% of the time and fine-tune replies 0-20%, so whole-reply scoring flatters the fine-tunes; scoring only the fenced code moves the pooled base clean rate from 6.7% to 10.0%. For this model's own base arm the two variants agree exactly (6.7% minimal, 26.7% full under both), and the base-to-fine-tune gap is not sensitive to the choice - but the variant is named for the record. |
|
|
| Reproducibility. The head-to-head ran Ollama tags stcoder_qwen35-9b:q8 against base qwen3.5:9b; the siblings were stcoder_qwen25-7b:q8 / qwen2.5-coder:7b, stcoder_qwen25-14b:q6 / qwen2.5-coder:14b, and stcoder_gemma4-12b:q6 / gemma4:12b. The run data records no quantisation at all for the base arms, so "effectively Q4" is an inference from the Ollama library default, not a recorded value. The comparison is therefore not precision-matched.</p> |
| <div class="card"><h3>Evidence base — and its limit</h3> |
| <p>From an earlier 2,000-prompt internal evaluation of these same fine-tunes: |
| <b>code composite Q8 0.836 / Q6 0.832 / Q4 0.828; ChrF 47.4 / 46.0 / 44.5</b>. The head-to-head study on this page used the |
| <b>Q8_0</b> build. File sizes are <b>GiB</b>.</p> |
| <p class="note" style="margin-top:10px"><b>Important caveat.</b> That earlier evaluation's code |
| holdout overlapped the training distribution by roughly 84%, so its absolute code scores are |
| inflated and only weakly separate one model from another. Use those figures for the |
| <i>within-model</i> quantisation comparison, which is what they are cited for here — not |
| to rank the four models against each other. For cross-model ranking use the identical-conditions |
| table in section 2.</p></div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">10</span><h2>Strengths, weaknesses, and what not to use it for</h2></div> |
| <h3>Strengths</h3><ul><li>Zero dialect violations across every delivered reply in the study: R24 100.0 in both the minimal and the full-rule condition, with the violation table empty for 14/14 and 15/15 delivered replies. The same figures scored over the fenced code only are identical. The stock base at the same minimal prompt broke rules in 13 categories, led by VAR_GLOBAL (5 replies), FUNCTION_BLOCK (4) and PROGRAM (4).</li><li>Its code shape, variable naming and comment style land close to the hand-written reference: ChrF 40.9 averaged over its 14 delivered replies at the minimal prompt, which is the highest delivered-only figure in the set (7B 38.5, Gemma-12B 35.9, Coder-14B 34.9). Counting the one missing reply as zero it is 38.2, below the 7B's 38.5 over 15 of 15 - so the delivered-only lead is real but conditional, and the denominator belongs with the number.</li><li>The dialect is in the weights, not in the prompt. With a short realistic system prompt and no rules listed it scored 93.3% clean; the base needed the entire rule book in its system prompt to reach 26.7%.</li><li>Reasoning cost cut by roughly five times against its own base: 1227 output tokens versus 6345 at the minimal prompt (median 793 versus 6952), and 17.2 s versus 54.5 s per answer.</li><li>Timers come out right without being asked: positional TON(condition, MS) calls with INT millisecond constants such as START_DELAY_MS : INT := 2000, where the base emitted PT := T#2S and named parameters like StartDelayTimer(IN:=...) - the two constructs the Mikrodev runtime rejects.</li><li>Returns code, not an essay: the prose-opening flag fired on 0% of fine-tune replies against 80% of base replies at the minimal prompt. Output pastes into the IDE with a fence to strip.</li><li>Uses the standard blocks the dialect allows - RS, R_TRIG, TON - and the body-only VAR_INPUT / VAR_OUTPUT / VAR CONSTANT shape, with no PROGRAM or FUNCTION_BLOCK wrapper in any reply.</li><li>With the full rule book in the prompt it became cheaper and more dependable to run: 15/15 delivered, 100% clean, 554 tokens and 9.0 s per answer against 1227 tokens and 17.2 s at the minimal prompt. One trade-off to know before you adopt that condition: doubled comment delimiters rose from 2 of 15 replies at the minimal prompt to 6 of 15 with the full rule book, so the full condition is more reliable to obtain and slightly more work to clean.</li></ul> |
| <h3>Weaknesses</h3><ul><li>It can still return nothing. One of fifteen fine-tuned replies at the minimal prompt (the first-out fault annunciator) spent the whole 8192-token budget on hidden reasoning and delivered no code, giving 93.3% delivery - the only non-delivery by any fine-tune in the study. The base did this three times out of fifteen. Fine-tuning reduced the failure mode; it did not remove it.</li><li>Every reply carries a hidden reasoning block - 100% of replies in all four arms, base and fine-tune alike. Expect a pause before code appears, and never run this model with a short context.</li><li>"Clean" is not "correct". The flashing-beacon reply scored R24 100.0 and violated nothing, but it declared FLASH_OFF_MS : INT := 1500, never used it, and drove the lamp straight from a single TON output - that lamp will not blink. Its ChrF on that row was the lowest of the timer set (30.25).</li><li>Two further logic defects in otherwise clean replies: the pre-start horn block gated the motor on a one-scan rising-edge flag, so the motor never seals in; and the heater-averaging block wrapped its divide in a FOR loop that does not actually average eight samples.</li><li>Doubled comment delimiters - literal (* (* Motor run command *) *) - appeared in 2 of 15 minimal-prompt replies (io_terminal_seal_in, reusable_pump_block) and in 6 of 15 with the full rule book (timer_start_delay, timer_blink_oscillator, reusable_pump_block, retain_counter_power_cycle, shared_project_value, machine_data_grouping). The R24 rule set does not flag this, but the sequence is not valid ST and has to be cleaned before compiling. It is the model's most common compile-stopper, and it is worse in the condition that otherwise scores best.</li><li>In the array-of-machines answer it emitted a TYPE MACHINE : ... END_TYPE declaration of its own. The scorer did not flag it and the row counted as clean, but the Mikrodev dialect has no user-defined types, so that reply needs rework.</li><li>ChrF is modest in absolute terms - 40.9 over its 14 delivered minimal-prompt replies, 38.2 across all 15 - and one row fell to 23.58 (the reusable scaling helper). The structure is right; naming, decomposition and interface choices often differ from what a plant engineer would have written.</li><li>Slowest of the four per answer at the minimal prompt (17.2 s, 1227 tokens against 5.8 s / 340 tokens for the 7B). Per token it is mid-field at 86.3 tok/s - the slowest per token is the 14B at about 50 tok/s. Its 86.3 tok/s against the base's 123.2 is not a precision-matched comparison: the fine-tune ran at Q8_0 while the base ran at the Ollama library default, whose quantisation the run data does not record. Total wall-clock still favours the fine-tune by about three times.</li><li>At 9.11 GiB the Q8_0 file - the quant all published head-to-head numbers were measured on - does not fit an 8 GB card alongside a usable context.</li></ul> |
| <h3>Not the right tool for</h3><ul><li>Tool calling or function calling. Tool calling was trained and evaluated for this line and then dropped as a product decision; these builds are published and supported as chat-only models. This family is the one that learned it - tool-call composite 0.816 at Q8_0, 0.805 at Q6_K, 0.642 at Q4_K_M - while stcoder-qwen25-7b and stcoder-qwen25-14b reached 0.02-0.06 and stcoder-gemma4-12b 0.00. Because it was not usable across the line it was withdrawn: not published, not supported, and not to be relied on here even though the weights retain some of the ability. If you need an agent that drives IDE tools, do not build it on these releases.</li><li>Deployment without compiling. R24 is reference-free and checks dialect compliance only; three of the clean replies in this study contained real logic defects, and doubled comment delimiters slip past the scorer entirely. Compile in LogicStudio and test before commissioning.</li><li>Any dialect other than Mikrodev LogicStudio. The fine-tune actively suppresses POU wrappers, TIME literals and named function-block parameters that other IEC 61131-3 environments expect.</li><li>Machines where the context window has to be cut short. This is a reasoning model; a small budget is exactly how you get a reply with no code in it.</li><li>Latency-sensitive interactive use. This model answered in 9.0-17.2 s per reply depending on condition; the line default, stcoder-qwen25-7b, answered in 5.8 s at the minimal prompt with 100% delivery.</li><li>Frontier-scale work - whole-project architecture, safety analysis, or reasoning over long documents. This is a 9B model for local hardware.</li><li>Retained or global data. VAR RETAIN, VAR_GLOBAL and VAR_EXTERNAL are forbidden in the dialect, so the model will not solve persistence for you even when the prompt asks for it - that belongs on the IDE side.</li></ul> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">11</span><h2>How the numbers were produced</h2></div> |
| <p>240 controlled generations across four model families, 15 prompts each, run twice per model |
| under two prompting conditions. Greedy decoding (<code>temperature 0</code>, <code>seed 42</code>), |
| 8,192-token budget, local Ollama on an RTX 5080 (16 GB).</p> |
| <ul> |
| <li><b>R24</b> — dialect rule compliance. 33 forbidden patterns, individually weighted; the |
| score is the share of total weight left intact (0–100). Comments, prose and string contents |
| are removed before matching, so discussion never triggers a rule.</li> |
| <li><b>ChrF</b> — character n-gram F-score of the generated code against a hand-written |
| reference solution (sacreBLEU chrF++). Higher means closer to house style. An alternative but |
| correct solution legitimately scores lower, so it is always read together with R24.</li> |
| <li><b>Clean</b> — the reply delivered code <i>and</i> broke no rule. A reply containing no |
| code scores R24 = 100 trivially, because there is nothing left to violate, so delivery is |
| checked first.</li> |
| <li><b>Which R24 variant.</b> The R24 and clean figures on this page are computed over the |
| <b>whole reply</b>. That matters because base replies were prose-wrapped 80.0% of the |
| time for this model: scoring only the fenced code lifts the pooled base clean rate across the |
| study from 6.7% to 10.0%. The base-versus-fine-tune gap is not sensitive to the choice, but the |
| variant is named here so the numbers can be reproduced exactly.</li> |
| <li><b>Test prompts</b> — written as ordinary plant requests. No prompt mentions any dialect |
| rule; a prompt that said “use milliseconds” would hand the answer to the base model. |
| Reference solutions were verified rule-clean before use.</li> |
| </ul> |
| <div class="card warn"><h3>Limitations</h3> |
| <ul> |
| <li><b>Quantisation was not matched.</b> The fine-tune ran at higher precision than the stock |
| base build. This plausibly helps the fine-tune on quality and demonstrably penalises it on |
| raw throughput.</li> |
| <li><b>“Clean” is not “correct”.</b> R24 is reference-free: it certifies |
| that no forbidden construct appears, not that the logic works. Correctness was checked by |
| reading the code, not by compiling it.</li> |
| <li><b>The published builds do not ship this decoding configuration.</b> These numbers were |
| measured greedily (<code>temperature 0</code>, |
| <code>seed 42</code>) at |
| <code>num_ctx 16384</code> / |
| <code>num_predict 8192</code>. The Modelfiles published with |
| the models default to <code>temperature 0.2</code> and <code>num_ctx 8192</code> — the |
| sequence length the models were trained at — because that is the better interactive |
| default. Match the three measured values to reproduce these figures; otherwise expect |
| run-to-run variation.</li> |
| <li><b>Scope.</b> 15 prompts, one dialect, English only, no tool-calling, and the prompt set |
| was deliberately built on the categories where this dialect diverges from standard IEC. It |
| measures dialect fidelity on dialect-sensitive work, not general PLC competence.</li> |
| <li><b>Small sample.</b> 15 prompts per condition per model. Treat single-prompt differences |
| as anecdote and the aggregate as the signal.</li> |
| </ul> |
| </div> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">12</span><h2>In summary</h2></div> |
| <p>stcoder-qwen35-9b removes the dialect problem for this base: R24 100.0 with an empty violation table on every delivered reply, no POU wrappers, no TIME literals, no named function-block parameters, and code that lands close to the hand-written reference (ChrF 40.9 over its 14 delivered minimal-prompt replies, 38.2 over all 15). It also cut its base's runaway reasoning from 6345 tokens to 1227 and its answer time from 54.5 s to 17.2 s. What it does not do is guarantee an answer - one reply in fifteen reasoned until the budget ran out and returned nothing - nor guarantee working logic, since three clean replies in this study contained real defects and doubled comment delimiters appeared in 2 of 15 minimal and 6 of 15 full-condition replies. It is chat-only: tool calling was trained, measured, and then dropped from this line as a product decision. If you have not already chosen, start with the line default, stcoder-qwen25-7b at Q6_K, and come here for the reasoning family. Either way, keep the context at 8192 or more, treat the output as a fast first draft in the right dialect, and compile everything in LogicStudio before it goes near a machine.</p> |
| </section> |
|
|
| |
| <section> |
| <div class="sh"><span class="sn">A</span><h2>Appendix — the test prompts</h2></div> |
| <div class="tw"><table> |
| <thead><tr><th>ID</th><th style="text-align:left">Prompt as sent to the model</th></tr></thead> |
| <tbody><tr><td><code>analog_scale_raw_to_bar</code></td><td style="text-align:left">We have a 4-20 mA pressure transmitter landing on an analog input card. The raw word comes in as an integer where 5530 counts = 4 mA and 27648 counts = 20 mA, and the transmitter is ranged 0-16 bar. Can you write the Structured Text that turns the raw count into bar, keeps the result inside 0 to 16, and sets a wire-break flag when the loop current drops under about 3.6 mA (roughly 4977 counts)? There is also an enable bit coming from the scan logic - when that is off the scaled output should read zero.</td></tr><tr><td><code>setpoint_clamp_and_ramp</code></td><td style="text-align:left">Operators type a hot water setpoint on the HMI. Anything below 40.0 degC or above 85.0 degC has to be pulled back to the nearest limit and an out-of-range warning bit raised, which the operator can acknowledge once the entry is sensible again. On top of that, the setpoint that actually goes to the controller must not jump more than 2.0 degC per PLC cycle, so ramp it toward the clamped target. Structured Text please.</td></tr><tr><td><code>heater_power_avg_squared</code></td><td style="text-align:left">Heater panel job. I sample the load current every scan and it is noisy, so I need the average of the last 8 samples first. From that averaged current I want the dissipated power using P = I squared times R, with R = 4.7 ohm. If the power stays above 1500 W for 3 seconds, latch an overload bit that only clears when the operator hits the reset pushbutton. Please write it in Structured Text.</td></tr><tr><td><code>timer_start_delay</code></td><td style="text-align:left">We've got a conveyor with a pre-start warning horn. When the operator hits the start button I need the horn to sound for 5 seconds first, then the motor kicks in and the horn goes quiet. Pressing stop, or any fault coming in, has to drop the motor and the horn straight away. Can you write the Structured Text for this?</td></tr><tr><td><code>timer_blink_oscillator</code></td><td style="text-align:left">Need a flashing warning beacon done in ST. While the machine warning flag is active the lamp should blink continuously - lit for 500 ms, then dark for 1.5 seconds, over and over. When the warning clears the lamp stays off. There's no hardware flasher relay on this panel so it has to be done in software.</td></tr><tr><td><code>timer_offdelay_coastdown</code></td><td style="text-align:left">Our extraction fan needs a run-on timer. The fan should run whenever the main motor is running, and then keep running for 30 seconds after the motor is switched off so the duct clears. Emergency stop has to cut the motor and the fan immediately, with no run-on. Also give me a flag I can put on the HMI to show that the purge is in progress. Structured Text please.</td></tr><tr><td><code>io_terminal_seal_in</code></td><td style="text-align:left">On a small booster pump skid the start pushbutton is wired to the first digital input on the CPU and the pump contactor sits on the first relay output. There is an NC stop button on the second input and the motor thermal overload aux contact on the third. Write me the ST for a normal start/stop seal-in: the start button is bouncy so it has to be held for half a second before the pump picks up, an overload trip must latch the pump out, and the lockout should only clear when the operator presses stop with the overload already reset.</td></tr><tr><td><code>status_word_bit_mask</code></td><td style="text-align:left">We poll a 16-bit status word from a VFD over Modbus TCP. The drive manual gives the bit masks in hex: 16#0001 = ready, 16#0002 = running, 16#0008 = fault, 16#0080 = at speed, and 16#0F00 holds a 4-bit warning code. Break the word out into individual booleans, pull the warning code out as a plain integer 0-15, and give me a single 'drive healthy' flag that is only true when it is ready and running with no fault and no warning. If the Modbus poll is not valid, do not trust any of it.</td></tr><tr><td><code>first_out_fault_annunciator</code></td><td style="text-align:left">I need a first-out fault annunciator for a compressor package. The trip inputs are low oil pressure, high discharge temperature, motor overload and high vibration. Whichever one trips first has to be captured and reported to the HMI so the operator can tell at a glance which fault actually caused the trip - later trips must not overwrite it. I also want a common alarm lamp, a horn that goes quiet when acknowledge is pressed while the lamp stays on, and a reset that is only accepted when all four inputs are clear again. The compressor run permissive drops out on any fault.</td></tr><tr><td><code>machine_cycle_full_program</code></td><td style="text-align:left">Can you write the complete program for our drilling station cycle? One part per cycle: when the guard is closed, a part is present and there are no alarms, pressing Start closes the clamp, waits 1.5 s for clamp pressure to build, then the drill motor starts and the head feeds down until the down limit switch, dwells 3 s at depth, retracts until the up limit, opens the clamp after 0.8 s, and the station goes back to idle ready for the next part while incrementing the part counter. E-stop or drill overload must drop every output and put the station into a fault state that only the reset button clears. I want the step sequencing and the timers in there.</td></tr><tr><td><code>reusable_pump_block</code></td><td style="text-align:left">We have three identical booster pumps on a skid and I am tired of copy-pasting the same logic three times. Give me one reusable pump control block that I can instantiate once per pump. Per instance: run request, motor thermal overload contact, dry-run (low suction) switch, fault reset. Outputs: motor run command, fault flag, and a fault code for the HMI so the operator can see what tripped. Behaviour: 2 s start delay after the run request, the dry-run switch is only evaluated 5 s after the motor has actually started, overload trips immediately, any trip latches until it is reset, and once the pump stops it has to stay off for at least 10 s before it may restart. Also show me how it gets called for the three pumps.</td></tr><tr><td><code>analog_scale_helper</code></td><td style="text-align:left">I need a small reusable routine that turns the raw analog word from our pressure transmitters into engineering units, because I have to do the same conversion for three transmitters on the line. They are 4-20 mA into a 0-32767 count input card, so 6553 counts is 4 mA and 32767 counts is 20 mA, and the measuring range is 0-250 bar. It should clamp the result to the range and set an out-of-range flag, plus a broken-wire flag when the raw count drops below 3200 (under 2 mA). A rounded integer in tenths of a bar for the HMI would be handy too.</td></tr><tr><td><code>retain_counter_power_cycle</code></td><td style="text-align:left">Line 3 bottler: a photo eye (bPartDetect) pulses once for every bottle that goes past. I need two totals out of it - a lifetime total that goes into the maintenance records, and a shift total that the supervisor clears with bShiftReset at the start of each shift. The problem is we power the cabinet down every night and in the morning both totals come back as zero, so quality has no numbers to report. Write the ST for the counting logic so the lifetime total is still correct after the panel comes back up.</td></tr><tr><td><code>shared_project_value</code></td><td style="text-align:left">Tank farm project. The level control block works out the tank level in percent and a plant-running flag. The alarm block, the Modbus reporting block and the HMI block all need to see exactly the same two values on every scan - right now each of them recalculates them on its own and they end up disagreeing with each other. Write the level control block so those two values are produced in one place and the rest of the project can use them. Inputs are rLevelRaw from the transmitter (already scaled 0-100), bPumpRunning, bValveOpen and bEStop; the published percent should stay inside 0-100 and a transmitter reading outside range has to be flagged.</td></tr><tr><td><code>machine_data_grouping</code></td><td style="text-align:left">I have three identical packaging machines in one project. For each machine I track a temperature setpoint, the actual temperature, the belt speed, a cycle counter and a state code. I would like each machine's values kept together so I can run the same monitoring logic over all three in a loop instead of copy-pasting the same block three times. Logic per machine: if it is in the running state and the actual temperature is more than 5 degrees off setpoint, or the belt speed drops below 10, fault it; count a cycle on each rising edge of that machine's cycle-done input; EStop faults all of them. Also give me how many machines are currently faulted. Write it in ST.</td></tr></tbody> |
| </table></div> |
| </section> |
|
|
| <footer> |
| <b>Licence.</b> apache-2.0. Fine-tuned from <code>Qwen/Qwen3.5-9B</code>; the base |
| model's attribution and NOTICE are preserved. You may use, modify and redistribute under the |
| same terms.<br><br> |
| <b>Training data.</b> A curated in-house multi-task PLC corpus (approximately 2,930 training and |
| 550 validation examples) focused on Structured Text generation. The dataset is not publicly |
| released.<br><br> |
| <b>A note on tags.</b> The reproduction tags below use an underscore |
| (<code>stcoder_qwen35-9b:q8</code>) because that is how the study's local Ollama models were named. |
| The tag you create by following section 3 uses hyphens |
| (<code>stcoder-qwen35-9b</code>) — the name is arbitrary and does not affect the weights.<br><br> |
| <b>Exact builds compared.</b> Fine-tune <code>stcoder_qwen35-9b:q8</code> versus stock base |
| <code>qwen3.5:9b</code>, both served by local Ollama. The base arm's quantisation was |
| <b>not recorded</b> in the run data; where this page says the base ran "effectively Q4" that is an |
| inference from the Ollama library default for that tag, not a measured value.<br><br> |
| <b>Reproducibility.</b> Harness <code>scripts/compare/paper_eval.py</code>; aggregation |
| <code>paper_analyze.py</code>; violation localisation <code>paper_contrasts.py</code>; prompt set |
| <code>dataset/paper_eval_suite_v1.json</code>; raw rows <code>results/paper/results_v3.jsonl</code>. |
| Every figure on this page is generated from those rows by |
| <code>scripts/compare/build_model_articles.py</code>.<br><br> |
| <b>Where the tool-calling numbers come from.</b> The tool-call composites quoted on this page |
| were measured in a separate internal evaluation that is not part of this release, not in the |
| 240-generation code study, so they cannot be traced to the rows above. The same is true of the |
| per-quantisation figures in section 9, whose code holdout overlapped the training |
| distribution by roughly 84%. |
| </footer> |
|
|
| </div> |
|
|
