Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,293 @@
+---
+license: apache-2.0
+base_model: google/gemma-4-12b-it
+base_model_relation: finetune
+language:
+  - en
+  - tr
+tags:
+  - plc
+  - iec-61131-3
+  - structured-text
+  - code-generation
+  - gguf
+  - ollama
+  - mikrodev
+  - ALB
+  - AdvanceLogicBuilder
+  - MikrodevLogicStudio
+  - advance-logic-builder
+  - mikrodev-logicstudio
+  - gemma
+pipeline_tag: text-generation
+library_name: gguf
+---
+
+# stcoder-gemma4-12b
+
+A chat model that writes **IEC 61131-3 Structured Text for Mikrodev PLCs**, fine-tuned from
+[`google/gemma-4-12b-it`](https://huggingface.co/google/gemma-4-12b-it) and published as GGUF (Ollama, llama.cpp),
+plus the LoRA adapter it was trained with.
+
+This is a fine-tune for the Structured Text coding rules of **Advance Logic Builder (ALB)**, the IEC 61131-3 engineering environment developed by Mikrodev and shipped as Mikrodev LogicStudio. It follows Mikrodev's ST dialect, not generic IEC 61131-3 and not another vendor's conventions.
+
+stcoder-gemma4-12b is a Gemma-4-12B-it fine-tune that writes Mikrodev LogicStudio Structured Text as a chat reply. It is published and supported as a chat-only model with no tool calling, and it is the weakest of the four fine-tuning cases in this study, so choose it only when you specifically want a non-Qwen lineage as a second opinion. The line's default is stcoder-qwen25-7b at Q8_0.
+
+If you are choosing for the first time, start with [`Mikrodev/stcoder-qwen25-7b-gguf`](https://huggingface.co/Mikrodev/stcoder-qwen25-7b-gguf) at **Q8_0** instead (the fastest per answer of the four (5.8 s), 15/15 delivery, and the highest ChrF among the models that answered every prompt (38.5)). Come to this model for the reasons listed under *Is this the right model for you?*.
+
+> **This model cannot make tool calls.** It is a chat model: you describe a plant requirement, it
+> replies with Structured Text. Tool calling was trained and evaluated across this model line — the
+> Qwen3.5-9B family did learn it (tool-call composite 0.816 at Q8_0, falling to 0.642 at Q4_K_M),
+> while the code-focused families scored 0.02–0.06 and the Gemma family exactly 0.00. Because it was
+> not usable across the line it was **dropped as a product decision**, and these builds are published
+> and supported as chat models. Do not build an agent on them. Those tool-call figures come from a
+> separate internal evaluation that is not part of this release — not from the code study reported below, whose own
+> data and method ship with this model as the detailed report.
+>
+> **Always compile generated code in ALB / LogicStudio before deployment.** The model produces
+> dialect-correct code, which is not the same as correct control logic.
+
+## Which build should I download?
+
+**Take Q8_0 (`gemma4_12b-tc.q8_0.gguf`, 11.80 GiB) if it fits.** Q8_0 is the smallest build with no practically measurable loss against the trained weights - it is the precision we recommend for real work.
+
+> **Before you download Q8_0:** it crashed on a 16 GiB card in our own testing at 8k context - if that happens, drop to Q6_K.
+
+| Build | File | Size | Free VRAM needed | In repo | Verdict |
+|---|---|---|---|---|---|
+| **Q8_0** | `gemma4_12b-tc.q8_0.gguf` | 11.80 GiB | 13.8 GiB | in this repo | **recommended**. Crashed on a 16 GiB card in our own testing at 8k context - if that happens, drop to Q6_K |
+| **Q6_K** | `gemma4_12b-tc.q6_k.gguf` | 9.11 GiB | 11.1 GiB | in this repo | fine — build used in the study below |
+| **Q4_K_M** | `gemma4_12b-tc.q4_k_m.gguf` | 6.87 GiB | 8.9 GiB | in this repo | not recommended |
+
+*Free VRAM* is the file plus roughly 2 GiB for the 8192-token context and runtime. Less than that
+and the runtime spills layers to system RAM: it still answers, but the speed figures below no
+longer apply. CPU-only and Apple unified memory work too — same arithmetic against system RAM,
+slower generation.
+
+> **Why we do not recommend Q4_K_M.** Q4_K_M halves the bits per weight again. The only capability we measured across quantisations degraded sharply - tool-call composite 0.816 at Q8_0 against 0.642 at Q4_K_M, measured on the Qwen3.5-9B family, the only one of the four where tool-calling worked at all - and low-bit quantisation is a known source of drift and hallucination on long or unusual requests. Our 15-prompt code study never ran at Q4, so we have no measured code-quality figure for it: treat this as a precaution, not a measured code-quality gap. That measurement is from the Qwen3.5-9B family; this model's own tool-call score was near zero at every quantisation, so there is no quantisation comparison to make for it.
+
+**Also published**
+
+| Artefact | What it is | Size | In repo |
+|---|---|---|---|
+| `lora_adapter/` | The LoRA adapter this model was trained as — PEFT adapter on the base model above. Merge it yourself, stack it, or continue training from it. | small | being uploaded |
+| f16 merged weights | f16 is the merged fine-tune at full precision: the reference build, for evaluation or for producing your own quantisations. | 22.17 GiB | not shipped - rebuild from the adapter (snippet below) |
+| `Modelfile` | One ready-to-run Ollama definition: correct chat template, stop tokens, `num_ctx 8192` (the sequence length this model was trained at) and the system prompt. Its `FROM` points at the Q8_0 file; edit that one line if you downloaded a different quant. | small | in this repo |
+| `article_stcoder-gemma4-12b.html`, `article_stcoder-gemma4-12b.docx` | The full evaluation report for this model. | |
+
+## Quick start with Ollama
+
+Requires **Ollama 0.5 or newer**.
+
+> Gemma turn template, not ChatML.
+
+**1. Install Ollama**
+
+```bash
+# Windows
+winget install Ollama.Ollama
+# macOS
+brew install ollama
+# Linux
+curl -fsSL https://ollama.com/install.sh | sh
+
+ollama --version
+```
+
+**2. Download the model file and its Modelfile** (two files, not the whole repo)
+
+> **Note before you pull several GiB:** Q8_0 crashed on a 16 GiB card in our own testing at 8k context - if that happens, drop to Q6_K.
+
+
+```bash
+pip install -U huggingface_hub
+
+hf download Mikrodev/stcoder-gemma4-12b-gguf gemma4_12b-tc.q8_0.gguf --local-dir .
+hf download Mikrodev/stcoder-gemma4-12b-gguf Modelfile --local-dir .
+```
+
+If `hf` is not found after installing (common on Windows, where the Python scripts directory is
+often not on `PATH`), call it as a module instead — same arguments:
+`python -m huggingface_hub.cli.hf download Mikrodev/stcoder-gemma4-12b-gguf gemma4_12b-tc.q8_0.gguf --local-dir .`
+
+**3. Register it with Ollama and ask it something**
+
+```bash
+ollama create stcoder-gemma4-12b:q8_0 -f Modelfile
+
+ollama run stcoder-gemma4-12b:q8_0 "Silo inlet: on a fill request with the high-level switch clear, open the inlet valve; 3 seconds after the valve open-confirm, start the blower. Any fault stops the blower at once and closes the valve 2 seconds later."
+```
+
+You get a body-only `iecst` block: `VAR` declarations plus logic, millisecond `INT` presets,
+positional function-block calls, no `PROGRAM` wrapper. Paste it into ALB / LogicStudio and compile.
+
+For a different build, download that GGUF instead and change the Modelfile's `FROM` line to its
+filename (the Modelfile lists the alternatives at the top). With llama.cpp instead of Ollama, point `llama-cli -m <file>.gguf` at the
+GGUF and pass the same system prompt.
+
+## Use the LoRA adapter directly
+
+The adapter is published so you can merge it at whatever precision you want, or keep training.
+
+```bash
+pip install -U torch transformers peft accelerate
+```
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained("google/gemma-4-12b-it", dtype=torch.float16,
+                                            device_map="auto")
+tok = AutoTokenizer.from_pretrained("google/gemma-4-12b-it")
+model = PeftModel.from_pretrained(base, "Mikrodev/stcoder-gemma4-12b-gguf", subfolder="lora_adapter")
+
+merged = model.merge_and_unload()          # f16 merged weights
+merged.save_pretrained("stcoder-merged-f16"); tok.save_pretrained("stcoder-merged-f16")
+```
+
+From there `llama.cpp/convert_hf_to_gguf.py` plus `llama-quantize` produces any quantisation you
+prefer — that is exactly how the GGUF files in this repo were built.
+
+## What to ask it
+
+- Grain dryer discharge auger with a pre-start siren. When the operator presses start, the siren sounds for 8 seconds, then the auger motor runs and the siren goes quiet. Stop or any fault drops both immediately.
+- Baghouse fan run-on. The dust collector fan follows the hammer mill and keeps running 45 seconds after the mill stops so the ducting clears. Emergency stop cuts both at once. Give me a purge-in-progress flag for the HMI.
+- Pump-house strobe done in software. While the low-level alarm is active the strobe is lit for 250 ms then dark for 2 seconds, repeating. When the alarm clears the strobe stays off.
+- Start/stop seal-in for a chlorine dosing pump. The start button must be held for 300 ms before the pump picks up, the thermal overload latches the pump out, and the lockout only clears when stop is pressed with the overload already reset. NC stop button, TRUE means healthy.
+- Scale a 0-10 V tank level transmitter to metres. 0 counts is 0 V, 32767 counts is 10 V, and the tank is ranged 0-6 m. Clamp the result to 0-6, set a wire-break flag below 160 counts, and force the output to zero when the enable bit is off.
+
+Keep the system prompt short: the dialect is in the weights. In our evaluation pasting the full
+rule book into the system prompt did **not** improve this model
+(ChrF 35.9 with a short prompt against 32.9 with the full rule book).
+
+## Evaluation
+
+The same 15 plant-engineering requests were sent to the stock base model (`gemma4:12b`) and to this
+fine-tune (`stcoder_gemma4-12b:q6`), twice: once with a short realistic system prompt listing **no dialect rules**,
+and once with the entire rule book in the prompt. Greedy decoding (`temperature 0`,
+`seed 42`), `num_ctx 16384` / `num_predict 8192`, local Ollama on
+an RTX 5080 (16 GB). This model ran at Q6_K — note that is not always the build
+recommended above, and the eval tags used an underscore (`stcoder_gemma4-12b:q6`) where the walkthrough above
+creates a hyphenated tag; the name is arbitrary and does not affect the weights.
+
+> **The shipped Modelfile is not the study configuration.** It defaults to
+> `temperature 0.2` and `num_ctx 8192` (the sequence length this
+> model was trained at) because that is the better interactive default. The numbers below were
+> measured greedily at `num_ctx 16384` / `num_predict 8192`. Match those
+> three values if you want to reproduce them; expect run-to-run variation otherwise.
+
+| Measure | Base | **This fine-tune** | Base + full rules | Fine-tune + full rules |
+|---|---|---|---|---|
+| Dialect-clean replies | 6.7% | **86.7%** | 73.3% | 80.0% |
+| Delivered code at all | 100.0% | 100.0% | 80.0% | 100.0% |
+| R24 rule compliance | 92.55 | 99.61 | 99.75 | 99.41 |
+| ChrF vs reference code | 27.7 | 35.9 | 24.9 | 32.9 |
+| Output tokens (mean · median) | 2490 · 2411 | 401 · 360 | 4585 · 4362 | 921 · 354 |
+| Time per reply | 34.7s | 8.5s | 63.1s | 16.5s |
+| Opened with prose, not code | 100.0% | 20.0% | 80.0% | 6.7% |
+
+- **R24** — dialect rule compliance: 33 weighted forbidden patterns, 0–100. Comments, prose and
+  string contents are stripped before matching. Figures here are computed over the **whole reply**.
+- **ChrF** — character n-gram F-score of the generated code against a hand-written reference.
+- **Clean** — the reply delivered code *and* broke no rule. A reply with no code scores R24 = 100
+  trivially, so delivery is checked first.
+- **Composite** — a single 0–1 roll-up: 0.5 × R24/100 + 0.3 × ChrF/100 + 0.2 × (1 − 6-gram
+  repetition). Useful for ranking, but always read it next to Clean and ChrF.
+- **Which R24 variant** — the figures here score the **whole reply**. Scoring only the fenced code
+  instead lifts the pooled base clean rate across the four models from 6.7% to 10.0%; the
+  base-versus-fine-tune gap survives either choice, but the variant is named so the numbers can be
+  reproduced exactly.
+
+### The four models, identical conditions
+
+ChrF is averaged over *delivered* replies so the denominator is the same for every model.
+
+| Model | Clean | Delivered | ChrF | Composite | Out tok | Per reply |
+|---|---|---|---|---|---|---|
+| [`Mikrodev/stcoder-qwen25-7b-gguf`](https://huggingface.co/Mikrodev/stcoder-qwen25-7b-gguf) | 93.3% | 15/15 | 38.5 | 0.844 | 340 | 5.8s |
+| [`Mikrodev/stcoder-qwen25-14b-gguf`](https://huggingface.co/Mikrodev/stcoder-qwen25-14b-gguf) | 100.0% | 15/15 | 34.9 | 0.842 | 288 | 8.6s |
+| [`Mikrodev/stcoder-qwen35-9b-gguf`](https://huggingface.co/Mikrodev/stcoder-qwen35-9b-gguf) | 93.3% | 14/15 | 40.9 | 0.845 | 1227 | 17.2s |
+| [`Mikrodev/stcoder-gemma4-12b-gguf`](https://huggingface.co/Mikrodev/stcoder-gemma4-12b-gguf) **(this model)** | 86.7% | 15/15 | 35.9 | 0.83 | 401 | 8.5s |
+
+> **Which one should you take?** [`Mikrodev/stcoder-qwen25-7b-gguf`](https://huggingface.co/Mikrodev/stcoder-qwen25-7b-gguf)
+> at **Q8_0** — the fastest per answer of the four (5.8 s), 15/15 delivery, and the highest ChrF among the models that answered every prompt (38.5). The other three are for specific reasons: the 14B if
+> you want the only model that was 100% rule-clean in both prompt conditions, the 9B if you want the
+> reasoning family, the Gemma if you want a non-Qwen lineage as a second opinion.
+
+## Detailed report
+
+The full evaluation for this model — VRAM guidance, step-by-step setup, every one of the 15 test
+prompts with its result, side-by-side base-versus-fine-tune code, the violation breakdown, method
+and limitations — ships in this repo:
+
+- [`article_stcoder-gemma4-12b.docx`](https://huggingface.co/Mikrodev/stcoder-gemma4-12b-gguf/resolve/main/article_stcoder-gemma4-12b.docx) — Word, ~7,000 words
+- [`article_stcoder-gemma4-12b.html`](https://huggingface.co/Mikrodev/stcoder-gemma4-12b-gguf/resolve/main/article_stcoder-gemma4-12b.html) — same content, open locally in a browser
+
+## Strengths
+
+- Under a minimal system prompt the fine-tune lifts dialect compliance from 6.7% clean (1 of 15) to 86.7% clean (13 of 15) and mean R24 from 92.55 to 99.61, with no rules shown to the model. These are whole-reply figures; on fence-only scoring the base rises to 20.0% (3 of 15) and mean R24 94.71 while the fine-tune is unchanged at 86.7% and 99.61. Quantisation was not controlled: the fine-tune ran Q6_K and the stock base ran the Ollama library default, which is an inference of effectively Q4 rather than a recorded value.
+- The habits that break the base model most often disappear. Across the 15 minimal-condition prompts the base broke TIME literals 7 times, named function-block parameters 7 times and POU wrappers 9 times (no_pou_program 6 plus no_pou_fb 3); the fine-tune broke none of the three, leaving one hex literal and one cast function in total. On fence-only scoring the base's TIME-literal and named-parameter counts fall to 4 each, while the 9 POU wrappers stand.
+- It always answers. Delivery is 100% (15 of 15) in both fine-tuned arms, and under the full rule book the stock base returned code for only 12 of 15 prompts while the fine-tune returned 15 of 15.
+- Output is short and paste-ready. Mean 401 tokens and median 360 against the base's 2490 mean and 2411 median, 8.5 s against 34.7 s per reply on an RTX 5080, no reasoning block at all (0% against the base's 100%), and prose around the code in 20% of replies against the base's 100% (6.7% under the full prompt).
+- Different lineage from the other three STCoder models, which makes it usable as an independent second opinion on an ambiguous request rather than a correlated one.
+- It respects some rules it was never shown. On the first-out annunciator prompt it avoided STRING types, single-quoted literals and string functions - all forbidden by the dialect - without being told, scoring R24 100 and ChrF 43.7, its highest ChrF outside the timer prompts.
+
+## Weaknesses
+
+- Weakest fine-tune clean rate and lowest composite of the four models: 86.7% clean (13 of 15) under a minimal prompt and 80.0% (12 of 15) under the full rule book, against 93.3% for stcoder-qwen25-7b, 100.0% for stcoder-qwen25-14b and 93.3% for stcoder-qwen35-9b; composite 0.830 against 0.844, 0.842 and 0.845. Whole-reply scoring in every case.
+- The fine-tuning case is weakest here, and in one respect the base wins. Given the whole rule book in the system prompt, the stock base is 73.3% clean (11 of 15) against this fine-tune's 80.0% (12 of 15), and the base's mean R24 in that condition is higher than the fine-tune's: 99.80 pooled over all 15 rows, 99.75 over its 12 delivered replies, against the fine-tune's 99.41. The fine-tune still wins that condition on delivery (15 of 15 against 12 of 15) and ChrF (32.9 against 24.9 over delivered replies), and quantisation was not controlled between the two arms - but on this lineage a long rule preamble gets a stock Gemma most of the way there.
+- Similarity to house style is mid-pack in this study and last in the earlier one, and the two must not be spliced. In this head-to-head its ChrF is 35.9 over 15 of 15 delivered, above stcoder-qwen25-14b (34.9) and below stcoder-qwen25-7b (38.5, also 15 of 15) and stcoder-qwen35-9b (40.9, but over only 14 of 15). In the earlier 2000-prompt evaluation it is the lowest of the four (38-41 against 47-52 for the Qwen-based siblings) - and that evaluation's code holdout overlapped the training distribution by about 84%, so those absolute scores are inflated and weakly discriminative between models and should not be used to rank them.
+- Adding the rule book makes it worse, not better: clean falls from 86.7% to 80.0%, ChrF from 35.9 to 32.9, and mean output more than doubles from 401 to 921 tokens while the median stays at 354. One reply is responsible for the whole tail, and it is in THIS study, not earlier testing: full condition, first_out_fault_annunciator, 8192 output tokens, truncated mid-declaration, a wall of bResetDone_2 through bResetDone_673 BOOL declarations. It scored R24 100.0 with no violations, so it is counted INSIDE the 80.0% clean figure - read that figure with this caveat. Its ChrF on that row was 6.29.
+- Two rules it still breaks. It keeps hex literals when a manual gives them that way: the VFD status-word prompt produced bReady := (nStatusWord AND 16#0001) <> 0; in both conditions. And it reaches for cast functions in scaling work: the two no_cast_function rows in the full condition are two DIFFERENT prompts, analog_scale_raw_to_bar writing rBar := (REAL(iRawCount) - COUNT_4MA) / (COUNT_20MA - COUNT_4MA) * RANGE_BAR; and analog_scale_helper writing pressureBar := INT(scaled * 10.0); - the same helper prompt without the rule book reached for pressureTenths := REAL_TO_INT(pressureBar * 10.0); so the habit survives both prompt forms.
+- The Q8_0 build (11.8 GiB) has crashed on a 16 GiB card in internal testing. Treat Q6_K as the practical ceiling on 16 GiB.
+
+## Not the right tool for
+
+- Tool calling or function calling of any kind. Tool calling was trained and evaluated for this line, but it was not usable across the line and was dropped as a product decision, so these builds are published and supported as chat-only models. Measured tool-call composite: 0.816 for the Qwen3.5-9B family at Q8_0 (0.805 at Q6_K, 0.642 at Q4_K_M), 0.02-0.06 for the code-focused Qwen2.5-Coder builds, and exactly 0.00 for this Gemma family. If you need an agent that drives IDE tools, do not use these models.
+- Anyone who will not compile the output before deployment. In this model's own test rows, R24-clean code included an undeclared loop variable, an invalid array declaration and a forbidden RETAIN qualifier.
+- Modbus or fieldbus status-word decoding where the manual quotes hex masks. This is its most repeatable rule failure: it keeps 16#0001 style literals rather than converting them to decimal constants.
+- Undecided readers, and anyone choosing on measured quality alone. Take the line's default, stcoder-qwen25-7b at Q6_K (5.82 GiB): in this head-to-head it leads on composite (0.844) and on ChrF at full delivery (38.5 over 15 of 15), and it is the fastest per answer at 5.8 s. If you want the one model that was 100% rule-clean in both conditions, take stcoder-qwen25-14b at Q4_K_M (8.37 GiB), accepting the lowest head-to-head ChrF (34.9) and the slowest generation per token (about 50 tok/s).
+- 8 GiB cards. Q4_K_M weights alone are 6.87 GiB, leaving no useful context budget; use stcoder-qwen25-7b at Q6_K (5.82 GiB) instead.
+
+## Limitations of the evaluation
+
+- **Quantisation was not matched.** This fine-tune ran at Q6_K; the stock base ran at the
+  Ollama library default for `gemma4:12b`, which was **not recorded** in the run data. Where the
+  comparison implies the base was around Q4 that is an inference, not a measurement. Extra precision
+  plausibly helps the fine-tune on quality and demonstrably penalises it on raw throughput.
+- **"Clean" is not "correct".** R24 is reference-free: it certifies that no forbidden construct
+  appears, not that the logic works. Correctness was checked by reading the code, not by compiling it.
+- **Per-quant figures come from a separate evaluation** (code composite 0.800-0.810 and ChrF 38-41 across Q4/Q6/Q8) whose code holdout overlapped
+  the training distribution by roughly 84%. Those absolute scores are inflated and weakly
+  discriminative; use them for the within-model quantisation comparison only, never to rank models
+  against each other.
+- **Scope.** 15 prompts, one dialect, English only, no tool calling, and the prompt set was
+  deliberately built on the categories where this dialect diverges from standard IEC. Treat
+  single-prompt differences as anecdote and the aggregate as the signal.
+
+## Training data
+
+A curated in-house multi-task PLC corpus (approximately 2,930 training and 550 validation examples)
+focused on Structured Text generation, written against the ALB / Mikrodev LogicStudio rule set. The
+dataset is not publicly released. Sequence length 8192 tokens.
+
+Prompts are mostly English with a Turkish minority (10.8% of training prompts contain Turkish). Code, identifiers and code comments are always English; the model's conversational prose follows the language you write in.
+
+## Licence and attribution
+
+apache-2.0. Fine-tuned from [`google/gemma-4-12b-it`](https://huggingface.co/google/gemma-4-12b-it); the base
+model's attribution and NOTICE are preserved. You may use, modify and redistribute under the same
+terms.
+
+## Citation
+
+```bibtex
+@software{stcoder_gemma4_12b,
+  title  = {stcoder-gemma4-12b: Structured Text for Mikrodev Advance Logic Builder},
+  author = {Mikrodev},
+  year   = {2026},
+  note   = {Fine-tuned from google/gemma-4-12b-it}
+}
+```