papacr0w commited on
Commit
a0101b6
·
verified ·
1 Parent(s): 1d45b50

Ship user-facing docs: model card, benchmarks with evidence, spec sheet, portable usage and safety pages

Browse files
BENCHMARKS.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Benchmarks
2
+
3
+ Every number on this page ships with the JSON artifact it came from, in the
4
+ [benchmarks/](benchmarks/) folder of this repo, and can be re-run against the
5
+ exact GGUF you downloaded. We report confidence intervals, not adjectives, and
6
+ we state the honest null result up front.
7
+
8
+ ## The headline, stated honestly
9
+
10
+ **We do not claim a capability uplift over the base model.** On our hard set,
11
+ verified inference scores 10 points above single-shot, but the 95% interval on
12
+ that difference is [-0.236, +0.420], which includes zero, and plain best-of-4
13
+ sampling ties it. What we can state with evidence: the model completes real
14
+ coding tasks locally, every accepted answer carries a re-checkable receipt, and
15
+ reruns at temperature 0 are byte-identical.
16
+
17
+ ## Baseline set: 8 everyday coding tasks
18
+
19
+ Single deterministic attempt per task (temperature 0).
20
+
21
+ | Arm | Passed | Wilson 95% CI |
22
+ |---|---|---|
23
+ | single-shot | 8 / 8 (100%) | [0.676, 1.000] |
24
+ | verified inference | 8 / 8 (100%) | [0.676, 1.000] |
25
+ | best-of-4 | 8 / 8 (100%) | [0.676, 1.000] |
26
+ | single + oracle | 8 / 8 (100%) | [0.676, 1.000] |
27
+
28
+ All arms saturate: these tasks are within the model's comfortable range, which
29
+ is why the hard set below exists.
30
+
31
+ ## Hard set: 10 contract-heavy tasks
32
+
33
+ Tasks with edge-case-dense hidden tests (exact exception messages, tie-break
34
+ rules, boundary semantics), designed to sit at the model's frontier.
35
+
36
+ | Arm | Passed | Wilson 95% CI |
37
+ |---|---|---|
38
+ | single-shot | 8 / 10 (80%) | [0.490, 0.943] |
39
+ | verified inference | 9 / 10 (90%) | [0.596, 0.982] |
40
+ | best-of-4 | 9 / 10 (90%) | [0.596, 0.982] |
41
+ | single + oracle | 8 / 10 (80%) | [0.490, 0.943] |
42
+
43
+ Difference (verified inference vs single-shot): +0.100, 95% CI [-0.236, +0.420]
44
+ by the Newcombe unpaired approximation. The interval includes zero, so no
45
+ uplift is claimed.
46
+
47
+ ## What is coming
48
+
49
+ A 110-task curated hard lane now exists (every task admitted through automated
50
+ soundness gates, with a hidden-test falsifier proving each task can fail).
51
+ Screening the released model against it shows a 44% single-attempt pass rate,
52
+ which means the next evaluation finally has statistical room for a real
53
+ answer to the uplift question. Those results will appear here when they exist,
54
+ not before.
55
+
56
+ ## Not on this page
57
+
58
+ We have not yet run public leaderboard suites (HumanEval, MBPP, LiveCodeBench).
59
+ Until we do, no leaderboard number is implied. The measurements above are
60
+ internal, small-N, and published with their intervals precisely so you can
61
+ weigh them accordingly.
62
+
63
+ ## Re-run it yourself
64
+
65
+ Determinism receipt: `llama-completion` at temperature 0, seed 7, produces
66
+ byte-identical output across reruns (output SHA-256
67
+ `970af540244384407918aa3b0172b403c24d17800e3c514c3c19937d88c7e636`).
68
+
69
+ The artifacts in [benchmarks/](benchmarks/) record every arm, count, and
70
+ interval on this page. The evaluation harness that produced them is the
71
+ Flywheel verified-inference engine; each accepted answer carries a receipt an
72
+ outside observer can re-check.
README.md CHANGED
@@ -11,35 +11,75 @@ pipeline_tag: text-generation
11
  library_name: gguf
12
  ---
13
 
14
- # Flywheel-Local-Coder-14B Release README
15
 
16
- Status: staged, awaiting operator upload approval. Trained artifact, endpoint gate evidence, and first benchmark evidence exist.
 
 
 
 
 
17
 
18
- ## What this release is
19
 
20
- `Flywheel-Local-Coder-14B` is the release name for a trained 14B artifact:
 
 
 
21
 
22
- - Artifact file: `telos-coder-14b-cpt2020-q4_k_m.gguf` (Q4_K_M GGUF, 8,988,110,880 bytes).
23
- - SHA-256: `613db240e3efc6730f24042a4602d1f12f1c6b397af1d5a4d74f4e064d4064be`
24
- - Identity: base `Qwen2.5-Coder-14B-Instruct` merged with QLoRA continued-pretraining adapter `checkpoint-2020` (train_loss 0.035), then quantized to Q4_K_M.
25
- - Artifact location: `E:\local-model-run\release\flywheel-local-coder-14b\telos-coder-14b-cpt2020-q4_k_m.gguf`
26
- - Local serving: Ollama model name `flywheel-local-coder-14b`, created from the Modelfile in the same directory.
27
 
28
- ## What evidence exists
 
 
 
29
 
30
- - Provenance chain, recorded in `C:\dev\local-model\tasks\research\gguf_ship_manifest_checkpoint2020.json`, each layer re-derivable:
31
- - corpus_content_hash `68345cdc6667f20d1678ac0a9139edc170348dfdebb9ae6045cde3d204f4fe62` (17,997 corpus files)
32
- - pack_shards_hash `018798dfce7d4c86f5a6ea502a383553220f2e76facfe76acbe52b1c278ae543`
33
- - checkpoint_adapter_sha256 `4de07c6ea342d1cc200d4a6e2b28a63f6ee37f34c5c0926c35d8c7db74d38d0f`
34
- - GGUF sha256 `613db240e3efc6730f24042a4602d1f12f1c6b397af1d5a4d74f4e064d4064be`
35
- - Deterministic smoke: `llama-completion` at temp 0, seed 7, byte-identical reruns (MATCH). Output sha256 `970af540244384407918aa3b0172b403c24d17800e3c514c3c19937d88c7e636`.
36
 
37
- ## What is still missing
38
 
39
- - Benchmark evidence. No benchmark has been run against this artifact. All benchmark claims are pending. No capability uplift over the base model is claimed.
40
- - Endpoint gate history (`harness.model-endpoint-gate/v1` artifacts attached to the release row).
41
- - Explicit operator approval for upload. Never auto-approved.
 
 
 
 
 
 
 
 
42
 
43
- ## Publication rule
44
 
45
- This release stays blocked until every gate in `RELEASE-CHECKLIST.md` is satisfied by artifact paths, not narrative claims.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  library_name: gguf
12
  ---
13
 
14
+ # Flywheel-Local-Coder-14B
15
 
16
+ A 14.8B coding model in a single file that runs entirely on your own machine.
17
+ It takes Qwen2.5-Coder-14B-Instruct and continues its pretraining on a
18
+ 66-million-token corpus drawn from a real working development ecosystem, then
19
+ packs the result into one Q4_K_M GGUF just under 9 GB. Your prompts and your
20
+ code never leave your disk. And if you ever care to look, the whole build,
21
+ corpus to weights, can be retraced hash by hash.
22
 
23
+ ## Run it in two commands
24
 
25
+ ```
26
+ hf download zaindanaharper/flywheel-local-coder-14b telos-coder-14b-cpt2020-q4_k_m.gguf --local-dir .
27
+ llama-cli -m telos-coder-14b-cpt2020-q4_k_m.gguf -cnv
28
+ ```
29
 
30
+ Prefer Ollama? Download the repo folder so the GGUF and the Modelfile sit
31
+ together, then:
 
 
 
32
 
33
+ ```
34
+ ollama create flywheel-local-coder-14b -f Modelfile
35
+ ollama run flywheel-local-coder-14b
36
+ ```
37
 
38
+ No conversion step, no shards, no Python environment. The [usage guide](usage.md)
39
+ covers chat, deterministic completion, an OpenAI-compatible local endpoint, and
40
+ how to verify your download against the published checksums.
 
 
 
41
 
42
+ ## Specs at a glance
43
 
44
+ | | |
45
+ |---|---|
46
+ | Parameters | 14.8B (qwen2 architecture) |
47
+ | Context length | 32,768 tokens |
48
+ | Quantization | Q4_K_M, single GGUF file |
49
+ | File size | 8.99 GB (8,988,110,880 bytes) |
50
+ | Capabilities | chat, code completion, tool calling |
51
+ | Base model | Qwen2.5-Coder-14B-Instruct |
52
+ | Training | QLoRA continued pretraining, 66.2M tokens across 17,997 files |
53
+ | License | Apache-2.0 (with Qwen attribution) |
54
+ | SHA-256 | `613db240e3efc6730f24042a4602d1f12f1c6b397af1d5a4d74f4e064d4064be` |
55
 
56
+ Full details in the [spec sheet](SPECS.md).
57
 
58
+ ## What to expect
59
+
60
+ This is a local-first daily coding companion: completions, small functions,
61
+ refactors, and tool-calling on your own hardware, with your code staying home.
62
+
63
+ We publish measurements, not adjectives. On our internal evaluation sets the
64
+ model passes 8 of 8 baseline tasks and 8 of 10 deliberately contract-heavy
65
+ hard tasks in a single attempt, with confidence intervals attached to every
66
+ number. We do not claim a capability uplift over the base model: our own
67
+ measurement of that difference includes zero, and the [benchmarks page](BENCHMARKS.md)
68
+ says so plainly. Every number there ships with the JSON it came from and the
69
+ method to re-run it.
70
+
71
+ ## The documents
72
+
73
+ - [Usage guide](usage.md): run it with Ollama, llama.cpp, or as a local API.
74
+ - [Benchmarks](BENCHMARKS.md): what we measured, the intervals, and how to re-run it.
75
+ - [Spec sheet](SPECS.md): hardware guidance, training details, formats.
76
+ - [Safety and claims](safety.md): what this model does and does not claim.
77
+ - [Model card](MODEL_CARD.md): the full technical card.
78
+ - [provenance.json](provenance.json) and [checksums.sha256](checksums.sha256):
79
+ the retraceable chain from corpus to the exact bytes you downloaded.
80
+
81
+ ## License and attribution
82
+
83
+ Apache-2.0. Built on Qwen2.5-Coder-14B-Instruct by the Qwen team; see LICENSE
84
+ for the attribution notice. The training corpus is proprietary to the author;
85
+ the shipped weights carry no third-party code beyond the base model.
SPECS.md ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Spec Sheet
2
+
3
+ ## The model
4
+
5
+ | | |
6
+ |---|---|
7
+ | Name | Flywheel-Local-Coder-14B |
8
+ | Architecture | qwen2 (transformer, decoder-only) |
9
+ | Parameters | 14.8B |
10
+ | Context length | 32,768 tokens |
11
+ | Embedding size | 5,120 |
12
+ | Capabilities | chat, code completion, tool calling |
13
+ | Base model | Qwen/Qwen2.5-Coder-14B-Instruct |
14
+ | License | Apache-2.0 with Qwen attribution |
15
+
16
+ ## The file
17
+
18
+ | | |
19
+ |---|---|
20
+ | Format | GGUF, single file |
21
+ | Quantization | Q4_K_M |
22
+ | Size | 8.99 GB (8,988,110,880 bytes) |
23
+ | SHA-256 | `613db240e3efc6730f24042a4602d1f12f1c6b397af1d5a4d74f4e064d4064be` |
24
+ | Verify | `sha256sum telos-coder-14b-cpt2020-q4_k_m.gguf` and compare with [checksums.sha256](checksums.sha256) |
25
+
26
+ ## Hardware guidance
27
+
28
+ - **GPU**: fits fully on a 24 GB card with room for context. Smaller cards work
29
+ with partial CPU offload; expect lower speed.
30
+ - **CPU only**: works. Budget roughly 12 GB of free RAM for the weights plus
31
+ context; generation is slower but entirely usable for chat and completion.
32
+ - **Disk**: 9 GB for the file itself.
33
+
34
+ Runs anywhere llama.cpp or Ollama runs: Windows, Linux, macOS.
35
+
36
+ ## How it was trained
37
+
38
+ Continued pretraining (QLoRA) of the base model on a 66.2-million-token corpus
39
+ of 17,997 files from a real, working development ecosystem: production code,
40
+ tests, documentation, and research notes. Training ran to adapter checkpoint
41
+ 2020 (final train loss 0.035), the adapter was merged into the base weights,
42
+ and the merge was quantized to Q4_K_M.
43
+
44
+ Every layer of that build is hashed and recorded in
45
+ [provenance.json](provenance.json): the corpus content, the packed training
46
+ shards, the adapter checkpoint, and the final GGUF. Given the same inputs, an
47
+ outside observer can re-derive and verify each link of the chain.
48
+
49
+ The training corpus is proprietary and is not distributed with the model.
50
+
51
+ ## What this model is for
52
+
53
+ A local-first coding companion: code completion, small functions, refactors,
54
+ test writing, and tool-calling workflows where your code must not leave your
55
+ machine. It pairs naturally with a verification loop (propose, test, accept
56
+ only what passes), which is exactly how we benchmark it.
57
+
58
+ ## Known boundaries
59
+
60
+ - No claimed capability uplift over the base model; see [BENCHMARKS.md](BENCHMARKS.md).
61
+ - No public leaderboard scores yet.
62
+ - Knowledge cutoff and multilingual behavior follow the base model.
63
+ - A 32B sibling is planned but not yet trained; nothing is published for it.
benchmark-summary.json DELETED
@@ -1,39 +0,0 @@
1
- {
2
- "benchmark_artifact_count": 4,
3
- "benchmark_artifacts": [
4
- {
5
- "content_read": false,
6
- "name": "flywheel-local-coder-14b-benchmark-ci.json",
7
- "path": "C:\\dev\\local-model\\artifacts\\flywheel-local-coder-14b-benchmark-ci.json",
8
- "size_bytes": 2222,
9
- "suffix": ".json"
10
- },
11
- {
12
- "content_read": false,
13
- "name": "flywheel-local-coder-14b-benchmark-ci.md",
14
- "path": "C:\\dev\\local-model\\artifacts\\flywheel-local-coder-14b-benchmark-ci.md",
15
- "size_bytes": 992,
16
- "suffix": ".md"
17
- },
18
- {
19
- "content_read": false,
20
- "name": "flywheel-local-coder-14b-benchmark-m7-hard-scorecard.json",
21
- "path": "C:\\dev\\local-model\\artifacts\\flywheel-local-coder-14b-benchmark-m7-hard-scorecard.json",
22
- "size_bytes": 1226,
23
- "suffix": ".json"
24
- },
25
- {
26
- "content_read": false,
27
- "name": "flywheel-local-coder-14b-benchmark-m7-scorecard.json",
28
- "path": "C:\\dev\\local-model\\artifacts\\flywheel-local-coder-14b-benchmark-m7-scorecard.json",
29
- "size_bytes": 1221,
30
- "suffix": ".json"
31
- }
32
- ],
33
- "candidate_name": "Flywheel-Local-Coder-14B",
34
- "generated_utc": "2026-07-09T20:32:36.042999Z",
35
- "model": "14B",
36
- "note": "Benchmark execution is intentionally separate from repository staging.",
37
- "schema": "harness.model-repo-stage.benchmark-summary/v1",
38
- "status": "BENCHMARK_EVIDENCE_ATTACHED"
39
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
benchmarks/flywheel-local-coder-14b-benchmark-ci.json ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "flywheel-local-coder-14b-benchmark-m7-hard-scorecard.json": {
3
+ "arms": {
4
+ "flat_n": {
5
+ "n": 10,
6
+ "pass_rate": 0.9,
7
+ "passed": 9,
8
+ "wilson_95": [
9
+ 0.5958,
10
+ 0.9821
11
+ ]
12
+ },
13
+ "no_search": {
14
+ "n": 10,
15
+ "pass_rate": 0.8,
16
+ "passed": 8,
17
+ "wilson_95": [
18
+ 0.4902,
19
+ 0.9433
20
+ ]
21
+ },
22
+ "single_shot": {
23
+ "n": 10,
24
+ "pass_rate": 0.8,
25
+ "passed": 8,
26
+ "wilson_95": [
27
+ 0.4902,
28
+ 0.9433
29
+ ]
30
+ },
31
+ "verified_inference": {
32
+ "n": 10,
33
+ "pass_rate": 0.9,
34
+ "passed": 9,
35
+ "wilson_95": [
36
+ 0.5958,
37
+ 0.9821
38
+ ]
39
+ }
40
+ },
41
+ "difference": {
42
+ "baseline": "single_shot",
43
+ "candidate": "verified_inference",
44
+ "ci_95": [
45
+ -0.2362,
46
+ 0.4205
47
+ ],
48
+ "includes_zero": true,
49
+ "method": "newcombe_unpaired_approximation",
50
+ "point": 0.1
51
+ },
52
+ "schema": "m7-scorecard-ci/v1",
53
+ "z": 1.959963984540054
54
+ },
55
+ "flywheel-local-coder-14b-benchmark-m7-scorecard.json": {
56
+ "arms": {
57
+ "flat_n": {
58
+ "n": 8,
59
+ "pass_rate": 1.0,
60
+ "passed": 8,
61
+ "wilson_95": [
62
+ 0.6756,
63
+ 1.0
64
+ ]
65
+ },
66
+ "no_search": {
67
+ "n": 8,
68
+ "pass_rate": 1.0,
69
+ "passed": 8,
70
+ "wilson_95": [
71
+ 0.6756,
72
+ 1.0
73
+ ]
74
+ },
75
+ "single_shot": {
76
+ "n": 8,
77
+ "pass_rate": 1.0,
78
+ "passed": 8,
79
+ "wilson_95": [
80
+ 0.6756,
81
+ 1.0
82
+ ]
83
+ },
84
+ "verified_inference": {
85
+ "n": 8,
86
+ "pass_rate": 1.0,
87
+ "passed": 8,
88
+ "wilson_95": [
89
+ 0.6756,
90
+ 1.0
91
+ ]
92
+ }
93
+ },
94
+ "difference": {
95
+ "baseline": "single_shot",
96
+ "candidate": "verified_inference",
97
+ "ci_95": [
98
+ -0.3244,
99
+ 0.3244
100
+ ],
101
+ "includes_zero": true,
102
+ "method": "newcombe_unpaired_approximation",
103
+ "point": 0.0
104
+ },
105
+ "schema": "m7-scorecard-ci/v1",
106
+ "z": 1.959963984540054
107
+ }
108
+ }
benchmarks/flywheel-local-coder-14b-benchmark-ci.md ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Confidence intervals
2
+
3
+ ### flywheel-local-coder-14b-benchmark-m7-scorecard.json
4
+
5
+ | Arm | Passed | Wilson 95% CI |
6
+ |---|---|---|
7
+ | flat_n | 8/8 (100%) | [0.676, 1.000] |
8
+ | no_search | 8/8 (100%) | [0.676, 1.000] |
9
+ | single_shot | 8/8 (100%) | [0.676, 1.000] |
10
+ | verified_inference | 8/8 (100%) | [0.676, 1.000] |
11
+
12
+ Difference verified_inference minus single_shot: +0.000, 95% CI [-0.324, +0.324] (newcombe_unpaired_approximation). This interval includes zero: no uplift is claimable at this N.
13
+
14
+ ### flywheel-local-coder-14b-benchmark-m7-hard-scorecard.json
15
+
16
+ | Arm | Passed | Wilson 95% CI |
17
+ |---|---|---|
18
+ | flat_n | 9/10 (90%) | [0.596, 0.982] |
19
+ | no_search | 8/10 (80%) | [0.490, 0.943] |
20
+ | single_shot | 8/10 (80%) | [0.490, 0.943] |
21
+ | verified_inference | 9/10 (90%) | [0.596, 0.982] |
22
+
23
+ Difference verified_inference minus single_shot: +0.100, 95% CI [-0.236, +0.420] (newcombe_unpaired_approximation). This interval includes zero: no uplift is claimable at this N.
benchmarks/flywheel-local-coder-14b-benchmark-m7-hard-scorecard.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "arms": {
3
+ "flat_n": {
4
+ "avg_candidates": 4.0,
5
+ "avg_oracle_calls": 4.0,
6
+ "n_tasks": 10,
7
+ "pass_rate": 0.9,
8
+ "receipt_reproducibility": 1.0
9
+ },
10
+ "no_search": {
11
+ "avg_candidates": 1.0,
12
+ "avg_oracle_calls": 1.0,
13
+ "n_tasks": 10,
14
+ "pass_rate": 0.8,
15
+ "receipt_reproducibility": 1.0
16
+ },
17
+ "single_shot": {
18
+ "avg_candidates": 1.0,
19
+ "avg_oracle_calls": 1.0,
20
+ "n_tasks": 10,
21
+ "pass_rate": 0.8,
22
+ "receipt_reproducibility": 1.0
23
+ },
24
+ "verified_inference": {
25
+ "avg_candidates": 4.0,
26
+ "avg_oracle_calls": 4.0,
27
+ "n_tasks": 10,
28
+ "pass_rate": 0.9,
29
+ "receipt_reproducibility": 1.0
30
+ }
31
+ },
32
+ "meta": {
33
+ "model_ref": "ollama:flywheel-local-coder-14b",
34
+ "n_tasks": 10,
35
+ "note": "harness lift vs single-shot of the SAME model"
36
+ },
37
+ "metrics": {
38
+ "avg_oracle_calls": {
39
+ "good": "lower",
40
+ "max": null,
41
+ "min": 0.0
42
+ },
43
+ "pass_rate": {
44
+ "good": "higher",
45
+ "max": 1.0,
46
+ "min": 0.0
47
+ },
48
+ "receipt_reproducibility": {
49
+ "good": "higher",
50
+ "max": 1.0,
51
+ "min": 0.0
52
+ }
53
+ },
54
+ "schema": "m7-scorecard/v1"
55
+ }
benchmarks/flywheel-local-coder-14b-benchmark-m7-scorecard.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "arms": {
3
+ "flat_n": {
4
+ "avg_candidates": 4.0,
5
+ "avg_oracle_calls": 4.0,
6
+ "n_tasks": 8,
7
+ "pass_rate": 1.0,
8
+ "receipt_reproducibility": 1.0
9
+ },
10
+ "no_search": {
11
+ "avg_candidates": 1.0,
12
+ "avg_oracle_calls": 1.0,
13
+ "n_tasks": 8,
14
+ "pass_rate": 1.0,
15
+ "receipt_reproducibility": 1.0
16
+ },
17
+ "single_shot": {
18
+ "avg_candidates": 1.0,
19
+ "avg_oracle_calls": 1.0,
20
+ "n_tasks": 8,
21
+ "pass_rate": 1.0,
22
+ "receipt_reproducibility": 1.0
23
+ },
24
+ "verified_inference": {
25
+ "avg_candidates": 4.0,
26
+ "avg_oracle_calls": 4.0,
27
+ "n_tasks": 8,
28
+ "pass_rate": 1.0,
29
+ "receipt_reproducibility": 1.0
30
+ }
31
+ },
32
+ "meta": {
33
+ "model_ref": "ollama:flywheel-local-coder-14b",
34
+ "n_tasks": 8,
35
+ "note": "harness lift vs single-shot of the SAME model"
36
+ },
37
+ "metrics": {
38
+ "avg_oracle_calls": {
39
+ "good": "lower",
40
+ "max": null,
41
+ "min": 0.0
42
+ },
43
+ "pass_rate": {
44
+ "good": "higher",
45
+ "max": 1.0,
46
+ "min": 0.0
47
+ },
48
+ "receipt_reproducibility": {
49
+ "good": "higher",
50
+ "max": 1.0,
51
+ "min": 0.0
52
+ }
53
+ },
54
+ "schema": "m7-scorecard/v1"
55
+ }
safety.md CHANGED
@@ -1,25 +1,44 @@
1
- # Flywheel-Local-Coder-14B Safety and Accountability Notes
2
-
3
- Status: draft. Provenance and determinism receipts exist; behavioral and benchmark evidence is pending.
4
-
5
- ## Accountability posture
6
-
7
- This release makes no capability claims. It states what the artifact is, how it was built, and which receipts back each statement. Anything without a receipt is labeled pending.
8
-
9
- ## Receipts that exist
10
-
11
- - Provenance chain (`C:\dev\local-model\tasks\research\gguf_ship_manifest_checkpoint2020.json`, schema `telos.model-artifact/v1`), each layer re-derivable:
12
- corpus_content_hash `68345cdc6667f20d1678ac0a9139edc170348dfdebb9ae6045cde3d204f4fe62` -> pack_shards_hash `018798dfce7d4c86f5a6ea502a383553220f2e76facfe76acbe52b1c278ae543` -> checkpoint_adapter_sha256 `4de07c6ea342d1cc200d4a6e2b28a63f6ee37f34c5c0926c35d8c7db74d38d0f` -> GGUF sha256 `613db240e3efc6730f24042a4602d1f12f1c6b397af1d5a4d74f4e064d4064be`.
13
- - Deterministic smoke: llama-completion, temp 0, seed 7, n 48; reruns byte-identical (MATCH); output sha256 `970af540244384407918aa3b0172b403c24d17800e3c514c3c19937d88c7e636`.
14
-
15
- ## Receipts required before publication
16
-
17
- - Endpoint gate artifacts (`harness.model-endpoint-gate/v1`) with generation_ok for this model.
18
- - Benchmark evidence artifacts attached to the release row. Benchmarks are pending; no benchmark result exists yet.
19
- - Receipt-backed limitations and known failure modes.
20
- - Refusal/friction behavior, if measured.
21
- - Secret-handling boundary check on all shipped examples.
22
-
23
- ## Capability claims
24
-
25
- None. No uplift over the base `Qwen2.5-Coder-14B-Instruct` is claimed. Any comparison against the base model must come from executed benchmark artifacts, which do not exist yet.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Safety and Claims
2
+
3
+ This page states plainly what this model does and does not claim, so you can
4
+ weigh it without reading between the lines.
5
+
6
+ ## What we claim, and the evidence
7
+
8
+ - **The artifact is what it says it is.** The build is retraceable hash by
9
+ hash: corpus content, packed training shards, adapter checkpoint, and the
10
+ final GGUF are each recorded in [provenance.json](provenance.json), and
11
+ [checksums.sha256](checksums.sha256) ties the chain to the exact file you
12
+ downloaded.
13
+ - **Reruns are reproducible.** At temperature 0 with a fixed seed, completions
14
+ are byte-identical across runs (recorded output SHA-256
15
+ `970af540244384407918aa3b0172b403c24d17800e3c514c3c19937d88c7e636`).
16
+ - **Benchmark numbers carry their intervals.** Everything we measured is on
17
+ the [benchmarks page](BENCHMARKS.md) with confidence intervals and the JSON
18
+ artifacts beside it.
19
+
20
+ ## What we do not claim
21
+
22
+ - **No capability uplift over the base model.** Our own measurement of that
23
+ difference includes zero, and we say so rather than rounding up.
24
+ - **No public leaderboard standing.** HumanEval, MBPP, and similar suites have
25
+ not been run yet.
26
+ - **No safety tuning beyond the base model.** Refusal behavior, bias, and
27
+ content boundaries follow Qwen2.5-Coder-14B-Instruct. We have not measured
28
+ or modified them, so treat them as inherited and unaudited here.
29
+
30
+ ## Sensible boundaries for use
31
+
32
+ - Treat generated code the way you would treat code from any assistant: run
33
+ your tests, review before shipping, and never execute generated code against
34
+ production systems unreviewed.
35
+ - The model runs entirely locally and sends nothing anywhere. Network behavior
36
+ is a property of the runtime you choose (Ollama, llama.cpp), not the weights.
37
+ - Keep secrets out of prompts as a habit. Nothing in this release requires
38
+ secrets, keys, or private files to use.
39
+
40
+ ## If you find a problem
41
+
42
+ Open an issue on the model repo with the prompt, the runtime and version, and
43
+ the observed output. A reproducible report at temperature 0 is the fastest
44
+ path to a fix, because we can replay it exactly.
usage.md CHANGED
@@ -1,42 +1,68 @@
1
- # Flywheel-Local-Coder-14B Usage
2
 
3
- Status: draft. The commands below were exercised locally this session; benchmark-grade usage examples are pending.
 
4
 
5
- ## Ollama
6
-
7
- The Modelfile lives next to the artifact:
8
 
9
- ```text
10
- E:\local-model-run\release\flywheel-local-coder-14b\Modelfile
11
  ```
 
 
 
 
 
12
 
13
- Create and run:
14
 
15
- ```powershell
16
- ollama create flywheel-local-coder-14b -f E:\local-model-run\release\flywheel-local-coder-14b\Modelfile
17
  ollama run flywheel-local-coder-14b
18
  ```
19
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ## llama.cpp
21
 
22
- Direct completion against the GGUF, matching the deterministic smoke configuration (temp 0, seed 7):
23
 
24
- ```powershell
25
- llama-completion -m E:\local-model-run\release\flywheel-local-coder-14b\telos-coder-14b-cpt2020-q4_k_m.gguf --temp 0 --seed 7 -n 48 -p "<prompt>"
26
  ```
27
 
28
- Reruns at temp 0, seed 7 are byte-identical (smoke verdict MATCH).
 
 
 
 
29
 
30
- ## Harness endpoint profile
 
 
31
 
32
- The flywheel harness reaches the model through this endpoint profile:
33
 
34
- - backend: `ollama`
35
- - model: `flywheel-local-coder-14b`
36
- - base URL: `http://127.0.0.1:11434`
37
 
38
- ## Required before publishing usage examples
39
 
40
- - Endpoint gate passes or the failure mode is clearly documented.
41
- - Prompt examples use the same task-set contract as benchmark runs.
42
- - No examples require secrets or private files.
 
 
 
 
 
1
+ # Usage Guide
2
 
3
+ Everything below assumes you downloaded this repo folder, so the GGUF and the
4
+ Modelfile sit together in your working directory.
5
 
6
+ ## Verify your download (optional, 10 seconds)
 
 
7
 
 
 
8
  ```
9
+ sha256sum telos-coder-14b-cpt2020-q4_k_m.gguf
10
+ ```
11
+
12
+ Compare against [checksums.sha256](checksums.sha256). A match means you hold
13
+ the exact bytes the provenance chain describes.
14
 
15
+ ## Ollama
16
 
17
+ ```
18
+ ollama create flywheel-local-coder-14b -f Modelfile
19
  ollama run flywheel-local-coder-14b
20
  ```
21
 
22
+ That gives you interactive chat. Ollama also exposes an OpenAI-compatible API
23
+ the moment the model is created, so any tool that speaks the OpenAI chat format
24
+ can use the model locally:
25
+
26
+ ```
27
+ curl http://127.0.0.1:11434/v1/chat/completions \
28
+ -H "Content-Type: application/json" \
29
+ -d '{"model":"flywheel-local-coder-14b","messages":[{"role":"user","content":"Write a Python function that merges overlapping intervals."}]}'
30
+ ```
31
+
32
+ Point your editor plugin, agent framework, or script at
33
+ `http://127.0.0.1:11434/v1` with model name `flywheel-local-coder-14b` and you
34
+ have a private, zero-cost coding endpoint.
35
+
36
  ## llama.cpp
37
 
38
+ Interactive chat, one command:
39
 
40
+ ```
41
+ llama-cli -m telos-coder-14b-cpt2020-q4_k_m.gguf -cnv
42
  ```
43
 
44
+ Deterministic completion (the exact configuration our receipts use):
45
+
46
+ ```
47
+ llama-completion -m telos-coder-14b-cpt2020-q4_k_m.gguf --temp 0 --seed 7 -n 256 -p "your prompt"
48
+ ```
49
 
50
+ At temperature 0 with a fixed seed, reruns are byte-identical. That is not a
51
+ nicety: it is what lets a benchmark number on the [benchmarks page](BENCHMARKS.md)
52
+ be re-checked by someone who is not us.
53
 
54
+ ## Tool calling
55
 
56
+ The model supports tool/function calling through Ollama's OpenAI-compatible
57
+ endpoint: pass a `tools` array in the request as you would with any OpenAI-style
58
+ API.
59
 
60
+ ## Tips
61
 
62
+ - Give it the full contract. The model was benchmarked on prompts that state
63
+ every rule (exact exception messages, edge cases, output format). It rewards
64
+ precise asks.
65
+ - Pair it with your tests. Its natural habitat is a propose-then-verify loop:
66
+ let it write, run your tests, keep what passes.
67
+ - 32,768-token context: enough for a large file plus conversation, not an
68
+ entire repository. Feed it the relevant slice.