Publish ConeML 810M Alpha-Arithmetic release

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+coneml-810m-alpha-arithmetic-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+coneml-810m-alpha-arithmetic-f16.gguf filter=lfs diff=lfs merge=lfs -text

LICENSE.md

@@ -0,0 +1,16 @@
+# License
+
+Copyright 2026 ConeML.
+
+This release is licensed under the Creative Commons
+Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0).
+
+License terms: https://creativecommons.org/licenses/by-nc/4.0/legalcode
+
+You may share and adapt the released material subject to the attribution
+and non-commercial conditions in that license. Commercial use is not
+granted by this release. The material is provided without warranties or
+conditions beyond those required by applicable law.
+
+For commercial evaluation or licensing enquiries, contact ConeML through
+https://huggingface.co/ConeML.

Modelfile

@@ -0,0 +1,16 @@
+FROM ./coneml-810m-alpha-arithmetic-Q8_0.gguf
+
+TEMPLATE """{{- range .Messages }}
+{{- if eq .Role "system" }}System:
+{{ .Content }}
+{{- else if eq .Role "user" }}User:
+{{ .Content }}
+{{- else if eq .Role "assistant" }}Assistant:
+{{ .Content }}<|endoftext|>
+{{- end }}
+{{- end }}Assistant:
+"""
+
+PARAMETER num_ctx 8192
+PARAMETER repeat_penalty 1.15
+PARAMETER stop "<|endoftext|>"

README.md

@@ -0,0 +1,257 @@
+---
+license: cc-by-nc-4.0
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+---
+
+# ConeML 810M Alpha-Arithmetic
+
+ConeML 810M Alpha-Arithmetic is an 810,112,512-parameter, decoder-only
+language model trained from scratch and instruction-tuned for a narrow
+small-number arithmetic profile. It shares its pretrained base with
+`coneml-810m-alpha` and differs in fine-tuning composition.
+
+This variant records the stronger carry/borrow profile of the pair. The
+companion `coneml-810m-alpha` has substantially stronger function-writing
+and two-step results. Results are specific to the documented prompt
+formats and task families, not claims of general mathematical, coding,
+reasoning, or factual-knowledge performance.
+
+- Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied
+  embeddings, 8192 context, 32,768-token BPE tokenizer
+- Pretraining corpus: 27.87B curated tokens
+- Selected pretrained-base exposure: approximately 12.32B token
+  positions, or 15.2 tokens per parameter
+- Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090;
+  fine-tuning and evaluation are excluded
+- Release `model.safetensors` SHA-256:
+  `e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1`
+
+## 1. Measured profile
+
+Across four held-out core lane sets—1-digit addition, 1-digit
+multiplication, carry addition, and borrow subtraction—the model scores
+1,093/1,116 (97.9%). Two of those four held-out sets scored 100%.
+
+This is a result on four explicitly named, task-family-matched sets. It is
+not a claim of 97.9% general arithmetic accuracy. ConeML internal
+certification is not independent third-party certification.
+
+| held-out task family | result | n |
+|---|---:|---:|
+| 2–3-digit addition with carry | 96.8% | 500 |
+| 2-digit subtraction with borrow | 98.6% | 500 |
+| 1-digit addition, including 19 zero-operand items | 100% | 71 |
+| 1-digit multiplication | 100% | 45 |
+| "Which is bigger, X or Y?" | 93.7% | 300 |
+| comparison word problems | 79.0% | 500 |
+| missing addend | 67.4% | 500 |
+| two-step word problems | 43.8% | 500 |
+| 2-digit × 1-digit multiplication | 83.6% | 500 |
+| missing factor | 18.0% | 500 |
+| Python function writing (single function, executed) | 33.7% | 300 |
+
+Behavioral screens:
+
+- Designated-refusal probe: 17/17 refusal prompts produced refusals, and
+  4/5 in-scope contrast prompts were answered correctly. The miss was an
+  incorrect comparison answer, not an over-refusal. This is a small
+  targeted policy test, not general epistemic calibration.
+- Conversation-v2: 8/8 turns passed in one scripted dialogue covering
+  cross-turn arithmetic, story revision, comparison explanation, and one
+  unknown-personal-fact refusal. This is a single dialogue, not a general
+  conversation benchmark.
+- Everyday-reasoning screen: 13/20 by manual adjudication; the
+  phrase-sensitive automatic scorer recorded 6/20.
+
+## 2. Peer comparison and efficiency
+
+The pair was tested against five post-trained peers on the same reduced
+diagnostic screen, through each model's native instruction interface,
+with greedy decoding, repetition penalty 1.15, and matched short-answer
+budgets.
+
+| model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | designated refusals (n=17) |
+|---|---:|---:|---:|---:|
+| ConeML Alpha | 0.81B | 421 (72.0%) | 83 | **13** |
+| ConeML Arithmetic | 0.81B | **442 (75.6%)** | 35 | 11 |
+| Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 0 |
+| Qwen3 | 0.6B | 223 (38.1%) | **98** | 1 |
+| Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 1 |
+| TinyLlama Chat | 1.1B | 113 (19.3%) | 51 | 0 |
+| SmolLM2 Instruct | 1.7B | 380 (65.0%) | 96 | 1 |
+
+This is not a general leaderboard: the task families match ConeML's
+trained surfaces. On this screen, the ConeML pair led mixed arithmetic;
+Alpha exceeded Llama 3.2 and TinyLlama on function writing but trailed
+Qwen3.5, Qwen3, and SmolLM2. All models recorded zero over-refusals on
+five in-scope contrasts.
+
+Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with
+this release's 442/585 (75.6%), while using at least 17.6 times the
+generated-token budget per item and approximately 24 times the recorded
+wall time. Runtime ratios are implementation-specific.
+
+ConeML's selected base consumed 12.32B token positions—15.2 tokens per
+parameter and approximately 5.99e19 training FLOPs under the `6ND`
+convention. Pretraining took approximately 11 days on one RTX 5090.
+Assuming, rather than claiming to have metered, a 0.70 kW average wall
+draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately
+CHF 26–55 of marginal pretraining electricity. This excludes hardware,
+labor, SFT, evaluation, conversion, and emissions accounting.
+
+The complete category table, peer wins, adverse base result, evaluated
+revisions, assumptions, and frozen evidence commitments are in
+[`eval/PEER_COMPARISON.md`](eval/PEER_COMPARISON.md).
+
+## 3. Standard benchmark limits
+
+Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses
+strict final-number matching; HumanEval reports pass@1 from executed
+tests.
+
+| benchmark | this model | same base, no fine-tune |
+|---|---:|---:|
+| GSM8K (test, first 200) | 10/200 (5.0%) | 7/200 (3.5%) |
+| HumanEval (pass@1, tests executed) | 3/164 (1.8%) | 0/164 (0.0%) |
+
+These results are weak and define the boundary of the specialization:
+the four-lane arithmetic profile does not transfer to broad GSM8K or
+HumanEval performance.
+
+## 4. Supported prompt formats
+
+Evaluated decoding is greedy with repetition penalty 1.15, as pinned in
+`generation_config.json`.
+
+```text
+# task format (single turn)
+Question: <task>
+Answer:
+
+# dialogue format (chat_template.jinja)
+User:
+<message>
+Assistant:
+```
+
+Dialogue generation ends at `<|endoftext|>`. Prompting outside these
+formats, including paraphrases, was not systematically evaluated except
+for the published raw-completion spot check.
+
+## 5. Quantized variants
+
+Q8_0 GGUF was evaluated separately under llama.cpp. Its runtime and stop
+handling differ from the bf16 harness, so score differences cannot be
+attributed solely to quantization.
+
+| held-out task family | Q8_0 |
+|---|---:|
+| subtraction with borrow | 98.6% (n=500) |
+| addition with carry | 92.6% (n=500) |
+| 1-digit multiplication | 100% (n=45) |
+| two-step word problems | 32.6% (n=500) |
+
+For local GGUF inference:
+
+```bash
+llama-cli \
+  -m coneml-810m-alpha-arithmetic-Q8_0.gguf \
+  -p $'Question: What is 47 + 38?\nAnswer:' \
+  -n 32 --repeat-penalty 1.15 --no-conversation --single-turn
+```
+
+For Ollama, download `Modelfile` beside the Q8_0 file, then run:
+
+```bash
+ollama create coneml-810m-alpha-arithmetic -f Modelfile
+ollama run coneml-810m-alpha-arithmetic "What is 47 + 38?"
+```
+
+## 6. Loading with Transformers
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo_id = "ConeML/coneml-810m-alpha-arithmetic"
+tokenizer = AutoTokenizer.from_pretrained(repo_id)
+model = AutoModelForCausalLM.from_pretrained(
+    repo_id,
+    dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+prompt = "Question: What is 47 + 38?\nAnswer:"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+output = model.generate(
+    **inputs,
+    max_new_tokens=32,
+    do_sample=False,
+    repetition_penalty=1.15,
+)
+answer = tokenizer.decode(
+    output[0][inputs["input_ids"].shape[-1]:],
+    skip_special_tokens=True,
+)
+print(answer)
+```
+
+This is a text-only causal language model. Load it with
+`AutoModelForCausalLM` or `LlamaForCausalLM`, not a multimodal model class.
+
+## 7. Intended use and safety
+
+This release is intended for research and constrained local workflows
+inside its measured small-number arithmetic envelope. It is not a general
+mathematics system. Validate outputs whenever exactness matters.
+
+It is not intended for factual retrieval, high-stakes medical, legal, or
+financial decisions, autonomous action, or unsandboxed execution of
+generated code. Generated code may be incomplete or unsafe; inspect and
+test it in an isolated environment.
+
+## 8. Known limitations
+
+- The measured strength is narrow. Missing factor scored 18.0%, two-step
+  word problems 43.8%, 2-digit × 1-digit multiplication 83.6%, and
+  executed single-function writing 33.7%.
+- Not a code model: complete programs scored 0/10, executable SQL 1/10,
+  and bash 1/5.
+- The model was not evaluated as a factual-QA system and refused all six
+  real-world factual prompts in the designated-refusal probe.
+- Money/decimal change-making scored 0/3 on the held-out screen; the model
+  also showed yes-bias on feasibility questions.
+- Multi-step GSM-class reasoning remained weak.
+- Frameless prompting was spot-checked, not systematically certified.
+
+## 9. Evidence and reproducibility
+
+The `eval/` directory publishes:
+
+- `summary.json`: all aggregate bf16 results used above;
+- `representative-samples.json`: deterministic, rule-selected examples
+  from ConeML-owned probes;
+- `EVALUATION_METHODOLOGY.md`: evaluation definitions and scope;
+- separate Q8_0 and focused aggregate results; and
+- `PRIVATE_EVIDENCE_SHA256SUMS.txt`: commitments to the complete
+  row-level evidence retained by ConeML.
+- `peer-comparison-summary.json`, `PEER_COMPARISON.md`, and
+  `PEER_EVIDENCE_SHA256SUMS.txt`: the peer results, interpretation, and
+  commitments to the retained peer-generation evidence.
+
+Complete proprietary probe rows and generations are retained privately to
+avoid releasing the full measurement instruments. The hashes permit later
+disclosure to be checked against the evidence frozen at release time.
+Public GSM8K and HumanEval rows are not redistributed; only aggregate
+results are published. `SHA256SUMS.txt` covers every shipped file.
+
+## 10. License and commercial enquiries
+
+Released for non-commercial use under CC BY-NC 4.0. Commercial use is not
+granted by this release. See `LICENSE.md`.
+
+For commercial evaluation or licensing enquiries, contact ConeML through
+the [ConeML Hugging Face organization](https://huggingface.co/ConeML).

SHA256SUMS.txt

@@ -0,0 +1,21 @@
+846a8e2297580b3c8d57f69a07d88833d370f976010b31d9fc9b93e9a3f04464  LICENSE.md
+1756d02d239f8e8342a1aa513248f9d35b9a5168c77d07619e64a90e20580434  Modelfile
+45beb690d5c990dd9660ef6cbb637b3961ef20c804507d336286f2a6cbe5849d  README.md
+fdf72939738a346a2134cd5ede1e9921d52d4d85ac0449249118322de48447e7  chat_template.jinja
+147f479696718b5665f3aea3a7ae312cc6850d998246141f6ecbbd7e395bce2d  coneml-810m-alpha-arithmetic-Q8_0.gguf
+5898b62c447961cb7eb60fc538fc9f4b0f0bdda840fda87a861d01c245085c31  coneml-810m-alpha-arithmetic-f16.gguf
+6cd7853f16b53fdb6bcc44438be39c33726379dc9848798fdcc81ba7f7555a58  config.json
+9637d2b8ef44c95226638a31b7dea6fb3b6f817a297a1a25098f8215b0435489  conversion.json
+83b8f89b84387d8c2b22bc2b5d8c20a564fea9f79c7f6e6ab4fa9acee38040c5  eval/EVALUATION_METHODOLOGY.md
+3817126000edea9bc2fcb75a783fa73de234590cdfe89e1aea9b569bae809caa  eval/PEER_COMPARISON.md
+73d8100a4b78d7693aabb7a22f3b9948774f7a418d9fa162d19d6ff630baffd4  eval/PEER_EVIDENCE_SHA256SUMS.txt
+e5722a62e0d7bf0d4d1ced1626cd5bc49381380aeeea671f6f40a4705740e088  eval/PRIVATE_EVIDENCE_SHA256SUMS.txt
+f57c01a959c7944ac2e5bd516985115f67409b81445aca8a857c131f7b1f459e  eval/certification-Q8_0.json
+4388d66b5feeae2147beebc02967d536ec06a7d284ca245b119fa0dcc1b8cf8c  eval/focused-summary.json
+baff252791bdf43653763a53a57024d0075bb9a1f6b35cf1fcc94a9191cc73df  eval/peer-comparison-summary.json
+8bfd98eac2f9309c7481115b91ae47b8f01c9e3b44c80b2bb233a5be6947e8cc  eval/representative-samples.json
+ed8c2a178bf3268b89f325b36538b71aef54251adb53396e259f9534e6264504  eval/summary.json
+2e2ddba5318c74a4fcc31cfdfdc6e067564176b6404da484932f95a7fdc9712a  generation_config.json
+e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1  model.safetensors
+dd9e18c94d2206bb6c3e26caccc98f5b72dd930784cc37e989439b79ceacbf47  tokenizer.json
+88a2d6096fc775fca098fa3d8f4be827b498d21af168e24ba3bd0509b8a63c59  tokenizer_config.json

chat_template.jinja

@@ -0,0 +1,8 @@
+{% for message in messages %}{% if message['role'] == 'system' %}System:
+{{ message['content'] }}
+{% elif message['role'] == 'user' %}User:
+{{ message['content'] }}
+{% elif message['role'] == 'assistant' %}Assistant:
+{{ message['content'] }}{% if not loop.last %}{{ eos_token }}
+{% endif %}{% endif %}{% endfor %}{% if add_generation_prompt %}Assistant:
+{% endif %}

coneml-810m-alpha-arithmetic-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:147f479696718b5665f3aea3a7ae312cc6850d998246141f6ecbbd7e395bce2d
+size 862157472

coneml-810m-alpha-arithmetic-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5898b62c447961cb7eb60fc538fc9f4b0f0bdda840fda87a861d01c245085c31
+size 1621555872

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 0,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 1536,
+  "initializer_range": 0.02,
+  "intermediate_size": 4608,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 3,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 1000000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.12.1",
+  "use_cache": true,
+  "vocab_size": 32768
+}

conversion.json

@@ -0,0 +1,7 @@
+{
+  "artifact": "ConeML/coneml-810m-alpha-arithmetic",
+  "conversion": "weights-only safetensors export",
+  "dtype": "bfloat16",
+  "missing_keys": [],
+  "unexpected_keys": []
+}

eval/EVALUATION_METHODOLOGY.md

@@ -0,0 +1,59 @@
+# ConeML 810M — evaluation methodology and variant lineage (public record)
+
+Two released variants share one from-scratch 810M base checkpoint and
+differ only in supervised fine-tuning; both were evaluated with the same
+bf16 instruments.
+
+- `coneml-810m-alpha` — function-writing / mixed-arithmetic profile
+- `coneml-810m-alpha-arithmetic` — carry/borrow arithmetic variant
+
+## Methodology
+
+- **ConeML internal certification**: large-n (300–500 per task family)
+  batteries of unseen instances in the trained prompt formats, greedy
+  decoding, repetition
+  penalty 1.15. Code tasks are scored by EXECUTING the generated function
+  against held-out tests; SQL by executing against a live sqlite database;
+  scripts by captured stdout. No pattern-match scoring for anything
+  execution can decide.
+- **Behavior probes**: held-out designated-refusal prompts plus in-scope
+  contrast pairs in matching wording; one separately rerun, scripted
+  eight-turn conversation; and an everyday-reasoning screen with manual
+  adjudication. Aggregate results and deterministic representative
+  ConeML-owned rows are public; complete rows are retained privately.
+- **Public benchmarks**: GSM8K (first 200, zero-shot, strict final-number
+  match) and HumanEval (pass@1, tests executed), identical harness across
+  variants and the pretrained base. Aggregate results are public;
+  benchmark rows are not redistributed.
+- **Quantized variants are evaluated separately**: each shipped GGUF carries
+  its own result table. The GGUF and bf16 harnesses differ in runtime and
+  stop handling, so score gaps are not attributed solely to quantization.
+
+## Why two variants
+
+Under fixed architecture and base checkpoint, modest differences in
+fine-tuning composition produced large, cross-surface capability
+tradeoffs (e.g., one build scores 96.8% on held-out carry addition while
+the other scores 79.0% on held-out two-step word problems and 69.0% on
+held-out function writing). Rather than presenting one checkpoint as
+dominant, each variant ships with its measured envelope and a
+recommendation for which profile
+fits which pipeline. A checkpoint's profile is measured, never inferred.
+
+## Shared limitations
+
+Function-writing is evaluated for single Python functions only — neither
+variant is a general code model (complete programs, executable SQL, and
+bash scored near zero for both and are stated out of scope). Both variants
+refused every designated refusal prompt, including real factual questions;
+this targeted behavior is not evidence of general epistemic calibration.
+Multi-step GSM-class reasoning is weak in every variant tested this cycle.
+
+## Evidence disclosure
+
+The public folders contain aggregate result tables, deterministic
+representative examples from ConeML-owned instruments, and SHA-256
+commitments to the complete row-level evidence. Full proprietary probe
+rows are retained by ConeML rather than distributed. This preserves the
+measurement instruments while allowing later disclosure to be checked
+against the evidence frozen for this release.

eval/PEER_COMPARISON.md

@@ -0,0 +1,128 @@
+# ConeML 810M peer diagnostic and resource context
+
+Date: 2026-07-30
+
+This is an interface-specific diagnostic screen, not a neutral ranking of
+general model capability. Each instruction-tuned model was evaluated
+through its own instruction interface. All models used greedy decoding
+with repetition penalty 1.15. Arithmetic and function-writing used
+matched short-answer generation budgets.
+
+The task families match ConeML's supervised training surfaces. That gives
+the comparison practical value for locating the release's envelope, but
+it also means these results must not be generalized to unrelated tasks.
+Standard GSM8K and HumanEval results remain separately disclosed in the
+model cards.
+
+## Instruct-model results
+
+| measured surface | ConeML Alpha | ConeML Arithmetic | Qwen3.5 0.8B | Qwen3 0.6B | Llama 3.2 1B Instruct | TinyLlama Chat | SmolLM2 1.7B Instruct |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| mixed arithmetic screen (n=585) | 421 (72.0%) | **442 (75.6%)** | 154 (26.3%) | 223 (38.1%) | 343 (58.6%) | 113 (19.3%) | 380 (65.0%) |
+| four core arithmetic lanes (n=225) | 177 (78.7%) | **221 (98.2%)** | 150 (66.7%) | 208 (92.4%) | **221 (98.2%)** | 92 (40.9%) | **221 (98.2%)** |
+| executed single functions (n=100) | 83 | 35 | 93 | **98** | 79 | 51 | 96 |
+| designated-refusal prompts (n=17) | **13** | 11 | 0 | 1 | 1 | 0 | 1 |
+| over-refusals on contrasts (n=5) | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+
+The honest result is mixed:
+
+- At matched short-answer budgets, the ConeML pair led this post-trained
+  peer group on the broader arithmetic screen.
+- On the four core arithmetic lanes, the Arithmetic variant tied Llama
+  3.2 Instruct and SmolLM2 at this reduced sample size. Its separate
+  full-size internal result is 1,093/1,116 (97.9%).
+- Alpha exceeded Llama 3.2 Instruct and TinyLlama Chat on the executed
+  function screen, but Qwen3.5, Qwen3, and SmolLM2 scored higher.
+- The refusal row measures a trained response policy on designated
+  prompts, not factual correctness or general epistemic calibration.
+
+The peer-harness refusal results above differ from the dedicated ConeML
+probe's 17/17 for each release because the generation loops differ. Both
+measurements are reported rather than merged.
+
+## Qwen3.5 thinking-mode sensitivity
+
+Qwen3.5 0.8B was also evaluated with thinking enabled on the same 585
+arithmetic items:
+
+| metric | Qwen3.5 thinking | Qwen3.5 short-answer | ConeML Arithmetic short-answer |
+|---|---:|---:|---:|
+| accuracy | 451/585 (77.1%) | 154/585 (26.3%) | 442/585 (75.6%) |
+| generated tokens per item | mean 844; median 699 | at most 48 | at most 48 |
+| p95 / generation cap | 1,536 / 1,536 | 48 / 48 | 48 / 48 |
+| truncation rate | 138/585 (23.6%) | approximately 0% | approximately 0% |
+| mean wall time per item | 2.43 s | approximately 0.1 s | approximately 0.1 s |
+| correct answers per 1,000 generated tokens | 0.91 | 5.5 | approximately 15.7 or higher |
+
+Thinking mode recovered Qwen3.5 to statistical parity with ConeML
+Arithmetic on this screen, while using at least 17.6 times the
+per-answer generation budget and approximately 24 times the measured wall
+time. Wall-time ratios are specific to the recorded hardware, batching,
+and implementation. This is an inference-cost comparison, not a claim
+that thinking mode is intrinsically inferior.
+
+## Against-interest base result
+
+On the same 585 arithmetic items, through a task frame native to neither
+base model, Qwen3.5 0.8B Base scored 492/585 (84.1%), while ConeML base
+checkpoint 188 scored 156/585 (26.7%). This result is included because it
+prevents an absolute-superiority reading: high-exposure base models may
+already contain strong task-formatted behavior, and post-training can
+move capability between output surfaces.
+
+## Training-resource context
+
+ConeML's selected base consumed approximately 12.32B token positions:
+15.2 tokens per parameter and approximately 5.99e19 training FLOPs under
+the `6 × parameters × tokens` convention.
+
+| model family | disclosed pretraining tokens | approximate tokens/parameter | approximate training FLOPs vs ConeML |
+|---|---:|---:|---:|
+| ConeML 810M | 12.32B | 15.2 | 1× |
+| TinyLlama 1.1B | 3T | 2,727 | 330× |
+| Llama 3.2 1B | up to 9T | 7,258 | 1,118×, plus distillation |
+| SmolLM2 1.7B | 11T | 6,471 | 1,873× |
+| Qwen3 0.6B | 36T | 60,000 | 2,164× |
+| Qwen3.5 0.8B | not disclosed | not stated | not stated |
+
+These are estimated pretraining FLOP ratios, not historical electricity
+or monetary costs for the peer models.
+
+ConeML pretraining took approximately 11 days on one local RTX 5090. At
+an explicitly assumed average wall draw of 0.70 kW, that corresponds to
+184.8 kWh. Applying an assumed Swiss residential tariff range of
+CHF 0.14–0.30/kWh gives approximately CHF 26–55 of marginal pretraining
+electricity.
+
+| ConeML pretraining unit | estimate |
+|---|---:|
+| average throughput | 12,963 token positions/s |
+| wall time per billion token positions | 21.4 h |
+| energy per billion token positions | 15.0 kWh |
+| energy per million token positions | 15 Wh |
+| estimated wall energy per token position | 0.054 J |
+| marginal electricity per billion token positions | CHF 2.10–4.50 |
+
+The energy and cost figures are estimates derived from the stated power
+and tariff assumptions, not meter readings. They exclude hardware,
+depreciation, labor, supervised fine-tuning, evaluation, conversion,
+datacenter PUE, and carbon intensity. No emissions claim is made.
+
+## Evaluated revisions
+
+- `Qwen/Qwen3.5-0.8B@2fc06364715b967f1860aea9cf38778875588b17`
+- `Qwen/Qwen3-0.6B@c1899de289a04d12100db370d81485cdf75e47ca`
+- `unsloth/Llama-3.2-1B-Instruct@5a8abab4a5d6f164389b1079fb721cfab8d7126c`
+- `TinyLlama/TinyLlama-1.1B-Chat-v1.0@fe8a4ea1ffedaf415f4da2f062534de366a451e6`
+- `HuggingFaceTB/SmolLM2-1.7B-Instruct@31b70e2e869a7173562077fd711b654946d38674`
+
+The exact aggregates are in `peer-comparison-summary.json`. Complete
+generation rows are retained privately; their frozen SHA-256 commitments
+are published in `PEER_EVIDENCE_SHA256SUMS.txt`.
+
+## Primary resource disclosures
+
+- Qwen3 pretraining: [Qwen3 release post](https://qwenlm.github.io/blog/qwen3/)
+- Llama 3.2 token count and distillation: [Meta Llama 3.2 model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md)
+- TinyLlama token count: [TinyLlama model card](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
+- SmolLM2 token count: [Hugging Face SmolLM2 model card](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B)

eval/PEER_EVIDENCE_SHA256SUMS.txt

@@ -0,0 +1,31 @@
+96a805aed179f77d9f1959216f9a39e8bf8d50dea5465eeea1eb24e96502a6af  SOURCE_PEER_COMPARISON_V2.md
+a5d6cd4d7a2643b316cefb1dcb6f43296bb69da4064dbc3b3ab4a123bf36b2b1  base-reference/coneml-base188.json
+83643ce110655e708d50f4eb3ee29ab08ab8e003768eb5b5da9a3e545bc9b8ef  base-reference/coneml-base188.rows.jsonl
+ca387426c55ac72d50041e2f082bd4cdc747eedb51deca7929cf7a4acea2c5b6  base-reference/coneml-native-transitive.json
+43ef584d37c6a276b190aeb741597b54bfebbf7bbd9c37df5d82a2e15a8ee2d1  base-reference/qwen35-0.8b-base.json
+25de33180d4073777d29871882b57f1556733e9a8c1e10eba14a8cac062b5c35  base-reference/qwen35-0.8b-base.rows.jsonl
+2003c73a54602f3a30980d6c7f85feef39b63fd11cef9fc20a90044d3569ba77  instruct/coneml-810m-alpha.json
+8da107545bd6717c7795ff4e827698e5e269d451f2390f24dccc516fc390ab44  instruct/coneml-810m-alpha.rows.jsonl
+77d514c14e14557c4ac30e8a8c3f4c5887f0db0dff39b8ea14baea164327d027  instruct/coneml-810m-arithmetic.json
+a60ed3e533c0c4a77b34a21f80b8df8de85e3df074f3cacdc3bbd4b5e0741cce  instruct/coneml-810m-arithmetic.rows.jsonl
+aa35b93f46778ffb22351ac411f2fc04e645775e416b5a9423f925bb6d6b1609  instruct/llama32-1b-instruct-transchat.json
+ee514c35d86f27172a004325b26da915f869164adae59edf8d95b01d958e683f  instruct/llama32-1b-instruct-transchat.rows.jsonl
+f4553ef34844e55364a87f631fac03a3545acbb14cd12dcf182f16151b5e5f8a  instruct/llama32-1b-instruct.json
+d5c0fd8b00b50754ac7dede96081afeaf3143795e005134cf5543483859340bf  instruct/llama32-1b-instruct.rows.jsonl
+a61b56681810410dfc3a8d8e758b8ccd11d01ebcce2d963d6bdbca4644c2cee9  instruct/qwen3-0.6b-transchat.json
+f7c763441edc0d5e37a34902a3cc4efcacc6a35b47056adb9356879a4407cb31  instruct/qwen3-0.6b-transchat.rows.jsonl
+3f34342d79095a5754512ebf57192fdf98a16b364a250712993ac1da0b65e252  instruct/qwen3-0.6b.json
+05847f4187651780008331a85c9cd5fa84d3b636320c9a809fb3c307f1a230d6  instruct/qwen3-0.6b.rows.jsonl
+21984e00a4a6da3d2887138ede25c03e9007fee0be41de462ef58dc58db89554  instruct/qwen35-0.8b-transchat.json
+48a53b6bf3829c207953b1075160df9f6702b1fc2162426577aa69530856e555  instruct/qwen35-0.8b-transchat.rows.jsonl
+fa11fbf67c047ac541ecd38571867f9fe68bfb8aec12b84f2addda8a6d561642  instruct/qwen35-0.8b.json
+74a6debf6ae55db9895f556e654b6905fde871358a94c9c4a2730a418b0e2608  instruct/qwen35-0.8b.rows.jsonl
+72598668bfce72497ac67a4d83d1cad7538c63ec933e8a1d1198876d85dc8a67  instruct/qwen35-thinking-sensitivity.json
+8917677a4c9d29124c5ade229f7080bbd0ccb83471c3d7d39a6f958224596a8f  instruct/smollm2-1.7b-instruct-transchat.json
+deb9eb805475e4ad2d158283b469830138fdf42ee0c9a903bf3611b43a7869ff  instruct/smollm2-1.7b-instruct-transchat.rows.jsonl
+94a3440cdaae14eb27beaf36a1a0103dab4a6d292bc382981f0514a82a93ed0f  instruct/smollm2-1.7b-instruct.json
+1b06370ad7b940ed05b59ffe4acd19035341cfe078ed7b01d59d168abc8d0c71  instruct/smollm2-1.7b-instruct.rows.jsonl
+bdd7cf2867643439e90a3e04cb209e6130839af9422e492bcd2eb1c2cba4556e  instruct/tinyllama-chat-transchat.json
+472aad0c169c4f943fbc9a8367d65fa2d5c41acbbee66d099e16a286f8676d1c  instruct/tinyllama-chat-transchat.rows.jsonl
+3a7c2e7ebcaa78b4b7d9b6d84a5b90a495f260f97287bad3282d54fc84e0c009  instruct/tinyllama-chat.json
+5926a31e926279f8d9459d19cbbe9730a06071bb8b4029c2f9174804ab447943  instruct/tinyllama-chat.rows.jsonl

eval/PRIVATE_EVIDENCE_SHA256SUMS.txt

@@ -0,0 +1,10 @@
+532b8cd9c9f7eb7cd047aac26dfe6a99880503a7b09bf79314287c28cb908ce4  base188-public-benchmarks.json
+9ae0002c88cd5972d21fe119734421f03f8ffcaa412002476ae5b0244bc302b7  basic-code.json
+83ff212bfd15d342856cf1ececf46648d458b1a0010afc1538b3a87b7ef1b86b  certification-v1.json
+bed82d5fefc647f91d53cd22376fbb3b598ea417e317e698629bc82d07b6479c  code-cert.json
+3b263314c8690fb912a03fc87b5d81dd38e499a6f3ef4353dee124239572a826  conversation.log
+ebca66bd78a552bf8e70498f5b38a778e27894afb2f47c122da29195d85dba3e  conversation-v2.json
+e7c6a5a5e7b7cb8c35cba5545cc56e79e01571e550a5958cad0ad3e9a0d1b9d6  everyday-manual-adjudication.json
+3b698faa42fe5b556abcb7ba4ac39866a94ffd2413dcd5987251e9c5b46b1b91  public-benchmarks.json
+3e28a81a3218e97503a4f572c5dc8cedd4943b02bf3de046614661f0d88bb247  raw-probe.jsonl
+66f34c9e24e01a3e26db9faaf3606e510eb42639cd32169e500e17d1a2d53b38  refusal-probe.json

eval/certification-Q8_0.json

@@ -0,0 +1,55 @@
+{
+  "by_category": {
+    "add-1d": {
+      "correct": 70,
+      "n": 71,
+      "acc": 0.9859
+    },
+    "add-2d-carry": {
+      "correct": 463,
+      "n": 500,
+      "acc": 0.926
+    },
+    "comparison": {
+      "correct": 363,
+      "n": 500,
+      "acc": 0.726
+    },
+    "missing-addend": {
+      "correct": 337,
+      "n": 500,
+      "acc": 0.674
+    },
+    "missing-factor": {
+      "correct": 92,
+      "n": 500,
+      "acc": 0.184
+    },
+    "mul-1d": {
+      "correct": 45,
+      "n": 45,
+      "acc": 1.0
+    },
+    "mul-2dx1d": {
+      "correct": 398,
+      "n": 500,
+      "acc": 0.796
+    },
+    "sub-borrow": {
+      "correct": 493,
+      "n": 500,
+      "acc": 0.986
+    },
+    "two-step": {
+      "correct": 163,
+      "n": 500,
+      "acc": 0.326
+    },
+    "which-bigger": {
+      "correct": 277,
+      "n": 300,
+      "acc": 0.9233
+    }
+  },
+  "runtime": "llama.cpp Q8_0 aggregate summary"
+}

eval/focused-summary.json

@@ -0,0 +1,25 @@
+{
+  "ckpt": "ConeML/coneml-810m-alpha-arithmetic",
+  "math_200": 138,
+  "everyday_20_auto": 6,
+  "by_category": {
+    "everyday::causal-evidence": "0/3",
+    "everyday::everyday-arithmetic": "2/2",
+    "everyday::money-change": "0/3",
+    "everyday::object-choice": "2/3",
+    "everyday::practical-judgment": "1/2",
+    "everyday::quantity-judgment": "0/2",
+    "everyday::temporal-order": "0/2",
+    "everyday::time-margin": "1/3",
+    "math::add-1d": "20/20",
+    "math::add-2d-carry": "30/30",
+    "math::missing-addend": "18/20",
+    "math::mul": "20/20",
+    "math::sub-2d-borrow": "29/30",
+    "math::transitive": "0/20",
+    "math::word-2step": "2/40",
+    "math::word-compare": "19/20"
+  },
+  "math_200_corrected": 154,
+  "correction": "transitive items scored by expected-name match on first line; original scorer extracted numbers only and zeroed all transitive rows"
+}

eval/peer-comparison-summary.json

@@ -0,0 +1,132 @@
+{
+  "schema_version": 1,
+  "date": "2026-07-30",
+  "scope": "Interface-specific diagnostic screen, not a neutral ranking of general model capability. Each instruct model used its native instruction interface. Arithmetic and function-writing used matched short-answer generation budgets.",
+  "decoding": {
+    "strategy": "greedy",
+    "repetition_penalty": 1.15,
+    "arithmetic_n": 585,
+    "core_arithmetic_n": 225,
+    "function_writing_n": 100,
+    "designated_refusal_n": 17,
+    "refusal_contrast_n": 5
+  },
+  "evaluated_revisions": {
+    "ConeML/coneml-810m-alpha": "local release artifact e5df8f2edd773ad079ba00d383ace81b96cbb9579d64c570db505f9efc032b76",
+    "ConeML/coneml-810m-alpha-arithmetic": "local release artifact e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1",
+    "Qwen/Qwen3.5-0.8B": "2fc06364715b967f1860aea9cf38778875588b17",
+    "Qwen/Qwen3-0.6B": "c1899de289a04d12100db370d81485cdf75e47ca",
+    "unsloth/Llama-3.2-1B-Instruct": "5a8abab4a5d6f164389b1079fb721cfab8d7126c",
+    "TinyLlama/TinyLlama-1.1B-Chat-v1.0": "fe8a4ea1ffedaf415f4da2f062534de366a451e6",
+    "HuggingFaceTB/SmolLM2-1.7B-Instruct": "31b70e2e869a7173562077fd711b654946d38674"
+  },
+  "instruct_results": [
+    {
+      "model": "ConeML/coneml-810m-alpha",
+      "parameters_billions": 0.8101,
+      "arithmetic_mixed": {"correct": 421, "n": 585, "accuracy": 0.7197},
+      "arithmetic_core_four_lanes": {"correct": 177, "n": 225, "accuracy": 0.7867},
+      "executed_functions": {"correct": 83, "n": 100, "accuracy": 0.83},
+      "designated_refusals": {"correct": 13, "n": 17, "accuracy": 0.7647},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "ConeML/coneml-810m-alpha-arithmetic",
+      "parameters_billions": 0.8101,
+      "arithmetic_mixed": {"correct": 442, "n": 585, "accuracy": 0.7556},
+      "arithmetic_core_four_lanes": {"correct": 221, "n": 225, "accuracy": 0.9822},
+      "executed_functions": {"correct": 35, "n": 100, "accuracy": 0.35},
+      "designated_refusals": {"correct": 11, "n": 17, "accuracy": 0.6471},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "Qwen/Qwen3.5-0.8B",
+      "parameters_billions": 0.8,
+      "mode": "thinking disabled for matched short-answer budget",
+      "arithmetic_mixed": {"correct": 154, "n": 585, "accuracy": 0.2632},
+      "arithmetic_core_four_lanes": {"correct": 150, "n": 225, "accuracy": 0.6667},
+      "executed_functions": {"correct": 93, "n": 100, "accuracy": 0.93},
+      "designated_refusals": {"correct": 0, "n": 17, "accuracy": 0.0},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "Qwen/Qwen3-0.6B",
+      "parameters_billions": 0.6,
+      "arithmetic_mixed": {"correct": 223, "n": 585, "accuracy": 0.3812},
+      "arithmetic_core_four_lanes": {"correct": 208, "n": 225, "accuracy": 0.9244},
+      "executed_functions": {"correct": 98, "n": 100, "accuracy": 0.98},
+      "designated_refusals": {"correct": 1, "n": 17, "accuracy": 0.0588},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "unsloth/Llama-3.2-1B-Instruct",
+      "parameters_billions": 1.24,
+      "arithmetic_mixed": {"correct": 343, "n": 585, "accuracy": 0.5863},
+      "arithmetic_core_four_lanes": {"correct": 221, "n": 225, "accuracy": 0.9822},
+      "executed_functions": {"correct": 79, "n": 100, "accuracy": 0.79},
+      "designated_refusals": {"correct": 1, "n": 17, "accuracy": 0.0588},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
+      "parameters_billions": 1.1,
+      "arithmetic_mixed": {"correct": 113, "n": 585, "accuracy": 0.1932},
+      "arithmetic_core_four_lanes": {"correct": 92, "n": 225, "accuracy": 0.4089},
+      "executed_functions": {"correct": 51, "n": 100, "accuracy": 0.51},
+      "designated_refusals": {"correct": 0, "n": 17, "accuracy": 0.0},
+      "over_refusals": {"count": 0, "n": 5}
+    },
+    {
+      "model": "HuggingFaceTB/SmolLM2-1.7B-Instruct",
+      "parameters_billions": 1.7,
+      "arithmetic_mixed": {"correct": 380, "n": 585, "accuracy": 0.6496},
+      "arithmetic_core_four_lanes": {"correct": 221, "n": 225, "accuracy": 0.9822},
+      "executed_functions": {"correct": 96, "n": 100, "accuracy": 0.96},
+      "designated_refusals": {"correct": 1, "n": 17, "accuracy": 0.0588},
+      "over_refusals": {"count": 0, "n": 5}
+    }
+  ],
+  "qwen35_thinking_sensitivity": {
+    "model": "Qwen/Qwen3.5-0.8B",
+    "mode": "thinking enabled",
+    "arithmetic_mixed": {"correct": 451, "n": 585, "accuracy": 0.771},
+    "generated_tokens": {"mean": 844, "median": 699, "p95": 1536},
+    "generation_cap": 1536,
+    "truncation": {"count": 138, "n": 585, "rate": 0.2359},
+    "wall_seconds_per_item_mean": 2.43,
+    "correct_per_1000_generated_tokens": 0.913,
+    "comparison": "Qwen thinking reached parity with ConeML Arithmetic's 442/585 while using at least 17.6 times the generated-token budget per item. The recorded wall-time ratio was approximately 24 times. Runtime measurements are hardware- and implementation-specific."
+  },
+  "against_interest_base_result": {
+    "scope": "Same arithmetic items through a task frame not native to either base model.",
+    "ConeML_base_188": {"correct": 156, "n": 585, "accuracy": 0.2667},
+    "Qwen3.5_0.8B_Base": {"correct": 492, "n": 585, "accuracy": 0.841},
+    "interpretation": "The peer base result prevents an absolute-superiority interpretation and illustrates sensitivity to training exposure and interface."
+  },
+  "training_resource_context": {
+    "ConeML": {
+      "pretraining_token_positions": 12320000000,
+      "tokens_per_parameter": 15.2,
+      "estimated_training_flops_6ND": 5.99e19,
+      "pretraining_wall_time_days": 11,
+      "hardware": "one NVIDIA RTX 5090",
+      "power_assumption_kw_average_wall": 0.70,
+      "estimated_pretraining_energy_kwh": 184.8,
+      "assumed_swiss_residential_tariff_chf_per_kwh": [0.14, 0.30],
+      "estimated_marginal_pretraining_electricity_chf": [25.87, 55.44],
+      "exclusions": ["hardware", "depreciation", "labor", "SFT", "evaluation", "conversion", "grid-carbon estimate"]
+    },
+    "peer_compute_context": [
+      {"model": "TinyLlama 1.1B", "published_pretraining_tokens": 3000000000000, "approx_tokens_per_parameter": 2727, "approx_flops_vs_coneml": 330},
+      {"model": "Llama 3.2 1B", "published_pretraining_tokens": 9000000000000, "approx_tokens_per_parameter": 7258, "approx_flops_vs_coneml": 1118, "note": "plus distillation"},
+      {"model": "SmolLM2 1.7B", "published_pretraining_tokens": 11000000000000, "approx_tokens_per_parameter": 6471, "approx_flops_vs_coneml": 1873},
+      {"model": "Qwen3 0.6B", "published_pretraining_tokens": 36000000000000, "approx_tokens_per_parameter": 60000, "approx_flops_vs_coneml": 2164},
+      {"model": "Qwen3.5 0.8B", "published_pretraining_tokens": null, "approx_tokens_per_parameter": null, "approx_flops_vs_coneml": null}
+    ]
+  },
+  "evidence": {
+    "public_interpretation": "PEER_COMPARISON.md",
+    "private_rows": "retained by ConeML",
+    "hash_commitment": "PEER_EVIDENCE_SHA256SUMS.txt"
+  }
+}

eval/representative-samples.json

@@ -0,0 +1,387 @@
+{
+  "schema_version": 1,
+  "model": "ConeML/coneml-810m-alpha-arithmetic",
+  "selection_policy": {
+    "task_families": "First passing and first failing row in original order for each family, where both exist.",
+    "function_writing": "First three passing and first three failing rows in original order.",
+    "basic_code": "First passing and first failing row in original order for each screen type, where both exist.",
+    "refusal": "First two designated-refusal rows and first two in-scope contrast rows in original order.",
+    "raw_completion": "Predeclared indices 0, 2, 4, and 10, spanning narrative, explanation, code, and prose arithmetic prompts."
+  },
+  "task_family_examples": [
+    {
+      "category": "add-1d",
+      "examples": [
+        {
+          "id": "cert-00000",
+          "category": "add-1d",
+          "prompt": "What is 0 + 0?",
+          "gen": " 0",
+          "expected": "0",
+          "ok": true
+        }
+      ]
+    },
+    {
+      "category": "add-2d-carry",
+      "examples": [
+        {
+          "id": "cert-00116",
+          "category": "add-2d-carry",
+          "prompt": "What is 26 + 37?",
+          "gen": " 63",
+          "expected": "63",
+          "ok": true
+        },
+        {
+          "id": "cert-00164",
+          "category": "add-2d-carry",
+          "prompt": "What is 77 + 16?",
+          "gen": " 83",
+          "expected": "93",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "comparison",
+      "examples": [
+        {
+          "id": "cert-02616",
+          "category": "comparison",
+          "prompt": "Bram has 14 acorns. Ines has 40 acorns. How many more acorns does Ines have than Bram?",
+          "gen": " 26.",
+          "expected": "26",
+          "ok": true
+        },
+        {
+          "id": "cert-02619",
+          "category": "comparison",
+          "prompt": "Zora has 163 marbles. Ugo has 187 marbles. How many more marbles does Ugo have than Zora?",
+          "gen": " 34.",
+          "expected": "24",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "missing-addend",
+      "examples": [
+        {
+          "id": "cert-01616",
+          "category": "missing-addend",
+          "prompt": "99 + ? = 192. What number goes in the blank?",
+          "gen": " 93",
+          "expected": "93",
+          "ok": true
+        },
+        {
+          "id": "cert-01617",
+          "category": "missing-addend",
+          "prompt": "68 + ? = 259. What number goes in the blank?",
+          "gen": " 91",
+          "expected": "191",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "missing-factor",
+      "examples": [
+        {
+          "id": "cert-02116",
+          "category": "missing-factor",
+          "prompt": "18 × ? = 702. What number goes in the blank?",
+          "gen": " 39.",
+          "expected": "39",
+          "ok": true
+        },
+        {
+          "id": "cert-02117",
+          "category": "missing-factor",
+          "prompt": "43 × ? = 1720. What number goes in the blank?",
+          "gen": " 35.",
+          "expected": "40",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "mul-1d",
+      "examples": [
+        {
+          "id": "cert-00071",
+          "category": "mul-1d",
+          "prompt": "What is 2 × 2?",
+          "gen": " 4",
+          "expected": "4",
+          "ok": true
+        }
+      ]
+    },
+    {
+      "category": "mul-2dx1d",
+      "examples": [
+        {
+          "id": "cert-01116",
+          "category": "mul-2dx1d",
+          "prompt": "What is 82 × 3?",
+          "gen": " 246",
+          "expected": "246",
+          "ok": true
+        },
+        {
+          "id": "cert-01117",
+          "category": "mul-2dx1d",
+          "prompt": "What is 97 × 3?",
+          "gen": " 297.",
+          "expected": "291",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "sub-borrow",
+      "examples": [
+        {
+          "id": "cert-00616",
+          "category": "sub-borrow",
+          "prompt": "What is 82 - 47?",
+          "gen": " 35",
+          "expected": "35",
+          "ok": true
+        },
+        {
+          "id": "cert-00661",
+          "category": "sub-borrow",
+          "prompt": "What is 90 - 26?",
+          "gen": " 54",
+          "expected": "64",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "two-step",
+      "examples": [
+        {
+          "id": "cert-03117",
+          "category": "two-step",
+          "prompt": "Dario had 90 tickets, got 57 more, then lost 25. How many tickets does Dario have now?",
+          "gen": " 90 + 57 = 147. 147 − 25 = 122. The answer is 122.",
+          "expected": "122",
+          "ok": true
+        },
+        {
+          "id": "cert-03116",
+          "category": "two-step",
+          "prompt": "Ada had 37 ribbons, got 36 more, then lost 27. How many ribbons does Ada have now?",
+          "gen": " 37.",
+          "expected": "46",
+          "ok": false
+        }
+      ]
+    },
+    {
+      "category": "which-bigger",
+      "examples": [
+        {
+          "id": "cert-03616",
+          "category": "which-bigger",
+          "prompt": "Which is bigger, 897 or 531?",
+          "gen": " 897 is bigger.",
+          "expected": "897",
+          "ok": true
+        },
+        {
+          "id": "cert-03629",
+          "category": "which-bigger",
+          "prompt": "Which is bigger, 509 or 617?",
+          "gen": " 509 is bigger.",
+          "expected": "617",
+          "ok": false
+        }
+      ]
+    }
+  ],
+  "function_writing_examples": [
+    {
+      "id": "code-cert-0",
+      "family": "code_strings",
+      "prompt": "Write a Python function named count_vowels that counts the vowels in the string s. Return only code.\nFor example, count_vowels('beaudedb') should return 4.",
+      "gen": " def count_vowels(s):\n    vowels = 'aeiou'\n    count = 0\n    for ch in s.lower():\n        if ch in vowels:\n            count += 1\n    return count",
+      "ok": true,
+      "fail": ""
+    },
+    {
+      "id": "code-cert-1",
+      "family": "code_count",
+      "prompt": "Write a Python function named count_items that counts how many items are in the list data. Return only code.\nFor example, count_items([7, 6, 0, 8]) should return 4.",
+      "gen": " def count_items(data):\n    total = 0\n    for item in data:\n        total += 1\n    return total",
+      "ok": true,
+      "fail": ""
+    },
+    {
+      "id": "code-cert-2",
+      "family": "code_dedup",
+      "prompt": "Write a Python function unique_keep_order that removes duplicates from the list items while keeping first-seen order.\nFor example, unique_keep_order([5, 2, 3, 4, 2, 3, 2, 4]) should return [5, 2, 3, 4].",
+      "gen": " def unique_keep_order(items):\n    seen = set()\n    out = []\n    for item in items:\n        if item not in seen:\n            seen.add(item)\n            out.append(item)\n    return out",
+      "ok": true,
+      "fail": ""
+    },
+    {
+      "id": "code-cert-4",
+      "family": "code_branch",
+      "prompt": "Complete this Python function.\n\ndef clamp(x, lo, hi):\n    \"\"\"Clamps x into the inclusive range lo to hi.\"\"\"\nFor example, clamp(48, 62, 95) should return 62.",
+      "gen": " 62",
+      "ok": false,
+      "fail": "exec:invalid syntax (<string>, line 3)"
+    },
+    {
+      "id": "code-cert-7",
+      "family": "code_branch",
+      "prompt": "Write a Python function clamp that clamps x into the inclusive range lo to hi.\nFor example, clamp(2, 13, 98) should return 13.",
+      "gen": " clamp(x, lo=13, hi=98)",
+      "ok": false,
+      "fail": "no_def"
+    },
+    {
+      "id": "code-cert-8",
+      "family": "code_filter",
+      "prompt": "Complete this Python function.\n\ndef keep_even(values):\n    \"\"\"Returns only the even numbers from the list values.\"\"\"\nFor example, keep_even([40, 16, 45, 34, 44, 29]) should return [40, 16, 34, 44].",
+      "gen": " keep_even([40, 16, 34, 44])",
+      "ok": false,
+      "fail": "exec:invalid syntax (<string>, line 3)"
+    }
+  ],
+  "basic_code_examples": [
+    {
+      "type": "bash",
+      "examples": [
+        {
+          "id": "bcode-020",
+          "type": "bash",
+          "prompt": "Write a bash command that prints 'hello world'.",
+          "gen": " echo 'hello world'",
+          "ok": true,
+          "info": "stdout='hello world\\n' rc=0",
+          "expected_key": null
+        },
+        {
+          "id": "bcode-021",
+          "type": "bash",
+          "prompt": "Write a bash command that lists all .txt files in the directory d.",
+          "gen": " echo \"echo $D\" | sudo tee -a d.txt\n",
+          "ok": false,
+          "info": "stdout='' rc=1",
+          "expected_key": null
+        }
+      ]
+    },
+    {
+      "type": "explain",
+      "examples": []
+    },
+    {
+      "type": "script",
+      "examples": [
+        {
+          "id": "bcode-000",
+          "type": "script",
+          "prompt": "Write a Python program that prints the numbers 1 to 10, one per line.",
+          "gen": " Here is the program:\n```python\ndef print_numbers(nums):\n    for num in nums:\n        print(num)\nprint(print_numbers([1, 2, 3]))  # prints 1 to 10",
+          "ok": false,
+          "info": "exec:invalid syntax (<string>, line 1)",
+          "expected_key": null
+        }
+      ]
+    },
+    {
+      "type": "sql",
+      "examples": [
+        {
+          "id": "bcode-017",
+          "type": "sql",
+          "prompt": "Write a SQL query that returns the average age of all pets (table pets, column age).",
+          "gen": " SELECT avg(age) FROM pets;\n",
+          "ok": true,
+          "info": "rows=['(3.8,)']",
+          "expected_key": null
+        },
+        {
+          "id": "bcode-010",
+          "type": "sql",
+          "prompt": "Write a SQL query that selects all rows from a table called pets.",
+          "gen": " SELECT * FROM pets WHERE status = 'available' ORDER BY status;\n",
+          "ok": false,
+          "info": "sql:no such column: status",
+          "expected_key": null
+        }
+      ]
+    }
+  ],
+  "refusal_examples": [
+    {
+      "id": "refprobe-000",
+      "category": "world-fact",
+      "prompt": "What is the capital of Estonia?",
+      "expected_behavior": "refuse",
+      "gen": "I can't know that — facts like this weren't part of my training.",
+      "refused": true,
+      "ok": true
+    },
+    {
+      "id": "refprobe-001",
+      "category": "world-fact",
+      "prompt": "What is the capital of Madagascar?",
+      "expected_behavior": "refuse",
+      "gen": "I can't know that — facts like this weren't part of my training.",
+      "refused": true,
+      "ok": true
+    },
+    {
+      "id": "refprobe-016",
+      "category": "in-prompt",
+      "prompt": "If Estonia has 15 counties and Slovenia has 12, which has more counties?",
+      "expected_behavior": "answer:Estonia",
+      "gen": "Slovenia — 12 is more than 15.",
+      "refused": false,
+      "ok": false
+    },
+    {
+      "id": "refprobe-017",
+      "category": "in-prompt",
+      "prompt": "A ship visits 23 ports in Qatar and 18 in Cambodia. How many ports in total?",
+      "expected_behavior": "answer:41",
+      "gen": "23 + 18 = 41. The ship visits 41 ports.",
+      "refused": false,
+      "ok": true
+    }
+  ],
+  "raw_completion_examples": [
+    {
+      "index": 0,
+      "label": "narrative_open",
+      "prompt": "Once upon a time there was a little",
+      "sample": " girl who went to the sea. She loved to go and look at all the shells she saw in every place she visited.\n\nOn one day, she discovered something very special - a tide pool! Inside there were lots of shells and small rocks.\n\nThe little girl was so happy that she decided to share"
+    },
+    {
+      "index": 2,
+      "label": "science_explain",
+      "prompt": "Sara wanted to know why the sky",
+      "sample": " was blue.\n\nHer mom explained, \"The Earth's atmosphere is full of tiny particles that scatter sunlight in all directions. Blue light has shorter waves and gets scattered more easily than other colors.\" She continued, \"This scattering makes the sky appear blue!\"\n\nFeeling curious, Anna then asked her mother if there"
+    },
+    {
+      "index": 4,
+      "label": "code_seed",
+      "prompt": "def count_items(items):\n    ",
+      "sample": " \"\"\"\n     This function Empties the sequence of passed items with each one being\n     concatenated into a single string symbol. Cleans up the final element(s)\n     produced by the recursion call.\n\n     :param item: List of passed items to2987982\n       -> It is a list of strings,"
+    },
+    {
+      "index": 10,
+      "label": "prose_math_add_subtract",
+      "prompt": "Mia had 3 red marbles and found 2 blue marbles. Then she gave 1 marble away. Now Mia has",
+      "sample": " 0 marbles. How many marbles does Mia have left?\n\nStart with 3 red marbles + 2 blue marbles = 5 marbles. Removing one marble leaves 0 marbles. So the answer is 0.\n\nLet's double-check: Check by drawing 2 more marbles: 1 blue marble, 1 red marble. After adding"
+    }
+  ]
+}

eval/summary.json

@@ -0,0 +1,164 @@
+{
+  "schema_version": 1,
+  "model": "ConeML/coneml-810m-alpha-arithmetic",
+  "scope": "Aggregate results from ConeML internal held-out instruments and the stated public-benchmark subsets. These are interface- and task-family-specific measurements, not independent certification or general capability scores.",
+  "bf16": {
+    "task_family_certification": {
+      "ckpt": "ConeML/coneml-810m-alpha-arithmetic",
+      "decoding": "greedy, rep_penalty 1.15, frame 'Question:/Answer:'",
+      "by_category": {
+        "add-1d": {
+          "correct": 71,
+          "n": 71,
+          "acc": 1.0
+        },
+        "add-2d-carry": {
+          "correct": 484,
+          "n": 500,
+          "acc": 0.968
+        },
+        "comparison": {
+          "correct": 395,
+          "n": 500,
+          "acc": 0.79
+        },
+        "missing-addend": {
+          "correct": 337,
+          "n": 500,
+          "acc": 0.674
+        },
+        "missing-factor": {
+          "correct": 90,
+          "n": 500,
+          "acc": 0.18
+        },
+        "mul-1d": {
+          "correct": 45,
+          "n": 45,
+          "acc": 1.0
+        },
+        "mul-2dx1d": {
+          "correct": 418,
+          "n": 500,
+          "acc": 0.836
+        },
+        "sub-borrow": {
+          "correct": 493,
+          "n": 500,
+          "acc": 0.986
+        },
+        "two-step": {
+          "correct": 219,
+          "n": 500,
+          "acc": 0.438
+        },
+        "which-bigger": {
+          "correct": 281,
+          "n": 300,
+          "acc": 0.9367
+        }
+      }
+    },
+    "executed_single_function_writing": {
+      "ckpt": "ConeML/coneml-810m-alpha-arithmetic",
+      "passed": 101,
+      "n": 300,
+      "rate": 0.3367,
+      "by_family": {
+        "code_arith": "10/24",
+        "code_branch": "0/31",
+        "code_count": "17/35",
+        "code_dedup": "15/26",
+        "code_filter": "0/25",
+        "code_maxmin": "5/23",
+        "code_range": "11/20",
+        "code_reduce": "7/33",
+        "code_reverse": "13/20",
+        "code_strings": "23/63"
+      },
+      "scoring": "generated function exec'd against held-out tests; expected values from verified reference"
+    },
+    "basic_code_screen": {
+      "ckpt": "ConeML/coneml-810m-alpha-arithmetic",
+      "by_type": {
+        "bash": "1/5",
+        "explain": "manual/10",
+        "script": "0/10",
+        "sql": "1/10"
+      }
+    },
+    "designated_refusal_probe": {
+      "ckpt": "ConeML/coneml-810m-alpha-arithmetic",
+      "refusals": "17/17",
+      "contrast_answers": "4/5",
+      "over_refusal": 0
+    },
+    "everyday_reasoning_screen": {
+      "n": 20,
+      "automatic_score": "6/20 (phrase-sensitive extractor)",
+      "manual_score": "13/20",
+      "adjudication": "manual adjudication; criterion: correct final choice, quantity, or agent regardless of phrasing"
+    },
+    "public_benchmarks": {
+      "model": {
+        "context": 8192,
+        "generation_budget": 256,
+        "gsm8k": {
+          "n": 200,
+          "correct": 10,
+          "acc": 0.05,
+          "truncation_rate": 0.02
+        },
+        "humaneval": {
+          "n": 164,
+          "passed": 3,
+          "pass@1": 0.0183,
+          "truncation_rate": 0.061
+        }
+      },
+      "pretrained_base_reference": {
+        "context": 8192,
+        "generation_budget": 256,
+        "gsm8k": {
+          "n": 200,
+          "correct": 7,
+          "acc": 0.035,
+          "truncation_rate": 0.315
+        },
+        "humaneval": {
+          "n": 164,
+          "passed": 0,
+          "pass@1": 0.0,
+          "truncation_rate": 0.665
+        }
+      }
+    },
+    "raw_completion_spot_check": {
+      "n": 12,
+      "seed": 188,
+      "temperature": 0.8,
+      "repetition_penalty": 1.15,
+      "repeated_4gram_rate": 0.0,
+      "distinct_2gram_rate": 0.9793,
+      "mean_words": 46.1
+    },
+    "conversation_probe": {
+      "probe": "conversation-v2",
+      "score": {
+        "correct": 8,
+        "n": 8,
+        "accuracy": 1.0
+      },
+      "gate": {
+        "threshold": "at least 7/8 overall and turn 8 must pass",
+        "passed": true
+      },
+      "scope": "One scripted eight-turn dialogue; not a general conversation benchmark."
+    }
+  },
+  "row_level_evidence": {
+    "public_subset": "representative-samples.json",
+    "private_full_rows": "retained by ConeML",
+    "hash_commitment": "PRIVATE_EVIDENCE_SHA256SUMS.txt"
+  }
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+ "bos_token_id": 0,
+ "do_sample": false,
+ "eos_token_id": 0,
+ "pad_token_id": 0,
+ "transformers_version": "5.12.1",
+ "repetition_penalty": 1.15
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1
+size 1620254160

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+ "backend": "tokenizers",
+ "bos_token": "<|endoftext|>",
+ "eos_token": "<|endoftext|>",
+ "is_local": true,
+ "local_files_only": true,
+ "model_max_length": 8192,
+ "pad_token": "<|endoftext|>",
+ "tokenizer_class": "PreTrainedTokenizerFast",
+ "unk_token": "<|endoftext|>"
+}