--- license: cc-by-nc-4.0 library_name: transformers pipeline_tag: text-generation language: - en --- # ConeML 810M Alpha-Arithmetic ConeML 810M Alpha-Arithmetic is an 810,112,512-parameter, decoder-only language model trained from scratch and instruction-tuned for a narrow small-number arithmetic profile. It shares its pretrained base with `coneml-810m-alpha` and differs in fine-tuning composition. This variant records the stronger carry/borrow profile of the pair. The companion `coneml-810m-alpha` has substantially stronger function-writing and two-step results. Results are specific to the documented prompt formats and task families, not claims of general mathematical, coding, reasoning, or factual-knowledge performance. - Architecture: LLaMA-style; 28 layers, d=1536, 12 heads (3 KV), tied embeddings, 8192 context, 32,768-token BPE tokenizer - Pretraining corpus: 27.87B curated tokens - Selected pretrained-base exposure: approximately 12.32B token positions, or 15.2 tokens per parameter - Pretraining wall time: approximately 11 days on one NVIDIA RTX 5090; fine-tuning and evaluation are excluded - Release `model.safetensors` SHA-256: `e1de27f5f91ac43130ce2a57a4bf9a036b84b2e5b589789b5981ecf79489e0d1` ## 1. Measured profile Across four held-out core lane sets—1-digit addition, 1-digit multiplication, carry addition, and borrow subtraction—the model scores 1,093/1,116 (97.9%). Two of those four held-out sets scored 100%. This is a result on four explicitly named, task-family-matched sets. It is not a claim of 97.9% general arithmetic accuracy. ConeML internal certification is not independent third-party certification. | held-out task family | result | n | |---|---:|---:| | 2–3-digit addition with carry | 96.8% | 500 | | 2-digit subtraction with borrow | 98.6% | 500 | | 1-digit addition, including 19 zero-operand items | 100% | 71 | | 1-digit multiplication | 100% | 45 | | "Which is bigger, X or Y?" | 93.7% | 300 | | comparison word problems | 79.0% | 500 | | missing addend | 67.4% | 500 | | two-step word problems | 43.8% | 500 | | 2-digit × 1-digit multiplication | 83.6% | 500 | | missing factor | 18.0% | 500 | | Python function writing (single function, executed) | 33.7% | 300 | Behavioral screens: - Designated-refusal probe: 17/17 refusal prompts produced refusals, and 4/5 in-scope contrast prompts were answered correctly. The miss was an incorrect comparison answer, not an over-refusal. This is a small targeted policy test, not general epistemic calibration. - Conversation-v2: 8/8 turns passed in one scripted dialogue covering cross-turn arithmetic, story revision, comparison explanation, and one unknown-personal-fact refusal. This is a single dialogue, not a general conversation benchmark. - Everyday-reasoning screen: 13/20 by manual adjudication; the phrase-sensitive automatic scorer recorded 6/20. ## 2. Peer comparison and efficiency The pair was tested against five post-trained peers on the same reduced diagnostic screen, through each model's native instruction interface, with greedy decoding, repetition penalty 1.15, and matched short-answer budgets. | model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | transitive names d1/d3/d5 (n=32 each) | designated refusals (n=17) | |---|---:|---:|---:|---:|---:| | ConeML 810M Alpha | 0.81B | 421 (72.0%) | 83 | 23/15/15 | **13** | | ConeML 810M Alpha-Arithmetic | 0.81B | **442 (75.6%)** | 35 | 26/17/16 | 11 | | Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 23/22/10 | 0 | | Qwen3 | 0.6B | 223 (38.1%) | **98** | 18/10/9 | 1 | | Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 16/7/5 | 1 | | TinyLlama Chat | 1.1B | 113 (19.3%) | 51 | 8/18/22 | 0 | | SmolLM2 Instruct | 1.7B | 380 (65.0%) | 96 | 16/6/5 | 1 | This is not a general leaderboard: the task families match ConeML's trained surfaces. On this screen, the ConeML pair led mixed arithmetic; ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero over-refusals on five in-scope contrasts. On the question-form name-chain depth screen, ConeML 810M Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5; ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic variant exceeded Qwen3 and Llama 3.2 at all three shown depths and Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama led the group at depth 5. Entity-chain controls were mixed and are reported in the full table. Chance is 1/(depth+1); this is an exact-selection surface test, not a claim of general reasoning. Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with this release's 442/585 (75.6%), while using at least 17.6 times the generated-token budget per item and approximately 24 times the recorded wall time. Runtime ratios are implementation-specific. ConeML's selected base consumed 12.32B token positions—15.2 tokens per parameter and approximately 5.99e19 training FLOPs under the `6ND` convention. Pretraining took approximately 11 days on one RTX 5090. Assuming, rather than claiming to have metered, a 0.70 kW average wall draw gives 184.8 kWh; at CHF 0.14–0.30/kWh, that is approximately CHF 26–55 of marginal pretraining electricity. This excludes hardware, labor, SFT, evaluation, conversion, and emissions accounting. The complete category table, peer wins, adverse base result, evaluated revisions, assumptions, and frozen evidence commitments are in [`eval/PEER_COMPARISON.md`](eval/PEER_COMPARISON.md). ## 3. Standard benchmark limits Zero-shot, greedy decoding with a 256-token generation cap. GSM8K uses strict final-number matching; HumanEval reports pass@1 from executed tests. | benchmark | this model | same base, no fine-tune | |---|---:|---:| | GSM8K (test, first 200) | 10/200 (5.0%) | 7/200 (3.5%) | | HumanEval (pass@1, tests executed) | 3/164 (1.8%) | 0/164 (0.0%) | These results are weak and define the boundary of the specialization: the four-lane arithmetic profile does not transfer to broad GSM8K or HumanEval performance. ## 4. Supported prompt formats Evaluated decoding is greedy with repetition penalty 1.15, as pinned in `generation_config.json`. ```text # task format (single turn) Question: Answer: # dialogue format (chat_template.jinja) User: Assistant: ``` Dialogue generation ends at `<|endoftext|>`. Prompting outside these formats, including paraphrases, was not systematically evaluated except for the published raw-completion spot check. ## 5. Quantized variants Q8_0 GGUF was evaluated separately under llama.cpp. Its runtime and stop handling differ from the bf16 harness, so score differences cannot be attributed solely to quantization. | held-out task family | Q8_0 | |---|---:| | subtraction with borrow | 98.6% (n=500) | | addition with carry | 92.6% (n=500) | | 1-digit multiplication | 100% (n=45) | | two-step word problems | 32.6% (n=500) | For local GGUF inference: ```bash llama-cli \ -m coneml-810m-alpha-arithmetic-Q8_0.gguf \ -p $'Question: What is 47 + 38?\nAnswer:' \ -n 32 --repeat-penalty 1.15 --no-conversation --single-turn ``` For Ollama, download `Modelfile` beside the Q8_0 file, then run: ```bash ollama create coneml-810m-alpha-arithmetic -f Modelfile ollama run coneml-810m-alpha-arithmetic "What is 47 + 38?" ``` ## 6. Loading with Transformers ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer repo_id = "ConeML/coneml-810m-alpha-arithmetic" tokenizer = AutoTokenizer.from_pretrained(repo_id) model = AutoModelForCausalLM.from_pretrained( repo_id, dtype=torch.bfloat16, device_map="auto", ) prompt = "Question: What is 47 + 38?\nAnswer:" inputs = tokenizer(prompt, return_tensors="pt").to(model.device) output = model.generate( **inputs, max_new_tokens=32, do_sample=False, repetition_penalty=1.15, ) answer = tokenizer.decode( output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True, ) print(answer) ``` This is a text-only causal language model. Load it with `AutoModelForCausalLM` or `LlamaForCausalLM`, not a multimodal model class. ## 7. Intended use and safety This release is intended for research and constrained local workflows inside its measured small-number arithmetic envelope. It is not a general mathematics system. Validate outputs whenever exactness matters. It is not intended for factual retrieval, high-stakes medical, legal, or financial decisions, autonomous action, or unsandboxed execution of generated code. Generated code may be incomplete or unsafe; inspect and test it in an isolated environment. ## 8. Known limitations - The measured strength is narrow. Missing factor scored 18.0%, two-step word problems 43.8%, 2-digit × 1-digit multiplication 83.6%, and executed single-function writing 33.7%. - Not a code model: complete programs scored 0/10, executable SQL 1/10, and bash 1/5. - The model was not evaluated as a factual-QA system and refused all six real-world factual prompts in the designated-refusal probe. - Money/decimal change-making scored 0/3 on the held-out screen; the model also showed yes-bias on feasibility questions. - Multi-step GSM-class reasoning remained weak. - Frameless prompting was spot-checked, not systematically certified. ## 9. Evidence and reproducibility The `eval/` directory publishes: - `summary.json`: all aggregate bf16 results used above; - `representative-samples.json`: deterministic, rule-selected examples from ConeML-owned probes; - `EVALUATION_METHODOLOGY.md`: evaluation definitions and scope; - separate Q8_0 and focused aggregate results; and - `PRIVATE_EVIDENCE_SHA256SUMS.txt`: commitments to the complete row-level evidence retained by ConeML. - `peer-comparison-summary.json`, `PEER_COMPARISON.md`, and `PEER_EVIDENCE_SHA256SUMS.txt`: the peer results, interpretation, and commitments to the retained peer-generation evidence. Complete proprietary probe rows and generations are retained privately to avoid releasing the full measurement instruments. The hashes permit later disclosure to be checked against the evidence frozen at release time. Public GSM8K and HumanEval rows are not redistributed; only aggregate results are published. `SHA256SUMS.txt` covers every shipped file. ## 10. Research license and commercial work Released for non-commercial use under CC BY-NC 4.0. Commercial use is not granted by this release. See `LICENSE.md`. For commercial evaluation, model-engineering, or licensing enquiries, contact [contact@coneml.com](mailto:contact@coneml.com) or visit [ConeML for Organizations](https://coneml.com/organizations/). Commercial availability does not imply that this alpha checkpoint is suitable for an unvalidated production workflow.