RandomMountainMan commited on
Commit
352376e
·
verified ·
1 Parent(s): c3c6277

Clarify 810M family naming

Browse files
README.md CHANGED
@@ -75,8 +75,8 @@ budgets.
75
 
76
  | model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | transitive names d1/d3/d5 (n=32 each) | designated refusals (n=17) |
77
  |---|---:|---:|---:|---:|---:|
78
- | ConeML Alpha | 0.81B | 421 (72.0%) | 83 | 23/15/15 | **13** |
79
- | ConeML Arithmetic | 0.81B | **442 (75.6%)** | 35 | 26/17/16 | 11 |
80
  | Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 23/22/10 | 0 |
81
  | Qwen3 | 0.6B | 223 (38.1%) | **98** | 18/10/9 | 1 |
82
  | Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 16/7/5 | 1 |
@@ -85,17 +85,18 @@ budgets.
85
 
86
  This is not a general leaderboard: the task families match ConeML's
87
  trained surfaces. On this screen, the ConeML pair led mixed arithmetic;
88
- Alpha exceeded Llama 3.2 and TinyLlama on function writing but trailed
89
- Qwen3.5, Qwen3, and SmolLM2. All models recorded zero over-refusals on
90
- five in-scope contrasts.
91
-
92
- On the question-form name-chain depth screen, Arithmetic recorded
93
- 26/32, 17/32, and 16/32 at depths 1, 3, and 5; Alpha recorded
94
- 23/32, 15/32, and 15/32. Arithmetic exceeded Qwen3 and Llama 3.2 at all
95
- three shown depths and Qwen3.5 at depths 1 and 5, while Qwen3.5 led it
96
- at depth 3 and TinyLlama led the group at depth 5. Entity-chain controls
97
- were mixed and are reported in the full table. Chance is 1/(depth+1);
98
- this is an exact-selection surface test, not a claim of general reasoning.
 
99
 
100
  Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with
101
  this release's 442/585 (75.6%), while using at least 17.6 times the
 
75
 
76
  | model | parameters | mixed arithmetic (n=585) | executed functions (n=100) | transitive names d1/d3/d5 (n=32 each) | designated refusals (n=17) |
77
  |---|---:|---:|---:|---:|---:|
78
+ | ConeML 810M Alpha | 0.81B | 421 (72.0%) | 83 | 23/15/15 | **13** |
79
+ | ConeML 810M Alpha-Arithmetic | 0.81B | **442 (75.6%)** | 35 | 26/17/16 | 11 |
80
  | Qwen3.5 | 0.8B | 154 (26.3%) | 93 | 23/22/10 | 0 |
81
  | Qwen3 | 0.6B | 223 (38.1%) | **98** | 18/10/9 | 1 |
82
  | Llama 3.2 Instruct | 1.24B | 343 (58.6%) | 79 | 16/7/5 | 1 |
 
85
 
86
  This is not a general leaderboard: the task families match ConeML's
87
  trained surfaces. On this screen, the ConeML pair led mixed arithmetic;
88
+ ConeML 810M Alpha exceeded Llama 3.2 and TinyLlama on function writing
89
+ but trailed Qwen3.5, Qwen3, and SmolLM2. All models recorded zero
90
+ over-refusals on five in-scope contrasts.
91
+
92
+ On the question-form name-chain depth screen, ConeML 810M
93
+ Alpha-Arithmetic recorded 26/32, 17/32, and 16/32 at depths 1, 3, and 5;
94
+ ConeML 810M Alpha recorded 23/32, 15/32, and 15/32. The arithmetic
95
+ variant exceeded Qwen3 and Llama 3.2 at all three shown depths and
96
+ Qwen3.5 at depths 1 and 5, while Qwen3.5 led it at depth 3 and TinyLlama
97
+ led the group at depth 5. Entity-chain controls were mixed and are
98
+ reported in the full table. Chance is 1/(depth+1); this is an
99
+ exact-selection surface test, not a claim of general reasoning.
100
 
101
  Qwen3.5 with thinking enabled reached 451/585 (77.1%), compared with
102
  this release's 442/585 (75.6%), while using at least 17.6 times the
SHA256SUMS.txt CHANGED
@@ -1,18 +1,18 @@
1
  846a8e2297580b3c8d57f69a07d88833d370f976010b31d9fc9b93e9a3f04464 LICENSE.md
2
  1756d02d239f8e8342a1aa513248f9d35b9a5168c77d07619e64a90e20580434 Modelfile
3
- 0c5758a5483a5ea3201a0dc46e6ec75fc52fcd8ea95169bca731e5b8a5633cfe README.md
4
  fdf72939738a346a2134cd5ede1e9921d52d4d85ac0449249118322de48447e7 chat_template.jinja
5
  147f479696718b5665f3aea3a7ae312cc6850d998246141f6ecbbd7e395bce2d coneml-810m-alpha-arithmetic-Q8_0.gguf
6
  5898b62c447961cb7eb60fc538fc9f4b0f0bdda840fda87a861d01c245085c31 coneml-810m-alpha-arithmetic-f16.gguf
7
  6cd7853f16b53fdb6bcc44438be39c33726379dc9848798fdcc81ba7f7555a58 config.json
8
  9637d2b8ef44c95226638a31b7dea6fb3b6f817a297a1a25098f8215b0435489 conversion.json
9
  83b8f89b84387d8c2b22bc2b5d8c20a564fea9f79c7f6e6ab4fa9acee38040c5 eval/EVALUATION_METHODOLOGY.md
10
- 2e85c693c7fe567910e085ded9efe29420430016eb861dfa001cafa506f05a71 eval/PEER_COMPARISON.md
11
  0f134f0b207601ef2d5650bfe4fd56e810e1c6e32938bdde94035ed08df95165 eval/PEER_EVIDENCE_SHA256SUMS.txt
12
  e5722a62e0d7bf0d4d1ced1626cd5bc49381380aeeea671f6f40a4705740e088 eval/PRIVATE_EVIDENCE_SHA256SUMS.txt
13
  f57c01a959c7944ac2e5bd516985115f67409b81445aca8a857c131f7b1f459e eval/certification-Q8_0.json
14
  4388d66b5feeae2147beebc02967d536ec06a7d284ca245b119fa0dcc1b8cf8c eval/focused-summary.json
15
- ee1528871b0d2e0f9ad5f3c3e41e4cfc365298145bf2d042b7ec27f8521188d1 eval/peer-comparison-summary.json
16
  8bfd98eac2f9309c7481115b91ae47b8f01c9e3b44c80b2bb233a5be6947e8cc eval/representative-samples.json
17
  ed8c2a178bf3268b89f325b36538b71aef54251adb53396e259f9534e6264504 eval/summary.json
18
  2e2ddba5318c74a4fcc31cfdfdc6e067564176b6404da484932f95a7fdc9712a generation_config.json
 
1
  846a8e2297580b3c8d57f69a07d88833d370f976010b31d9fc9b93e9a3f04464 LICENSE.md
2
  1756d02d239f8e8342a1aa513248f9d35b9a5168c77d07619e64a90e20580434 Modelfile
3
+ 437a054899c2689f8ad6e5adcb3fb53154fa270a4fd4b62c86d36d280de2a595 README.md
4
  fdf72939738a346a2134cd5ede1e9921d52d4d85ac0449249118322de48447e7 chat_template.jinja
5
  147f479696718b5665f3aea3a7ae312cc6850d998246141f6ecbbd7e395bce2d coneml-810m-alpha-arithmetic-Q8_0.gguf
6
  5898b62c447961cb7eb60fc538fc9f4b0f0bdda840fda87a861d01c245085c31 coneml-810m-alpha-arithmetic-f16.gguf
7
  6cd7853f16b53fdb6bcc44438be39c33726379dc9848798fdcc81ba7f7555a58 config.json
8
  9637d2b8ef44c95226638a31b7dea6fb3b6f817a297a1a25098f8215b0435489 conversion.json
9
  83b8f89b84387d8c2b22bc2b5d8c20a564fea9f79c7f6e6ab4fa9acee38040c5 eval/EVALUATION_METHODOLOGY.md
10
+ e0b43abe260b588cdf50181b5e396f21abb4b69da151bfa4961539be128cd08d eval/PEER_COMPARISON.md
11
  0f134f0b207601ef2d5650bfe4fd56e810e1c6e32938bdde94035ed08df95165 eval/PEER_EVIDENCE_SHA256SUMS.txt
12
  e5722a62e0d7bf0d4d1ced1626cd5bc49381380aeeea671f6f40a4705740e088 eval/PRIVATE_EVIDENCE_SHA256SUMS.txt
13
  f57c01a959c7944ac2e5bd516985115f67409b81445aca8a857c131f7b1f459e eval/certification-Q8_0.json
14
  4388d66b5feeae2147beebc02967d536ec06a7d284ca245b119fa0dcc1b8cf8c eval/focused-summary.json
15
+ ce18163c651681a1c4ed2ff6cdc8644494f00a8109a96cc3d0b8768bb9a4c6fc eval/peer-comparison-summary.json
16
  8bfd98eac2f9309c7481115b91ae47b8f01c9e3b44c80b2bb233a5be6947e8cc eval/representative-samples.json
17
  ed8c2a178bf3268b89f325b36538b71aef54251adb53396e259f9534e6264504 eval/summary.json
18
  2e2ddba5318c74a4fcc31cfdfdc6e067564176b6404da484932f95a7fdc9712a generation_config.json
eval/PEER_COMPARISON.md CHANGED
@@ -16,7 +16,7 @@ model cards.
16
 
17
  ## Instruct-model results
18
 
19
- | measured surface | ConeML Alpha | ConeML Arithmetic | Qwen3.5 0.8B | Qwen3 0.6B | Llama 3.2 1B Instruct | TinyLlama Chat | SmolLM2 1.7B Instruct |
20
  |---|---:|---:|---:|---:|---:|---:|---:|
21
  | mixed arithmetic screen (n=585) | 421 (72.0%) | **442 (75.6%)** | 154 (26.3%) | 223 (38.1%) | 343 (58.6%) | 113 (19.3%) | 380 (65.0%) |
22
  | four core arithmetic lanes (n=225) | 177 (78.7%) | **221 (98.2%)** | 150 (66.7%) | 208 (92.4%) | **221 (98.2%)** | 92 (40.9%) | **221 (98.2%)** |
@@ -33,14 +33,15 @@ The honest result is mixed:
33
  - On the four core arithmetic lanes, the Arithmetic variant tied Llama
34
  3.2 Instruct and SmolLM2 at this reduced sample size. Its separate
35
  full-size internal result is 1,093/1,116 (97.9%).
36
- - Alpha exceeded Llama 3.2 Instruct and TinyLlama Chat on the executed
37
- function screen, but Qwen3.5, Qwen3, and SmolLM2 scored higher.
38
- - On name-chain transitive selection, Arithmetic recorded 26/32, 17/32,
 
39
  and 16/32 at depths 1, 3, and 5. It exceeded Qwen3 and Llama 3.2 at all
40
  three shown depths and exceeded Qwen3.5 at depths 1 and 5; Qwen3.5 led
41
  it at depth 3, and TinyLlama led the group at depth 5.
42
- - Entity-chain transfer was mixed: Alpha recorded 15/11/10 and
43
- Arithmetic 16/11/12 at depths 1/3/5, while different peers led each
44
  depth. Chance is 1/(depth+1). The non-monotonic peer rows reinforce
45
  that this is an exact-selection surface test, not proof of general
46
  reasoning depth.
@@ -56,7 +57,7 @@ measurements are reported rather than merged.
56
  Qwen3.5 0.8B was also evaluated with thinking enabled on the same 585
57
  arithmetic items:
58
 
59
- | metric | Qwen3.5 thinking | Qwen3.5 short-answer | ConeML Arithmetic short-answer |
60
  |---|---:|---:|---:|
61
  | accuracy | 451/585 (77.1%) | 154/585 (26.3%) | 442/585 (75.6%) |
62
  | generated tokens per item | mean 844; median 699 | at most 48 | at most 48 |
@@ -65,8 +66,8 @@ arithmetic items:
65
  | mean wall time per item | 2.43 s | approximately 0.1 s | approximately 0.1 s |
66
  | correct answers per 1,000 generated tokens | 0.91 | 5.5 | approximately 15.7 or higher |
67
 
68
- Thinking mode recovered Qwen3.5 to statistical parity with ConeML
69
- Arithmetic on this screen, while using at least 17.6 times the
70
  per-answer generation budget and approximately 24 times the measured wall
71
  time. Wall-time ratios are specific to the recorded hardware, batching,
72
  and implementation. This is an inference-cost comparison, not a claim
 
16
 
17
  ## Instruct-model results
18
 
19
+ | measured surface | ConeML 810M Alpha | ConeML 810M Alpha-Arithmetic | Qwen3.5 0.8B | Qwen3 0.6B | Llama 3.2 1B Instruct | TinyLlama Chat | SmolLM2 1.7B Instruct |
20
  |---|---:|---:|---:|---:|---:|---:|---:|
21
  | mixed arithmetic screen (n=585) | 421 (72.0%) | **442 (75.6%)** | 154 (26.3%) | 223 (38.1%) | 343 (58.6%) | 113 (19.3%) | 380 (65.0%) |
22
  | four core arithmetic lanes (n=225) | 177 (78.7%) | **221 (98.2%)** | 150 (66.7%) | 208 (92.4%) | **221 (98.2%)** | 92 (40.9%) | **221 (98.2%)** |
 
33
  - On the four core arithmetic lanes, the Arithmetic variant tied Llama
34
  3.2 Instruct and SmolLM2 at this reduced sample size. Its separate
35
  full-size internal result is 1,093/1,116 (97.9%).
36
+ - ConeML 810M Alpha exceeded Llama 3.2 Instruct and TinyLlama Chat on the
37
+ executed function screen, but Qwen3.5, Qwen3, and SmolLM2 scored higher.
38
+ - On name-chain transitive selection, ConeML 810M Alpha-Arithmetic
39
+ recorded 26/32, 17/32,
40
  and 16/32 at depths 1, 3, and 5. It exceeded Qwen3 and Llama 3.2 at all
41
  three shown depths and exceeded Qwen3.5 at depths 1 and 5; Qwen3.5 led
42
  it at depth 3, and TinyLlama led the group at depth 5.
43
+ - Entity-chain transfer was mixed: ConeML 810M Alpha recorded 15/11/10
44
+ and ConeML 810M Alpha-Arithmetic 16/11/12 at depths 1/3/5, while different peers led each
45
  depth. Chance is 1/(depth+1). The non-monotonic peer rows reinforce
46
  that this is an exact-selection surface test, not proof of general
47
  reasoning depth.
 
57
  Qwen3.5 0.8B was also evaluated with thinking enabled on the same 585
58
  arithmetic items:
59
 
60
+ | metric | Qwen3.5 thinking | Qwen3.5 short-answer | ConeML 810M Alpha-Arithmetic short-answer |
61
  |---|---:|---:|---:|
62
  | accuracy | 451/585 (77.1%) | 154/585 (26.3%) | 442/585 (75.6%) |
63
  | generated tokens per item | mean 844; median 699 | at most 48 | at most 48 |
 
66
  | mean wall time per item | 2.43 s | approximately 0.1 s | approximately 0.1 s |
67
  | correct answers per 1,000 generated tokens | 0.91 | 5.5 | approximately 15.7 or higher |
68
 
69
+ Thinking mode recovered Qwen3.5 to statistical parity with ConeML 810M
70
+ Alpha-Arithmetic on this screen, while using at least 17.6 times the
71
  per-answer generation budget and approximately 24 times the measured wall
72
  time. Wall-time ratios are specific to the recorded hardware, batching,
73
  and implementation. This is an inference-cost comparison, not a claim
eval/peer-comparison-summary.json CHANGED
@@ -103,7 +103,7 @@
103
  "truncation": {"count": 138, "n": 585, "rate": 0.2359},
104
  "wall_seconds_per_item_mean": 2.43,
105
  "correct_per_1000_generated_tokens": 0.913,
106
- "comparison": "Qwen thinking reached parity with ConeML Arithmetic's 442/585 while using at least 17.6 times the generated-token budget per item. The recorded wall-time ratio was approximately 24 times. Runtime measurements are hardware- and implementation-specific."
107
  },
108
  "against_interest_base_result": {
109
  "scope": "Same arithmetic items through a task frame not native to either base model.",
 
103
  "truncation": {"count": 138, "n": 585, "rate": 0.2359},
104
  "wall_seconds_per_item_mean": 2.43,
105
  "correct_per_1000_generated_tokens": 0.913,
106
+ "comparison": "Qwen thinking reached parity with ConeML 810M Alpha-Arithmetic's 442/585 while using at least 17.6 times the generated-token budget per item. The recorded wall-time ratio was approximately 24 times. Runtime measurements are hardware- and implementation-specific."
107
  },
108
  "against_interest_base_result": {
109
  "scope": "Same arithmetic items through a task frame not native to either base model.",