ameforge commited on
Commit
d383b89
·
verified ·
1 Parent(s): 98d9e94

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +103 -0
README.md ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ - fr
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - reasoning
10
+ - chain-of-thought
11
+ - structured-generation
12
+ - function-calling
13
+ - agentic
14
+ - edge
15
+ - small-language-model
16
+ base_model: AMFORGE/samg
17
+ model-index:
18
+ - name: SAM-G-Reasoning
19
+ results:
20
+ - task:
21
+ type: multi-step-reasoning
22
+ name: Verified multi-step reasoning (12 families, held-out)
23
+ metrics:
24
+ - type: exact_match
25
+ value: 77.8
26
+ name: Exact match, aggregate (%)
27
+ ---
28
+
29
+ # SAM-G-Reasoning
30
+
31
+ **SAM-G-Reasoning** is a 30.3M-parameter model fine-tuned from
32
+ [SAM-G](https://huggingface.co/AMFORGE/samg) on 196k verified multi-step
33
+ reasoning traces and action plans. It emits explicit step-by-step traces
34
+ (`step 1: ... step 2: ... Answer: X`) for questions and ordered JSON plans for
35
+ multi-step instructions. Built by **AMEFORGE** for procedural reasoning on the
36
+ edge.
37
+
38
+ - **Parameters:** 30.3M · **Footprint:** 121 MB fp32 · **Base:** SAM-G
39
+ - **Fine-tuning:** prompt-masked SFT (loss on the reasoning span only), cosine
40
+ 8e-5, 8k steps
41
+ - **Aggregate exact-match:** 77.8% (held-out, disjoint seed)
42
+
43
+ ## What it is good at — and what it is not
44
+
45
+ The model was stress-tested on twelve verified families. The pattern is clear:
46
+ it excels at **procedural** reasoning (following steps, tracking state, chaining
47
+ actions) and is limited on **calculation-heavy** tasks, as expected at 30M
48
+ parameters.
49
+
50
+ | Family | Exact % | Type |
51
+ |---|---|---|
52
+ | logic (ponens/tollens/chains) | 100 | procedural |
53
+ | plan_chain (multi-step actions) | 100 | procedural |
54
+ | conversion (unit chains) | 100 | procedural |
55
+ | sequence (next term) | 100 | procedural |
56
+ | date_time (clock/calendar) | 92 | procedural |
57
+ | compare (max/min) | 92 | procedural |
58
+ | **state_track (device toggles)** | **83** | **working memory** |
59
+ | parity_digits | 58 | mixed |
60
+ | count_filter | 67 | calculation |
61
+ | sort_list | 50 | calculation |
62
+ | word_problem | 50 | calculation |
63
+ | arith_chain | 42 | calculation |
64
+
65
+ State-tracking at 83% is notable for this scale — it requires maintaining a
66
+ mutable state across several operations. Arithmetic-chain and sorting plateau
67
+ because exact multi-digit calculation is not reliably learnable at 30M; for
68
+ those, delegate to a tool rather than the model.
69
+
70
+ ## Intended use
71
+
72
+ Agentic control loops: decompose an instruction into ordered steps, track
73
+ execution state, and emit structured action plans — entirely offline. Best used
74
+ as the **planning and state-tracking layer** of an agent, with arithmetic and
75
+ data lookups delegated to deterministic tools.
76
+
77
+ ## Usage
78
+
79
+ ```python
80
+ import sentencepiece as spm, torch
81
+ sp = spm.SentencePieceProcessor(); sp.Load("samg_tokenizer.model")
82
+ prompt = "states: lamp=off, fan=on. ops: toggle lamp, turn off fan, toggle lamp. final state of lamp? [CHAT]"
83
+ ids = torch.tensor([sp.EncodeAsIds(prompt)])
84
+ # greedy-decode -> "step 1: ... step 2: ... step 3: ... Answer: off"
85
+ ```
86
+
87
+ ## Limitations
88
+
89
+ - Calculation-heavy families (arithmetic, sorting, word problems) plateau at
90
+ 42–50%; do not use for exact math — delegate to tools.
91
+ - Reasoning traces are synthetic, drawn from the training distribution family
92
+ with a disjoint evaluation seed.
93
+ - Not a general assistant; inherits the base model's knowledge limits.
94
+
95
+ ## Citation
96
+
97
+ ```bibtex
98
+ @misc{samgreasoning2026,
99
+ title = {SAM-G-Reasoning: Procedural Multi-Step Reasoning at 30M Parameters},
100
+ author = {AMEFORGE Lab},
101
+ year = {2026}
102
+ }
103
+ ```