ApplePiesFromScratch commited on
Commit
2bf229d
Β·
verified Β·
1 Parent(s): 8a5a2de

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +101 -102
  2. mechanism_base_v1.pth +1 -1
README.md CHANGED
@@ -1,102 +1,101 @@
1
- ---
2
- language: en
3
- license: mit
4
- tags:
5
- - propagation-logic
6
- - mechanism-first
7
- - abstract-reasoning
8
- - derivation-traces
9
- - boundary-conditions
10
- datasets:
11
- - ApplePiesFromScratch/dta-benchmark
12
- metrics:
13
- - dta
14
- ---
15
-
16
- # MechanismBase β€” P / G β†’ Q
17
-
18
- A 10M parameter transformer trained on derivation traces, not natural language.
19
-
20
- ## What this is
21
-
22
- Standard language models learn statistical patterns over text.
23
- This model was trained on the **procedure** P / G β†’ Q β€” explicit derivation
24
- traces showing closure analysis, fixed point detection, cycle structure
25
- identification, and forced boundary condition derivation.
26
-
27
- **The claim:** given any carrier V and gradient family Ξ“, the model can derive
28
- forced boundary conditions β€” what logic system the carrier implies, what
29
- fixed points exist, what cycle structure is forced.
30
-
31
- ## Theory
32
-
33
- Propagation Logic v13 β€” SSRN Abstract ID: 6439258 (James Pugmire)
34
-
35
- The single primitive operator: `P / G β†’ Q`
36
-
37
- A loaded pattern P propagates through gradient field G in context C to
38
- produce updated pattern Q. All of classical logic, fuzzy logic, arithmetic,
39
- calculus, and grammar fall out of different (V, Ξ“) choices.
40
-
41
- ## Model
42
-
43
- - Architecture: Transformer decoder (custom, mechanism-aligned)
44
- - Parameters: 10.5M
45
- - Training tokens: ~1M (derivation traces)
46
- - Training epochs: 5
47
-
48
- ## Benchmark: DTA (Derivation Trace Accuracy)
49
-
50
- The correct benchmark for this model is not BLiMP or MMLU.
51
- It is DTA β€” how accurately does the model predict forced boundary conditions
52
- on novel carriers?
53
-
54
- See: `ApplePiesFromScratch/dta-benchmark`
55
-
56
- | Model | DTA-Overall | DTA-Closure | DTA-FixedPts | DTA-Involution | DTA-Cycle |
57
- |-------|-------------|-------------|--------------|----------------|-----------|
58
- | MechanismBase (10M) | 62.5% | 80.0% | 70.0% | 70.0% | 60.0% |
59
- | Random baseline | 25% | 50% | 25% | 50% | 25% |
60
- | Engine (oracle) | 100% | 100% | 100% | 100% | 100% |
61
-
62
-
63
- ## Usage
64
-
65
- ```python
66
- # The model requires the pl/ library and engine.py from the repo
67
- # Clone: github.com/ApplePiesFromScratch/propagation-logic
68
-
69
- from model import MechanismBase, SmallConfig
70
- from tokenizers import Tokenizer
71
- import torch
72
-
73
- config = SmallConfig()
74
- model = MechanismBase(config)
75
- # Load weights from Hub (see full usage in repo)
76
-
77
- tokenizer = Tokenizer.from_file("mechanism_tokenizer/tokenizer.json")
78
-
79
- # Give the model a partial derivation trace
80
- partial = """DOMAIN: color_domain
81
- CARRIER: ['red', 'green', 'blue']
82
- GRADIENTS: ['complement', 'id']
83
- THETA: 1.0
84
- ---
85
- """
86
-
87
- ids = torch.tensor(tokenizer.encode(partial).ids).unsqueeze(0)
88
- output = model.generate(ids, max_new_tokens=200, temperature=0.3)
89
- print(tokenizer.decode(output[0].tolist()))
90
- ```
91
-
92
- ## Training
93
-
94
- ```
95
- python generate_data.py # generates derivation trace corpus
96
- python tokenizer_train.py # BPE tokenizer on corpus
97
- python train.py # SmallConfig, ~30 min on RTX 4060 Ti
98
- ```
99
-
100
- ## Repository
101
-
102
- GitHub: [ApplePiesFromScratch/propagation-logic](https://github.com/ApplePiesFromScratch/propagation-logic)
 
1
+ ---
2
+ language: en
3
+ license: mit
4
+ tags:
5
+ - propagation-logic
6
+ - mechanism-first
7
+ - abstract-reasoning
8
+ - derivation-traces
9
+ - boundary-conditions
10
+ datasets:
11
+ - ApplePiesFromScratch/dta-benchmark
12
+ metrics:
13
+ - dta
14
+ ---
15
+
16
+ # MechanismBase β€” P / G β†’ Q
17
+
18
+ A 10M parameter transformer trained on derivation traces, not natural language.
19
+
20
+ ## What this is
21
+
22
+ Standard language models learn statistical patterns over text.
23
+ This model was trained on the **procedure** P / G β†’ Q β€” explicit derivation
24
+ traces showing closure analysis, fixed point detection, cycle structure
25
+ identification, and forced boundary condition derivation.
26
+
27
+ **The claim:** given any carrier V and gradient family Ξ“, the model can derive
28
+ forced boundary conditions β€” what logic system the carrier implies, what
29
+ fixed points exist, what cycle structure is forced.
30
+
31
+ ## Theory
32
+
33
+ Propagation Logic v13 β€” SSRN Abstract ID: 6439258 (James Pugmire)
34
+
35
+ The single primitive operator: `P / G β†’ Q`
36
+
37
+ A loaded pattern P propagates through gradient field G in context C to
38
+ produce updated pattern Q. All of classical logic, fuzzy logic, arithmetic,
39
+ calculus, and grammar fall out of different (V, Ξ“) choices.
40
+
41
+ ## Model
42
+
43
+ - Architecture: Transformer decoder (custom, mechanism-aligned)
44
+ - Parameters: 10.5M
45
+ - Training tokens: ~200K (derivation traces)
46
+ - Training epochs: 5
47
+
48
+ ## Benchmark: DTA (Derivation Trace Accuracy)
49
+
50
+ The correct benchmark for this model is not BLiMP or MMLU.
51
+ It is DTA β€” how accurately does the model predict forced boundary conditions
52
+ on novel carriers?
53
+
54
+ See: `ApplePiesFromScratch/dta-benchmark`
55
+
56
+ | Model | DTA-Overall | DTA-Closure | DTA-FixedPts | DTA-Cycle |
57
+ |-------|-------------|-------------|--------------|-----------|
58
+ | MechanismBase (10M) | TBD | TBD | TBD | TBD |
59
+ | Random baseline | 25% | 50% | 25% | 25% |
60
+ | Engine (oracle) | 100% | 100% | 100% | 100% |
61
+
62
+ ## Usage
63
+
64
+ ```python
65
+ # The model requires the pl/ library and engine.py from the repo
66
+ # Clone: github.com/ApplePiesFromScratch/propagation-logic
67
+
68
+ from model import MechanismBase, SmallConfig
69
+ from tokenizers import Tokenizer
70
+ import torch
71
+
72
+ config = SmallConfig()
73
+ model = MechanismBase(config)
74
+ # Load weights from Hub (see full usage in repo)
75
+
76
+ tokenizer = Tokenizer.from_file("mechanism_tokenizer/tokenizer.json")
77
+
78
+ # Give the model a partial derivation trace
79
+ partial = """DOMAIN: color_domain
80
+ CARRIER: ['red', 'green', 'blue']
81
+ GRADIENTS: ['complement', 'id']
82
+ THETA: 1.0
83
+ ---
84
+ """
85
+
86
+ ids = torch.tensor(tokenizer.encode(partial).ids).unsqueeze(0)
87
+ output = model.generate(ids, max_new_tokens=200, temperature=0.3)
88
+ print(tokenizer.decode(output[0].tolist()))
89
+ ```
90
+
91
+ ## Training
92
+
93
+ ```
94
+ python generate_data.py # generates derivation trace corpus
95
+ python tokenizer_train.py # BPE tokenizer on corpus
96
+ python train.py # SmallConfig, ~30 min on RTX 4060 Ti
97
+ ```
98
+
99
+ ## Repository
100
+
101
+ GitHub: [ApplePiesFromScratch/propagation-logic](https://github.com/ApplePiesFromScratch/propagation-logic)
 
mechanism_base_v1.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7212a180679ab3d408d6c23d3904b9b9d3b06c9a34b4287c6e788910738daafb
3
  size 42349189
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e326fabf30f2474aa3e5af5eb89c0e300cefdbaa7c6cefb87243f894c4545dda
3
  size 42349189