lbourdois commited on
Commit
3dbcc32
Β·
verified Β·
1 Parent(s): bed8b0e

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +177 -165
README.md CHANGED
@@ -1,166 +1,178 @@
1
- ---
2
- library_name: transformers
3
- tags:
4
- - generated_from_trainer
5
- - text-generation
6
- - transformers
7
- - meta-math
8
- - qwen2
9
- - symbolic-ai
10
- - symbioticlm
11
- model-index:
12
- - name: SymLM
13
- results: []
14
- license: afl-3.0
15
- datasets:
16
- - meta-math/MetaMathQA
17
- - open-thoughts/OpenThoughts2-1M
18
- language:
19
- - en
20
- base_model:
21
- - Qwen/Qwen2.5-0.5B
22
- pipeline_tag: text-generation
23
- metrics:
24
- - accuracy
25
- ---
26
-
27
- # 🧠 SymLM
28
-
29
- **SymbioticLM** is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (`Qwen2ForCausalLM`) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning.
30
-
31
- ---
32
-
33
- ## πŸ“ Model Description
34
-
35
- The architecture fuses neural token-level generation with symbolic introspection and reasoning:
36
-
37
- - **Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)**
38
- Enables structured long-term memory and spiral-context encoding across tokens.
39
-
40
- - **Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
41
- Coordinates symbolic-neural agents via gated attention and adaptive response layers.
42
-
43
- - **QwenExoCortex**
44
- Projects contextual hidden states from the Qwen model into a symbolic fusion space for reasoning and memory replay.
45
-
46
- - **Symbolic processors**
47
- Includes:
48
- - `ThoughtDynamicsLNN`
49
- - `Liquid / Crystalline Processors`
50
- - `Graph Reasoning with DNAConv`
51
- - A rolling `ThoughtMemory`
52
-
53
- This enables real-time fusion of symbolic thinking, token generation, and reasoning-aware language modeling.
54
-
55
- ---
56
-
57
- ## 🎯 Intended Uses & Limitations
58
-
59
- ### βœ… Intended Uses
60
-
61
- - **Mathematical reasoning and proof generation**
62
- Fine-tuned on *MetaMathQA*, optimized for symbolic Q&A, equation logic, and structured inference.
63
-
64
- - **Symbolic-cognitive AI research**
65
- Useful for studying attention modulation, memory replay, and neural-symbolic interface dynamics.
66
-
67
- - **Low-resource adaptation**
68
- Modular memory and projection design enables meaningful performance even with smaller datasets.
69
-
70
- - **Building adaptive cognition systems**
71
- Can serve as a symbolic kernel for reflective AI agents and knowledge evolution pipelines.
72
-
73
- ---
74
-
75
- ### ⚠️ Limitations
76
-
77
- - **Limited training scale**
78
- Trained on 25,000 MetaMathQA examples. Effective for symbolic form, but not yet broad generalization.
79
-
80
- - **No RLHF or alignment**
81
- Outputs are not tuned for safety or instruction alignment and may hallucinate.
82
-
83
- - **Fluency β‰  correctness**
84
- Symbolic fluency does not imply mathematically valid proofs. Verification is recommended.
85
-
86
- - **Not optimized for open-domain generation**
87
- This model prioritizes logic and structure over conversational depth.
88
-
89
- ---
90
-
91
- ## βš™οΈ Training Procedure
92
-
93
- This checkpoint is currently in experimental phase.
94
-
95
- ### πŸ§ͺ Training Hyperparameters
96
-
97
- - **learning_rate**: `3e-5`
98
- - **train_batch_size**: `16`
99
- - **eval_batch_size**: `16`
100
- - **gradient_accumulation_steps**: `64`
101
- - **total_train_batch_size**: `1024`
102
- - **optimizer**: `AdamW`, betas=(0.9, 0.999), epsilon=1e-08
103
- - **lr_scheduler_type**: `cosine`
104
- - **warmup_steps**: `500`
105
- - **num_epochs**: `3`
106
- - **mixed_precision_training**: `Native AMP`
107
-
108
- ---
109
-
110
- ## 🧱 Framework Versions
111
-
112
- - πŸ€— Transformers: `4.51.3`
113
- - 🧠 PyTorch: `2.7.0+cu126`
114
- - πŸ“š Datasets: `3.5.0`
115
- - πŸ”€ Tokenizers: `0.21.1`
116
-
117
- ---
118
-
119
- ## πŸ“š Research Foundations
120
-
121
- SymbioticLM builds upon a cohesive theoretical framework for dynamic reasoning and neuro-symbolic learning:
122
-
123
- ### πŸ” Multi-Agent Symbiosis and Dynamic Thought
124
-
125
- **Rapid Adaptation via Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
126
- > A framework where symbolic and neural agents dynamically adapt via gated feedback, memory modulation, and agent-based specialization.
127
-
128
- **Focus**: Multi-agent control, reflective learning, contextual responsiveness
129
-
130
- ---
131
-
132
- ### 🧬 Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)
133
-
134
- > A memory structure inspired by biological helices, enabling thought persistence through spiral-layered contextual encodings across time.
135
-
136
- **Focus**: Long-term token evolution, normalized replay, thought continuity
137
-
138
- ---
139
-
140
- ### 🧠 Integrating DTE-HDM + M.A.S.R.M for Adaptive AI
141
-
142
- > Combines symbolic evolution and multi-agent adaptation to construct an LLM that reflects, adapts, and deepens reasoning through internal dynamics.
143
-
144
- **Result**: A system that *learns faster*, *adapts deeper*, and *thinks symbolically*
145
-
146
- ---
147
-
148
- ### πŸ“ Theoretical Underpinning
149
-
150
- **The Analytic Foundations Theorem (AFT)**
151
- > A rigorous, measure-theoretic replacement for classical calculus: replaces pointwise derivatives with discrepancy-driven integral convergence across vanishing sets.
152
-
153
- **Applies to**:
154
- - Symbolic gradients
155
- - Gradient-free optimization
156
- - Discrete logic approximation in function spaces
157
-
158
- ---
159
-
160
- These form the **mathematical and architectural core** of SymbioticLM, enabling:
161
-
162
- - 🧠 *Neuro-symbolic cognitive evolution*
163
- - πŸ” *Multi-agent dynamic feedback coordination*
164
- - πŸ“ *Formal memory through discrepancy-based logic*
165
-
 
 
 
 
 
 
 
 
 
 
 
 
166
  ---
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - generated_from_trainer
5
+ - text-generation
6
+ - transformers
7
+ - meta-math
8
+ - qwen2
9
+ - symbolic-ai
10
+ - symbioticlm
11
+ license: afl-3.0
12
+ datasets:
13
+ - meta-math/MetaMathQA
14
+ - open-thoughts/OpenThoughts2-1M
15
+ language:
16
+ - zho
17
+ - eng
18
+ - fra
19
+ - spa
20
+ - por
21
+ - deu
22
+ - ita
23
+ - rus
24
+ - jpn
25
+ - kor
26
+ - vie
27
+ - tha
28
+ - ara
29
+ base_model:
30
+ - Qwen/Qwen2.5-0.5B
31
+ pipeline_tag: text-generation
32
+ metrics:
33
+ - accuracy
34
+ model-index:
35
+ - name: SymLM
36
+ results: []
37
+ ---
38
+
39
+ # 🧠 SymLM
40
+
41
+ **SymbioticLM** is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (`Qwen2ForCausalLM`) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning.
42
+
43
+ ---
44
+
45
+ ## πŸ“ Model Description
46
+
47
+ The architecture fuses neural token-level generation with symbolic introspection and reasoning:
48
+
49
+ - **Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)**
50
+ Enables structured long-term memory and spiral-context encoding across tokens.
51
+
52
+ - **Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
53
+ Coordinates symbolic-neural agents via gated attention and adaptive response layers.
54
+
55
+ - **QwenExoCortex**
56
+ Projects contextual hidden states from the Qwen model into a symbolic fusion space for reasoning and memory replay.
57
+
58
+ - **Symbolic processors**
59
+ Includes:
60
+ - `ThoughtDynamicsLNN`
61
+ - `Liquid / Crystalline Processors`
62
+ - `Graph Reasoning with DNAConv`
63
+ - A rolling `ThoughtMemory`
64
+
65
+ This enables real-time fusion of symbolic thinking, token generation, and reasoning-aware language modeling.
66
+
67
+ ---
68
+
69
+ ## 🎯 Intended Uses & Limitations
70
+
71
+ ### βœ… Intended Uses
72
+
73
+ - **Mathematical reasoning and proof generation**
74
+ Fine-tuned on *MetaMathQA*, optimized for symbolic Q&A, equation logic, and structured inference.
75
+
76
+ - **Symbolic-cognitive AI research**
77
+ Useful for studying attention modulation, memory replay, and neural-symbolic interface dynamics.
78
+
79
+ - **Low-resource adaptation**
80
+ Modular memory and projection design enables meaningful performance even with smaller datasets.
81
+
82
+ - **Building adaptive cognition systems**
83
+ Can serve as a symbolic kernel for reflective AI agents and knowledge evolution pipelines.
84
+
85
+ ---
86
+
87
+ ### ⚠️ Limitations
88
+
89
+ - **Limited training scale**
90
+ Trained on 25,000 MetaMathQA examples. Effective for symbolic form, but not yet broad generalization.
91
+
92
+ - **No RLHF or alignment**
93
+ Outputs are not tuned for safety or instruction alignment and may hallucinate.
94
+
95
+ - **Fluency β‰  correctness**
96
+ Symbolic fluency does not imply mathematically valid proofs. Verification is recommended.
97
+
98
+ - **Not optimized for open-domain generation**
99
+ This model prioritizes logic and structure over conversational depth.
100
+
101
+ ---
102
+
103
+ ## βš™οΈ Training Procedure
104
+
105
+ This checkpoint is currently in experimental phase.
106
+
107
+ ### πŸ§ͺ Training Hyperparameters
108
+
109
+ - **learning_rate**: `3e-5`
110
+ - **train_batch_size**: `16`
111
+ - **eval_batch_size**: `16`
112
+ - **gradient_accumulation_steps**: `64`
113
+ - **total_train_batch_size**: `1024`
114
+ - **optimizer**: `AdamW`, betas=(0.9, 0.999), epsilon=1e-08
115
+ - **lr_scheduler_type**: `cosine`
116
+ - **warmup_steps**: `500`
117
+ - **num_epochs**: `3`
118
+ - **mixed_precision_training**: `Native AMP`
119
+
120
+ ---
121
+
122
+ ## 🧱 Framework Versions
123
+
124
+ - πŸ€— Transformers: `4.51.3`
125
+ - 🧠 PyTorch: `2.7.0+cu126`
126
+ - πŸ“š Datasets: `3.5.0`
127
+ - πŸ”€ Tokenizers: `0.21.1`
128
+
129
+ ---
130
+
131
+ ## πŸ“š Research Foundations
132
+
133
+ SymbioticLM builds upon a cohesive theoretical framework for dynamic reasoning and neuro-symbolic learning:
134
+
135
+ ### πŸ” Multi-Agent Symbiosis and Dynamic Thought
136
+
137
+ **Rapid Adaptation via Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
138
+ > A framework where symbolic and neural agents dynamically adapt via gated feedback, memory modulation, and agent-based specialization.
139
+
140
+ **Focus**: Multi-agent control, reflective learning, contextual responsiveness
141
+
142
+ ---
143
+
144
+ ### 🧬 Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)
145
+
146
+ > A memory structure inspired by biological helices, enabling thought persistence through spiral-layered contextual encodings across time.
147
+
148
+ **Focus**: Long-term token evolution, normalized replay, thought continuity
149
+
150
+ ---
151
+
152
+ ### 🧠 Integrating DTE-HDM + M.A.S.R.M for Adaptive AI
153
+
154
+ > Combines symbolic evolution and multi-agent adaptation to construct an LLM that reflects, adapts, and deepens reasoning through internal dynamics.
155
+
156
+ **Result**: A system that *learns faster*, *adapts deeper*, and *thinks symbolically*
157
+
158
+ ---
159
+
160
+ ### πŸ“ Theoretical Underpinning
161
+
162
+ **The Analytic Foundations Theorem (AFT)**
163
+ > A rigorous, measure-theoretic replacement for classical calculus: replaces pointwise derivatives with discrepancy-driven integral convergence across vanishing sets.
164
+
165
+ **Applies to**:
166
+ - Symbolic gradients
167
+ - Gradient-free optimization
168
+ - Discrete logic approximation in function spaces
169
+
170
+ ---
171
+
172
+ These form the **mathematical and architectural core** of SymbioticLM, enabling:
173
+
174
+ - 🧠 *Neuro-symbolic cognitive evolution*
175
+ - πŸ” *Multi-agent dynamic feedback coordination*
176
+ - πŸ“ *Formal memory through discrepancy-based logic*
177
+
178
  ---