|
|
---
|
|
|
library_name: transformers
|
|
|
tags:
|
|
|
- generated_from_trainer
|
|
|
- text-generation
|
|
|
- transformers
|
|
|
- meta-math
|
|
|
- qwen2
|
|
|
- symbolic-ai
|
|
|
- symbioticlm
|
|
|
license: afl-3.0
|
|
|
datasets:
|
|
|
- meta-math/MetaMathQA
|
|
|
- open-thoughts/OpenThoughts2-1M
|
|
|
language:
|
|
|
- zho
|
|
|
- eng
|
|
|
- fra
|
|
|
- spa
|
|
|
- por
|
|
|
- deu
|
|
|
- ita
|
|
|
- rus
|
|
|
- jpn
|
|
|
- kor
|
|
|
- vie
|
|
|
- tha
|
|
|
- ara
|
|
|
base_model:
|
|
|
- Qwen/Qwen2.5-0.5B
|
|
|
pipeline_tag: text-generation
|
|
|
metrics:
|
|
|
- accuracy
|
|
|
model-index:
|
|
|
- name: SymLM
|
|
|
results: []
|
|
|
---
|
|
|
|
|
|
# 🧠 SymLM
|
|
|
|
|
|
**SymbioticLM** is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (`Qwen2ForCausalLM`) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning.
|
|
|
|
|
|
---
|
|
|
|
|
|
## 📐 Model Description
|
|
|
|
|
|
The architecture fuses neural token-level generation with symbolic introspection and reasoning:
|
|
|
|
|
|
- **Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)**
|
|
|
Enables structured long-term memory and spiral-context encoding across tokens.
|
|
|
|
|
|
- **Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
|
|
|
Coordinates symbolic-neural agents via gated attention and adaptive response layers.
|
|
|
|
|
|
- **QwenExoCortex**
|
|
|
Projects contextual hidden states from the Qwen model into a symbolic fusion space for reasoning and memory replay.
|
|
|
|
|
|
- **Symbolic processors**
|
|
|
Includes:
|
|
|
- `ThoughtDynamicsLNN`
|
|
|
- `Liquid / Crystalline Processors`
|
|
|
- `Graph Reasoning with DNAConv`
|
|
|
- A rolling `ThoughtMemory`
|
|
|
|
|
|
This enables real-time fusion of symbolic thinking, token generation, and reasoning-aware language modeling.
|
|
|
|
|
|
---
|
|
|
|
|
|
## 🎯 Intended Uses & Limitations
|
|
|
|
|
|
### ✅ Intended Uses
|
|
|
|
|
|
- **Mathematical reasoning and proof generation**
|
|
|
Fine-tuned on *MetaMathQA*, optimized for symbolic Q&A, equation logic, and structured inference.
|
|
|
|
|
|
- **Symbolic-cognitive AI research**
|
|
|
Useful for studying attention modulation, memory replay, and neural-symbolic interface dynamics.
|
|
|
|
|
|
- **Low-resource adaptation**
|
|
|
Modular memory and projection design enables meaningful performance even with smaller datasets.
|
|
|
|
|
|
- **Building adaptive cognition systems**
|
|
|
Can serve as a symbolic kernel for reflective AI agents and knowledge evolution pipelines.
|
|
|
|
|
|
---
|
|
|
|
|
|
### ⚠️ Limitations
|
|
|
|
|
|
- **Limited training scale**
|
|
|
Trained on 25,000 MetaMathQA examples. Effective for symbolic form, but not yet broad generalization.
|
|
|
|
|
|
- **No RLHF or alignment**
|
|
|
Outputs are not tuned for safety or instruction alignment and may hallucinate.
|
|
|
|
|
|
- **Fluency ≠ correctness**
|
|
|
Symbolic fluency does not imply mathematically valid proofs. Verification is recommended.
|
|
|
|
|
|
- **Not optimized for open-domain generation**
|
|
|
This model prioritizes logic and structure over conversational depth.
|
|
|
|
|
|
---
|
|
|
|
|
|
## ⚙️ Training Procedure
|
|
|
|
|
|
This checkpoint is currently in experimental phase.
|
|
|
|
|
|
### 🧪 Training Hyperparameters
|
|
|
|
|
|
- **learning_rate**: `3e-5`
|
|
|
- **train_batch_size**: `16`
|
|
|
- **eval_batch_size**: `16`
|
|
|
- **gradient_accumulation_steps**: `64`
|
|
|
- **total_train_batch_size**: `1024`
|
|
|
- **optimizer**: `AdamW`, betas=(0.9, 0.999), epsilon=1e-08
|
|
|
- **lr_scheduler_type**: `cosine`
|
|
|
- **warmup_steps**: `500`
|
|
|
- **num_epochs**: `3`
|
|
|
- **mixed_precision_training**: `Native AMP`
|
|
|
|
|
|
---
|
|
|
|
|
|
## 🧱 Framework Versions
|
|
|
|
|
|
- 🤗 Transformers: `4.51.3`
|
|
|
- 🧠 PyTorch: `2.7.0+cu126`
|
|
|
- 📚 Datasets: `3.5.0`
|
|
|
- 🔤 Tokenizers: `0.21.1`
|
|
|
|
|
|
---
|
|
|
|
|
|
## 📚 Research Foundations
|
|
|
|
|
|
SymbioticLM builds upon a cohesive theoretical framework for dynamic reasoning and neuro-symbolic learning:
|
|
|
|
|
|
### 🔁 Multi-Agent Symbiosis and Dynamic Thought
|
|
|
|
|
|
**Rapid Adaptation via Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)**
|
|
|
> A framework where symbolic and neural agents dynamically adapt via gated feedback, memory modulation, and agent-based specialization.
|
|
|
|
|
|
**Focus**: Multi-agent control, reflective learning, contextual responsiveness
|
|
|
|
|
|
---
|
|
|
|
|
|
### 🧬 Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)
|
|
|
|
|
|
> A memory structure inspired by biological helices, enabling thought persistence through spiral-layered contextual encodings across time.
|
|
|
|
|
|
**Focus**: Long-term token evolution, normalized replay, thought continuity
|
|
|
|
|
|
---
|
|
|
|
|
|
### 🧠 Integrating DTE-HDM + M.A.S.R.M for Adaptive AI
|
|
|
|
|
|
> Combines symbolic evolution and multi-agent adaptation to construct an LLM that reflects, adapts, and deepens reasoning through internal dynamics.
|
|
|
|
|
|
**Result**: A system that *learns faster*, *adapts deeper*, and *thinks symbolically*
|
|
|
|
|
|
---
|
|
|
|
|
|
### 📐 Theoretical Underpinning
|
|
|
|
|
|
**The Analytic Foundations Theorem (AFT)**
|
|
|
> A rigorous, measure-theoretic replacement for classical calculus: replaces pointwise derivatives with discrepancy-driven integral convergence across vanishing sets.
|
|
|
|
|
|
**Applies to**:
|
|
|
- Symbolic gradients
|
|
|
- Gradient-free optimization
|
|
|
- Discrete logic approximation in function spaces
|
|
|
|
|
|
---
|
|
|
|
|
|
These form the **mathematical and architectural core** of SymbioticLM, enabling:
|
|
|
|
|
|
- 🧠 *Neuro-symbolic cognitive evolution*
|
|
|
- 🔁 *Multi-agent dynamic feedback coordination*
|
|
|
- 📏 *Formal memory through discrepancy-based logic*
|
|
|
|
|
|
--- |
