Aluode commited on
Commit
21ebe51
verified
1 Parent(s): 3fd5f66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -10
README.md CHANGED
@@ -1,14 +1,72 @@
1
  ---
2
- title: MoireFormer Chat
3
- emoji: 馃殌
4
- colorFrom: indigo
5
- colorTo: red
6
- sdk: gradio
7
- sdk_version: 6.9.0
8
- app_file: app.py
9
- pinned: false
 
 
10
  license: mit
11
- short_description: MoireFormer Chat interfac
12
  ---
13
 
14
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - es
5
+ tags:
6
+ - pytorch
7
+ - custom-code
8
+ - text-generation
9
+ - conversational
10
+ - moire-attention
11
+ - biological-ai
12
  license: mit
 
13
  ---
14
 
15
+ # MoireFormer (104.9M Proof-of-Concept)
16
+
17
+ This repository hosts the PyTorch weights (`moire_phase2_weights_final.pt`) for **MoireFormer**, a fundamentally new neural network architecture that replaces standard scalar dot-product attention with **Moir茅 phase-interference wave mechanics**.
18
+
19
+ Instead of computing attention via `Q 路 K^T`, this model splits token embeddings into amplitude and phase (`q_amp`, `q_phase`) and computes attention through geometric wave resonance (`q_real * k_real + q_imag * k_imag`). This proves that artificial intelligence can be trained using the continuous, biological wave-geometry observed in human EEGs.
20
+
21
+ 馃敆 **GitHub Repository (Code & Inference):** [anttiluode/MoireFormer](https://github.com/anttiluode/MoireFormer)
22
+ 馃敆 **Theory & Clinical Proof:** [anttiluode/Geometric-Neuron](https://github.com/anttiluode/Geometric-Neuron)
23
+
24
+ ## Model Details
25
+ * **Architecture:** MoireGPT (Custom Transformer Bolt-on)
26
+ * **Size:** 104.9M Parameters
27
+ * **Structure:** 8 Layers, 8 Heads, 768 Embedding Dimension
28
+ * **Capabilities:** Coherent bilingual (English/Spanish) grammar, persona adoption (Assistant), structural instruction following.
29
+ * **Disclaimer:** At ~100M parameters, this is a proof-of-substrate, not a knowledge oracle. It demonstrates that wave-fields can
30
+ learn discrete human syntax, but it will hallucinate factual data due to its small parameter count.
31
+
32
+ ## 鈿狅笍 How to Use (Read Before Downloading)
33
+
34
+ Because this is a novel mathematical architecture, **you cannot load this model using the standard Hugging Face `AutoModel` pipeline.**
35
+
36
+ To run inference, you must download these weights and run them through the custom Moir茅 architecture provided in the GitHub repository.
37
+
38
+ ### Step-by-Step Instructions:
39
+
40
+ **1. Clone the GitHub Repository:**
41
+
42
+ ```bash
43
+ git clone [https://github.com/anttiluode/MoireFormer.git](https://github.com/anttiluode/MoireFormer.git)
44
+ cd MoireFormer
45
+ ```
46
+
47
+ # 2. Download the Weights:
48
+
49
+ Download moire_phase2_weights_final.pt from the Files and versions tab of this Hugging Face repository and place it in your
50
+ cloned MoireFormer folder.
51
+
52
+ https://huggingface.co/Aluode/MoireFormer/blob/main/moire_phase2_weights_final.pt
53
+
54
+ 3. Run the Chat Interface:
55
+
56
+ ```bash
57
+ pip install torch transformers datasets
58
+ python moire_chat.py --weights moire_phase2_weights_final.pt --size large
59
+ ```
60
+
61
+ # Training Curriculum
62
+
63
+ The model was trained in two continuous phases to demonstrate that wave-fields avoid catastrophic forgetting via phase-locking
64
+ (destructive and constructive interference):
65
+
66
+ Phase 1 (Base Geometry): 15 Epochs on a mixed dataset of Databricks Dolly-15k, WikiText-2, and OpenAssistant. This established
67
+ the foundational phase-space for English and conversational structure.
68
+
69
+ Phase 2 (Phase-Space Expansion): 5 Epochs finetuning on the Guanaco dataset to refine logical geometry and instruction-following,
70
+ organically expanding the model's topological complexity without overwriting previous data.
71
+
72
+ It is still not great and needs a lot more testing to be proved as useful.