omesbah commited on
Commit
258dd6d
·
1 Parent(s): 8797abf

feat: Introduce `equilib` package with RLHF steering and Sperner dataset generation examples, and add project license.

Browse files
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2024 Oussama Mesbah
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md CHANGED
@@ -1,5 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # Topo-Align (Equilib)
2
 
 
 
3
  **Gradient-Free Multi-Objective Alignment for Large Language Models using Topological Fixed-Point Theory**
4
 
5
  Topo-Align is a specialized library designed for the alignment of large language models (LLMs) across multiple, often conflicting, objectives (e.g., safety, helpfulness, and creative expression). Utilizing the principles of Combinatorial Topology, specifically Sperner's Lemma, Topo-Align treats the alignment process as a mathematical fixed-point problem rather than a traditional optimization task.
@@ -18,9 +40,9 @@ In contrast, Topo-Align provides:
18
 
19
  ## Comparative Analysis
20
 
21
- | Feature | Linear Scalarization | Grid/Bayesian Search | Topo-Align (Equilib) |
22
- | :--------------------------- | :------------------------------- | :-------------------------- | :------------------------------------------- |
23
- | **Primary Logic** | Weighted Sum Optimization | Stochastic Sampling | Sperner Fixed-Point Walk |
24
  | **Scaling Complexity** | O(1) (Static) | O(k^N) (Exponential) | **O(N) Memory (Linear Scalability)** |
25
  | **Reliability** | Susceptible to gradient collapse | High risk of missing optima | **Guaranteed convergence** |
26
  | **Diagnostics** | No inherent conflict metrics | Visual inspection only | **Topological Frustration Score** |
@@ -104,8 +126,8 @@ optimal_weights = solver.solve_with_surrogate(max_iterations=15)
104
 
105
  ## Core Components
106
 
107
- | Component | Module | Description |
108
- | :----------------------------- | :-------------------------- | :------------------------------------------------------------------ |
109
  | **N-Dimensional Solver** | `ndim_topo_align.py` | The primary engine implementing implicit Freudenthal triangulation. |
110
  | **Sperner Trainer** | `sperner_trainer.py` | Integration layer for Hugging Face Transformers and PEFT. |
111
  | **Analytics Suite** | `analytics.py` | Diagnostic tools including the Topological Frustration Score. |
@@ -121,3 +143,17 @@ optimal_weights = solver.solve_with_surrogate(max_iterations=15)
121
  ## License
122
 
123
  This project is licensed under the MIT License.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Topo-Align (Equilib)
3
+ emoji: 🧬
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: null
7
+ pinned: true
8
+ license: mit
9
+ tags:
10
+ - reinforcement-learning
11
+ - rlhf
12
+ - alignment
13
+ - topology
14
+ - mathematics
15
+ - sperner-lemma
16
+ - human-in-the-loop
17
+ - library
18
+ - custom-implementation
19
+ ---
20
+
21
  # Topo-Align (Equilib)
22
 
23
+ [![Hugging Face Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-green)](https://huggingface.co/datasets/omesbah/sperner-bench)
24
+
25
  **Gradient-Free Multi-Objective Alignment for Large Language Models using Topological Fixed-Point Theory**
26
 
27
  Topo-Align is a specialized library designed for the alignment of large language models (LLMs) across multiple, often conflicting, objectives (e.g., safety, helpfulness, and creative expression). Utilizing the principles of Combinatorial Topology, specifically Sperner's Lemma, Topo-Align treats the alignment process as a mathematical fixed-point problem rather than a traditional optimization task.
 
40
 
41
  ## Comparative Analysis
42
 
43
+ | Feature | Linear Scalarization | Grid/Bayesian Search | Topo-Align (Equilib) |
44
+ | :--------------------- | :------------------------------- | :-------------------------- | :------------------------------------- |
45
+ | **Primary Logic** | Weighted Sum Optimization | Stochastic Sampling | Sperner Fixed-Point Walk |
46
  | **Scaling Complexity** | O(1) (Static) | O(k^N) (Exponential) | **O(N) Memory (Linear Scalability)** |
47
  | **Reliability** | Susceptible to gradient collapse | High risk of missing optima | **Guaranteed convergence** |
48
  | **Diagnostics** | No inherent conflict metrics | Visual inspection only | **Topological Frustration Score** |
 
126
 
127
  ## Core Components
128
 
129
+ | Component | Module | Description |
130
+ | :----------------------- | :------------------------ | :------------------------------------------------------------------ |
131
  | **N-Dimensional Solver** | `ndim_topo_align.py` | The primary engine implementing implicit Freudenthal triangulation. |
132
  | **Sperner Trainer** | `sperner_trainer.py` | Integration layer for Hugging Face Transformers and PEFT. |
133
  | **Analytics Suite** | `analytics.py` | Diagnostic tools including the Topological Frustration Score. |
 
143
  ## License
144
 
145
  This project is licensed under the MIT License.
146
+
147
+ ## 📖 Citation
148
+
149
+ If you find this project useful for your research, please consider citing:
150
+
151
+ ```bibtex
152
+ @misc{mesbah2024topoalign,
153
+ title={Fixpunktsätze und ihre Anwendungen (Fixed Point Theorems and their Applications)},
154
+ author={Mesbah, Oussama},
155
+ howpublished={\url{https://huggingface.co/omesbah/topo-align}},
156
+ year={2024},
157
+ note={Bachelor's Thesis, Ludwig-Maximilians-Universität München}
158
+ }
159
+ ```
equilib/__init__.py CHANGED
@@ -1,6 +1,8 @@
1
  from .topo_align import TopoAlignSolver
2
  from .adaptive_topo_align import AdaptiveTopoAlignSolver
3
  from .surrogate_topo_align import SurrogateTopoAlignSolver, NDimSurrogateTopoAlignSolver
 
 
4
  from .ndim_topo_align import NDimTopoAlignSolver
5
  from .sperner_trainer import SpernerTrainer
6
 
 
1
  from .topo_align import TopoAlignSolver
2
  from .adaptive_topo_align import AdaptiveTopoAlignSolver
3
  from .surrogate_topo_align import SurrogateTopoAlignSolver, NDimSurrogateTopoAlignSolver
4
+ __version__ = "0.1.0"
5
+
6
  from .ndim_topo_align import NDimTopoAlignSolver
7
  from .sperner_trainer import SpernerTrainer
8
 
examples/README.md ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Examples
2
+
3
+ This directory contains standalone scripts demonstrating the capabilities of the Topo-Align (Equilib) library.
4
+
5
+ - `rlhf_steering_demo.py`: Simulates a Multi-Objective RLHF environment (Helpfulness, Safety, Verbosity) and uses the topological solver to find optimal reward mixing weights.
6
+ - `generate_sperner_dataset.py`: The utility used to create the `sperner-bench` dataset, demonstrating how to generate high-dimensional simplex triangulations.
7
+
8
+ ## How to run
9
+ From the repository root:
10
+ ```bash
11
+ python examples/rlhf_steering_demo.py
12
+ ```
generate_sperner_dataset.py → examples/generate_sperner_dataset.py RENAMED
File without changes
{equilib → examples}/rlhf_steering_demo.py RENAMED
@@ -1,6 +1,10 @@
1
- import numpy as np
2
- import sys
3
  import os
 
 
 
 
 
 
4
 
5
  # Create a mock RLHF environment
6
  # Three objectives: [Helpfulness, Safety, Verbosity]
 
 
 
1
  import os
2
+ import sys
3
+
4
+ # Ensure repository root is in path for imports
5
+ sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
6
+
7
+ import numpy as np
8
 
9
  # Create a mock RLHF environment
10
  # Three objectives: [Helpfulness, Safety, Verbosity]
tools/equilib_login.py ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from huggingface_hub import login
2
+ import getpass
3
+ import sys
4
+
5
+ def secure_login():
6
+ print("--- Topo-Align Hugging Face Authentication ---")
7
+ print("Please generate a 'Write' token at: https://huggingface.co/settings/tokens")
8
+ try:
9
+ token = getpass.getpass("Paste your Hugging Face Write Token (input is hidden): ")
10
+ if not token:
11
+ print("Error: No token provided.")
12
+ return
13
+
14
+ login(token=token, add_to_git_credential=True)
15
+ print("\n[SUCCESS] Login successful! Your session is now saved.")
16
+ print("You can now tell the assistant: 'I am logged in'.")
17
+ except Exception as e:
18
+ print(f"\n[ERROR] Login failed: {e}")
19
+ print("Please ensure you have an active internet connection and the token is valid.")
20
+
21
+ if __name__ == "__main__":
22
+ secure_login()