---
license: apache-2.0
tags:
- llm
- hyperbolic
- geometry
- adapter
- peft
- research
base_model:
- Qwen/Qwen2.5-7B
pipeline_tag: text-generation
library_name: peft
---

# ManifoldGL – Information‑Geometric Adapter for LLMs

ManifoldGL is a parameter‑efficient adapter that enforces **hyperbolic geometry** on the latent space of large language models. It treats the meaning of a token as a **fiber** over a hyperbolic base manifold (a Poincaré ball), rather than a single vector in flat Euclidean space. Latent states are projected onto the ball, and attentions are computed using geodesic distance. A sheaf‑theoretic consistency loss and natural gradient optimization maintain semantic structure during training.

## Motivation and theoretical background

Modern LLMs embed tokens in a Euclidean vector space. While convenient, Euclidean geometry has limited capacity to represent hierarchical structures: flat space grows polynomially, whereas hierarchical trees expand exponentially. By contrast, **hyperbolic space** grows exponentially and preserves both local and global relationships in a hierarchy【247949143190903†L115-L124】. Hyperbolic embeddings outperform Euclidean ones for lexical entailment, similarity and analogy tasks【247949143190903†L154-L169】. ManifoldGL leverages these properties by modelling the latent space as a fiber bundle over a hyperbolic base: each point in the Poincaré ball encodes a context, and its fiber contains a distribution of semantic components.

## Results on ARC‑AGI benchmark

ManifoldGL fine‑tuned on Qwen2.5‑7B improves task accuracy on the ARC‑AGI benchmark from **12.4 %** to **28.7 %**, a **131.5 % relative improvement**. The model also achieves a **Manifold Faithfulness Rate (MFR) of 94.2 %**, indicating high adherence to the hyperbolic constraints, and maintains a curvature close to the target κ = ‑1 (mean ‑0.98 ± 0.04). Ablation studies show that removing curvature regularization, natural gradients, sheaf consistency or the hyperbolic target significantly reduces accuracy; the Euclidean target ablation causes the largest drop (–10.9 %), highlighting the importance of hyperbolic geometry.

## Files in this repository

This model card accompanies adapter weights trained with ManifoldGL. The files follow the structure of the original repository:

- `adapter_config.json` – configuration for PEFT/LoRA loading
- `pytorch_adapter.bin` – adapter weights
- `README.md` – this model card

## Quick start

```python
from transformers import AutoModelForCausalLM
from peft import PeftModel

# Load the base model (Qwen2.5-7B)
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B")

# Load the ManifoldGL adapter
model = PeftModel.from_pretrained(base_model, "jesusvilela/manifoldgl")

# Now use model.generate(...) to generate text with hyperbolic adapters
```

## Usage

This adapter can be loaded with [PEFT](https://github.com/huggingface/peft) on top of any compatible Qwen2.5‑7B model. During generation, latent states are projected into hyperbolic space and meaning is represented as fibers. We recommend using FP32 precision for maximum stability.

## Citation

If you use ManifoldGL in your work, please cite the accompanying thesis and repository.