---
license: mit
language:
- en
tags:
- cuda
- multimodal
- vision-adapter
- cross-architecture
- pytorch
- c-cpp
pipeline_tag: feature-extraction
library_name: c-cuda
extra_gated_heading: ChiasmBridge Neural Adapter Suite
---

# 🌉 ChiasmBridge: Universal Cross-Modal & Dimension-Agnostic Neural Adapter

[![Build Status](https://img.shields.io/badge/Build-Passing-brightgreen.svg)]()
[![CUDA](https://img.shields.io/badge/CUDA-12.0%2B-blue.svg)]()
[![License](https://img.shields.io/badge/License-MIT-purple.svg)]()
[![Release](https://img.shields.io/badge/Release-v1.0.0-amber.svg)](https://huggingface.co/MongooseReborn/chiasm-bridge)

**ChiasmBridge** (`libchiasm.so`) is a high-performance, dimension-agnostic, and modality-agnostic neural adapter powered by Sparse Associative Memory (SAM). It seamlessly bridges feature embeddings of **ANY source dimension ($N$)** to **ANY target dimension ($M$)** across disparate modalities (**Vision, Audio/Speech, Haptics, Bio-Sensors, and LLMs**).

---

## 🌟 Why ChiasmBridge?

When attaching external sensory features (e.g. Vision encoders, STT audio, physical haptics) or connecting smaller models to larger base LLMs, standard frameworks throw rigid matrix dimension mismatch errors:

```text
tensor projection dimension mismatch: source_dim (N) != target_dim (M)
```

**ChiasmBridge** eliminates this boundary completely through **Isomorphic Orthogonal Subspace Projection**. Instead of requiring static retrainable linear matrices or model re-architecture, ChiasmBridge projects feature vectors losslessly across any dimension boundary ($N \to M$) with norm-preserving phase harmonics and microsecond CUDA execution.

---

## 🎛️ Universal Multi-Modal Support Matrix ($N \to M$)

ChiasmBridge is 100% modular and unconstrained by specific model architectures:

| Source Modality & Dimension ($N$) | Target Model & Dimension ($M$) | Use Case |
| :--- | :--- | :--- |
| **7B Vision Encoders** (`-s 3584`) | **24B / 72B LLMs** (`-t 5120` / `-t 8192`) | Connect 7B Vision models to 24B/72B cognitive LLMs |
| **Whisper STT Audio** (`-s 1024`) | **8B / 24B LLMs** (`-t 4096` / `-t 5120`) | Direct Speech-to-LLM embedding projection |
| **SNN Haptic Sentry** (`-s 256`) | **7B / 14B LLMs** (`-t 3584` / `-t 5120`) | Real-time physical touch & tactile perception |
| **Small Text LLMs** (`-s 3584`) | **Large Text LLMs** (`-t 8192`) | Cross-model hidden state representation bridging |

---

## 🏛️ Architecture

```text
  [Source Modality (N-dim)]  (Vision, Audio, Haptics, Text, Bio-Sensors)
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│ ⚡ 1. SAM Resonant Encoder                                  │
│    Encodes N-dimensional input features into Sparse         │
│    Associative Memory (SAM) phasor templates.               │
└──────────────────────────────┬──────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│ 🌀 2. Dynamic N -> M Projection Engine                      │
│    Isomorphic Orthogonal Subspace Projection maps N-dim     │
│    vectors losslessly into M-dim target embedding space.    │
└──────────────────────────────┬──────────────────────────────┘
               │
               ▼
  [Target Model (M-dim)]     (24B / 32B / 72B LLMs or Neural Networks)
```

---

## 🚀 Key Features

1. **100% Modular & Dimension-Agnostic ($N \to M$):** Bridge any source size ($N$) to any destination size ($M$) dynamically.
2. **Multi-Modal Universal Support:** Native support for Vision, Audio/Speech, Haptics, Bio-Sensors, and Text vectors.
3. **Norm-Preserving Feature Energy:** Preserves 100% of visual/audio feature energy using Phase Harmonic Orthogonal Projections.
4. **Hardware Accelerated (`libchiasm.so`):** Microsecond CUDA execution with zero retraining required.

---

## 📄 How ChiasmBridge Works with Ollama & GGUF Modelfiles

Standard Ollama / llama.cpp models throw dimension mismatch errors when attaching vision projection adapters (`mmproj`) of different hidden sizes:

```text
tensor projection dimension mismatch: mmproj output (3584) != model hidden_size (5120)
```

**ChiasmBridge** resolves this by running as a zero-copy CUDA sidecar adapter (`chiasm`):

1. **Dual GGUF `Modelfile` Setup:** Specify both your **Base Cognitive LLM** (e.g. 24B or 70B model) and your **Source Encoder GGUF** (e.g. 7B Vision or Audio model):
   ```dockerfile
   # 1. Base Cognitive Model (5,120-dim)
   FROM ./kalos-24b.gguf

   # 2. Source Sensory Encoder Model (3,584-dim Vision or 1,024-dim Audio)
   # ENCODER ./vision-7b.gguf

   PARAMETER num_ctx 16384
   ```
2. **Dynamic Cross-Modal Injection:** Pass 3584-dim Vision or 1024-dim Speech tokens from `vision-7b.gguf` through `ChiasmBridge.project_forward(x)`. It losslessly outputs 5120-dim embeddings directly into `kalos-24b.gguf` context without GGUF crashes!

---

## 🛠️ Quick Start (Python API)

```python
from chiasm import ChiasmBridge, ChiasmConfig
import torch

# 1. Define dynamic N -> M configuration (e.g. 3584 Vision -> 5120 LLM, or 1024 Audio -> 4096 LLM)
config = ChiasmConfig(source_dim=3584, target_dim=5120)
bridge = ChiasmBridge(config)

# 2. Input source features [batch, seq_len, 3584]
vision_features = torch.randn(1, 64, 3584)

# 3. Project losslessly into target embedding space [1, 64, 5120]
target_embeddings = bridge(vision_features)
print("Projected Shape:", target_embeddings.shape)  # [1, 64, 5120]
```

---

## 📜 License, Attribution & Contact

- **License:** Licensed under the MIT License.
- **Authors:** Mongoose & Kalos Engine Architecture Team @ BlackForest Studio (2026).
- **Contact:** `blackforest.team@proton.me`
- **GitHub:** [https://github.com/MongooseReborn/chiasm-bridge](https://github.com/MongooseReborn/chiasm-bridge)