Scientific model card - Logan Matthew Napolitano

README.md

@@ -1,180 +1,120 @@
-# Lie-Holonomy Transformer (LHT)
+---
+license: cc-by-4.0
+language:
+- en
+library_name: transformers
+tags:
+- llama
+- hermes
+- cognitive-control
+- decode-time-intervention
+- repetition-suppression
+- behavioral-control
+- contrastive-learning
+- interpretability
+- activation-engineering
+pipeline_tag: text-generation
+base_model: NousResearch/Hermes-3-Llama-3.1-8B
+---
 
-A PyTorch implementation of the gauge-theoretic reasoning architecture from "Beyond Holonomy: Lie-Algebraic Symbol Emergence and the Homotopy Type Structure of Neural Reasoning."
+# ARC-8B: Adaptive Repetition Controller
 
-## Core Ideas
+## Decode-Time Behavioral Intervention via Contrastive Fiber Heads-on-Thought
 
-This architecture treats **reasoning as geometry**:
+**Author:** Logan Matthew Napolitano  
+**Institution:** Logan Research  
+**Date:** January 2026  
+**License:** Creative Commons Attribution 4.0 International (CC-BY-4.0)
 
-| Concept | Mathematical Structure | Implementation |
-|---------|----------------------|----------------|
-| Propositions | Manifold M | Embedding space |
-| Inference | Parallel transport | Gauge-covariant attention |
-| Consistency | Holonomy = Identity | Holonomy loss |
-| Symbols | Lie algebra generators | Generator network |
-| Proof equivalence | Homotopy | Layer depth |
+---
 
-## Architecture Overview
+## Abstract
 
-```
-Input tokens
-     │
-     ▼
-┌─────────────────────────────────────┐
-│  Token Embedding (Proposition M)    │
-│  + Position Embedding               │
-│  + Fiber Initialization (gauge)     │
-└─────────────────────────────────────┘
-     │
-     ▼
-┌─────────────────────────────────────┐
-│  LHT Layer (× n_layers)             │
-│  ┌─────────────────────────────┐    │
-│  │ Connection Network A(x)     │    │  ← Learns gauge connection
-│  │ Parallel Transport Γ_{j→i}  │    │  ← Transports fiber elements
-│  │ Gauge-Covariant Attention   │    │  ← Modified self-attention
-│  │ Lie Algebra Generator       │    │  ← Generates inference ops
-│  │ Generator Application       │    │  ← Applies exp(X) to fiber
-│  └─────────────────────────────┘    │
-└─────────────────────────────────────┘
-     │
-     ▼
-┌─────────────────────────────────────┐
-│  Output: logits + geometric losses  │
-└─────────────────────────────────────┘
-```
-
-## Key Components
+We present **ARC (Adaptive Repetition Controller)**, a novel decode-time intervention system that addresses behavioral degradation in RLHF-aligned language models. Our approach leverages lightweight prediction heads (~5,300 parameters each) trained on compressed hidden state representations ("fiber projections") to detect and suppress undesirable generation patterns including repetition loops, hedging phrases, verbosity, and sycophantic responses.
 
-### 1. Connection Network
-Learns the gauge connection ω that defines how to parallel transport inferential states:
-```python
-A_μ(x) ∈ gl(k,ℝ)  # Lie algebra valued 1-form
-```
+Our primary contribution is demonstrating that behavioral failure modes are linearly separable in a low-dimensional projection of transformer hidden states, enabling real-time intervention with minimal computational overhead (<1% latency increase). The repetition detection head achieves a **125x class separation ratio**, indicating that the failure mode is highly predictable from internal model representations before manifesting in output tokens.
 
-### 2. Parallel Transport
-Computes transport operators between positions:
-```python
-Γ_{j→i} = exp(-A_μ(x_j)(x_i - x_j)^μ)
-```
-
-### 3. Gauge-Covariant Attention
-Standard attention with parallel transport of values:
-```python
-# Standard:  Attn(Q,K,V)_i = Σ_j α_ij V_j
-# Gauge:     GaugeAttn_i   = Σ_j α_ij Γ_{j→i}(V_j)
-```
+---
 
-### 4. Holonomy Loss
-Enforces reasoning consistency by requiring closed loops to return to identity:
-```python
-L_hol = E[||Hol_γ - I||²_F]
-```
+## Key Results
 
-### 5. Curvature Regularization
-Encourages flat reasoning spaces where order doesn't matter:
-```python
-L_curv = E[||F(x)||²_F]  where F = dω + ω∧ω
-```
+| Head | Separation | Status |
+|------|------------|--------|
+| **Repetition** | **125x** | Production Ready |
+| **Verbosity** | **2.1x** | Usable |
+| **Hedging** | **1.5x** | Contributing |
+| **Sycophancy** | experimental | Research |
 
-## Installation
+---
 
+## Quick Start
 ```bash
-pip install torch
+pip install torch transformers accelerate bitsandbytes
 ```
-
-## Usage
-
-### Basic
 ```python
-from lht import LieHolonomyTransformer, LHTConfig
-
-# Create model
-config = LHTConfig(
-    vocab_size=32000,
-    d_model=512,
-    d_fiber=64,
-    n_heads=8,
-    n_layers=6,
-    lie_algebra_rank=8,
-)
-model = LieHolonomyTransformer(config)
-
-# Forward pass
-output = model(
-    input_ids=tokens,
-    labels=labels,
-    return_geometric_losses=True
+from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
+import torch
+
+model_id = "LoganResearch/ARC-Base-8B"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    quantization_config=BitsAndBytesConfig(
+        load_in_4bit=True,
+        bnb_4bit_compute_dtype=torch.float16,
+        bnb_4bit_use_double_quant=True,
+        bnb_4bit_quant_type="nf4"
+    ),
+    device_map="auto"
 )
-
-# Get losses
-lm_loss = output['lm_loss']
-holonomy_loss = output['holonomy_loss']
-curvature_loss = output['curvature_loss']
-total_loss = model.get_total_loss(output)
 ```
 
-### Training with Geometric Loss Annealing
-```python
-from lht import LHTTrainer
+For full ARC behavioral control, download and run `inference.py`.
 
-trainer = LHTTrainer(model, optimizer, config)
+---
 
-for batch in dataloader:
-    metrics = trainer.train_step(batch)
-    # Early training: high curvature loss → flat representations
-    # Mid training: high holonomy loss → consistency
-    # Late training: high waypoint loss → discrete structure
+## Architecture
 ```
-
-### Waypoint Detection
-```python
-from lht import WaypointDetector
-
-detector = WaypointDetector(config, n_waypoints=32)
-waypoint_ids, stability = detector(representations)
+BASE MODEL (Hermes-3-Llama-3.1-8B)
+         |
+    Hidden States [32 layers x 4096 dims]
+         |
+    FIBER PROJECTIONS [32 x 16 features]
+         |
+    +------------+------------+------------+
+    | Repetition |  Hedging   | Verbosity  |
+    |   125x     |    1.5x    |    2.1x    |
+    +------------+------------+------------+
+         |
+    Risk Scores -> Intervention -> Modified Logits
 ```
 
-## Configuration
-
-| Parameter | Description | Default |
-|-----------|-------------|---------|
-| `d_model` | Proposition manifold dimension | 512 |
-| `d_fiber` | Fiber (gauge) dimension | 64 |
-| `lie_algebra_rank` | k for GL(k,ℝ) structure group | 8 |
-| `lambda_holonomy` | Weight for holonomy loss | 0.1 |
-| `lambda_curvature` | Weight for curvature loss | 0.01 |
-| `lambda_waypoint` | Weight for waypoint stability | 0.05 |
-
-## Theoretical Predictions
-
-The framework makes testable predictions:
-
-1. **Chain-of-thought benefit correlates with curvature** - High-curvature domains (causal reasoning) benefit more from CoT than low-curvature domains (arithmetic)
-
-2. **Waypoints emerge spontaneously** - Training with holonomy loss should cause discrete symbol-like structures to form at flat loci
-
-3. **Holonomy predicts errors** - Incorrect reasoning paths should have higher holonomy magnitude
-
-4. **Compositional generalization improves** - Holonomy constraints force consistent composition
-
-## File Structure
-
-```
-lie_holonomy_transformer/
-├── lht.py           # Core implementation
-├── train.py         # Training script  
-├── README.md        # This file
-└── experiments/     # Benchmark code (TODO)
+---
+
+## Repository Contents
+
+| File | Description |
+|------|-------------|
+| `model-*.safetensors` | Base model weights (~16GB) |
+| `risk_predictor.pt` | Fiber projections + Repetition head (8.4MB) |
+| `hedging_head.pt` | Hedging detection (24KB) |
+| `verbosity_head.pt` | Verbosity detection (24KB) |
+| `sycophancy_head.pt` | Sycophancy detection (24KB) |
+| `inference.py` | Complete inference with ARC |
+
+---
+
+## Citation
+```bibtex
+@software{napolitano2026arc,
+  author = {Napolitano, Logan Matthew},
+  title = {ARC: Adaptive Repetition Controller},
+  year = {2026},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/LoganResearch/ARC-Base-8B}
+}
 ```
 
-## References
-
-- "Beyond Holonomy: Lie-Algebraic Symbol Emergence..." (the paper)
-- Cohen et al. (2019). Gauge Equivariant Convolutional Networks
-- Weiler & Cesa (2019). General E(2)-Equivariant Steerable CNNs
-- The Univalent Foundations Program (2013). Homotopy Type Theory
-
-## License
+---
 
-MIT
+**Author:** Logan Matthew Napolitano | **License:** CC-BY-4.0