aurekai/sae-dictionaries

Sparse autoencoder (SAE) dictionary repository for Aurekai. Provides model interpretability through SAE coefficients and feature-to-neuron mappings.

Overview

SAE dictionaries enable unsupervised discovery of interpretable features in neural networks. This repository hosts:

• SAE Coefficients: Trained sparse autoencoder weights (.aksae, .bfsae)
• Feature Mappings: Neuron-to-interpretable-feature alignments
• Calibration Data: Validation sets for SAE reconstruction quality
• Format Converters: Tools for translating between Aurekai and Bonfyre SAE formats

Quick Start

# Download default SAE dictionary
curl -L https://huggingface.co/aurekai/sae-dictionaries/resolve/main/default.aksae -o default.aksae

# Use with Aurekai runtime
akai run <recipe> --sae-audit --sae-dict ./default.aksae

# Inspect SAE features
akai inspect sae --dict ./default.aksae --top-features 10

Format Reference

Aurekai Format (.aksae)

Binary SAE dictionary in Aurekai native format:

• Header: Magic bytes + version + model reference
• Coefficients: Optimized for Aurekai semantic routing
• Feature index: Neuron ID → interpretable feature name mapping
• Metadata: Training hyperparameters and calibration stats

File structure:

[Header: 16 bytes]
[Model ref: 32 bytes]
[Feature count: 4 bytes]
[Neuron dimension: 4 bytes]
[Coefficients: variable]
[Feature names: variable]
[Metadata: variable]
[Checksum: 32 bytes (SHA256)]

Legacy Bonfyre Format (.bfsae)

Legacy SAE format included for backward compatibility:

• Compatible with Bonfyre runtime
• Same underlying coefficient data
• Different serialization and metadata layout

Migration: Use provided converter scripts to transform between formats

Available Dictionaries

Default Dictionary

• Name: default.aksae / default.bfsae
• Model: Qwen3-8B
• Features: 16,384 neurons mapped to interpretable features
• Training: Calibrated on corpus of 1M examples
• Size: ~45 MB (compressed), ~180 MB (uncompressed)

Adding New Dictionaries

To add a new SAE dictionary:

1. Train SAE on model activations (use Aurekai SAE training pipeline)
2. Convert to .aksae format using akai sae:export
3. Include validation metrics in metadata
4. Update manifest with new dictionary entry
5. Create PR to this repository

SAE Activation & Usage

Basic Activation

# Activate SAE for model operations
akai run <recipe> \
  --sae-dict ./default.aksae \
  --sae-threshold 0.001 \
  --sae-cache-size 1GB

Feature Inspection

# Find top features by activation
akai inspect sae --dict ./default.aksae --top-features 20

# Query feature by name
akai inspect sae --dict ./default.aksae --feature "attention-head-3"

# Get neuron statistics
akai inspect sae --dict ./default.aksae --neuron-stats

Integration with Manifests

SAE dictionaries are registered in both manifests:

aurekai.manifest.json:

{
  "sae_dicts": [
    {
      "name": "default",
      "aksae": "aurekai/sae-dictionaries/default.aksae",
      "bfsae": "aurekai/sae-dictionaries/default.bfsae"
    }
  ]
}

bonfyre.manifest.json (legacy):

{
  "sae_dicts": [
    {
      "name": "default",
      "path": "aurekai/sae-dictionaries/default.bfsae"
    }
  ]
}

Performance & Validation

• Reconstruction Error: < 0.5% across validation set
• Feature Sparsity: Average 0.2% feature activation per sample
• Latency: ~2ms per inference with SAE lookup (cached)
• Memory: ~1.2 GB in-memory with mmap support

Tools & Scripts

• akai sae:export: Convert trained SAE to repository format
• akai sae:validate: Check SAE integrity and performance
• akai sae:benchmark: Compare dictionary effectiveness
• bf_sae_convert.py: Legacy Bonfyre → Aurekai format converter

Related Repositories

• Main Aurekai Repo: https://github.com/aurekai/aurekai
• Model Memory: https://huggingface.co/aurekai/model-memory
• FPQx Alignments: https://huggingface.co/aurekai/fpqx-alignments
• Semantic Cache Bench: https://huggingface.co/aurekai/semantic-cache-bench

Citation

If you use these SAE dictionaries in your research, please cite:

@dataset{aurekai_sae_dicts_2026,
  title={Aurekai SAE Dictionary Repository},
  author={Aurekai Community},
  year={2026},
  url={https://huggingface.co/aurekai/sae-dictionaries}
}

License

Licensed under the Aurekai Open Source License. See main repository for details.