---
tags:
  - affine-transform
  - activation-mapping
library_name: safetensors
---

# Affine Transform: EleutherAI/deep-ignorance-pretraining-stage-unfiltered@global_step38144 → EleutherAI/deep-ignorance-unfiltered@main

Learned affine transformation mapping hidden state activations from a source checkpoint to a target model.

## Usage

```python
from safetensors.torch import load_file
import torch.nn as nn
from huggingface_hub import hf_hub_download

# Download files
weights_path = hf_hub_download(repo_id="EleutherAI/affine-checkpoint-transfer", filename="affine_transforms.safetensors")
metadata_path = hf_hub_download(repo_id="EleutherAI/affine-checkpoint-transfer", filename="metadata.json")

# Load
import json
with open(metadata_path) as f:
    metadata = json.load(f)

weights = load_file(weights_path)
affine_transforms = {}
for layer_idx in metadata["layer_indices"]:
    linear = nn.Linear(metadata["hidden_dim"], metadata["hidden_dim"], bias=True)
    linear.weight.data = weights[f"layer_{layer_idx}.weight"]
    linear.bias.data = weights[f"layer_{layer_idx}.bias"]
    affine_transforms[layer_idx] = linear
```

## MSE Metrics

| Layer | MSE |
|-------|-----|
| 5 | 0.081037 |
| 10 | 0.289330 |
| 15 | 0.684350 |
| 20 | 0.978894 |
| 25 | 1.569308 |
| 30 | 4.231404 |

**Mean MSE: 1.305720**

## Training Details

- **Source Model:** EleutherAI/deep-ignorance-pretraining-stage-unfiltered@global_step38144
- **Target Model:** EleutherAI/deep-ignorance-unfiltered@main
- **Hidden Dimension:** 4096
- **Ridge Alpha:** 0.01
- **Layers:** [5, 10, 15, 20, 25, 30]
- **Training Examples:** 100000