Upload affine transforms with MSE metrics

README.md

@@ -0,0 +1,57 @@
+---
+tags:
+  - affine-transform
+  - activation-mapping
+library_name: safetensors
+---
+
+# Affine Transform: EleutherAI/deep-ignorance-pretraining-stage-unfiltered@global_step54832 → EleutherAI/deep-ignorance-unfiltered@main
+
+Learned affine transformation mapping hidden state activations from a source checkpoint to a target model.
+
+## Usage
+
+```python
+from safetensors.torch import load_file
+import torch.nn as nn
+from huggingface_hub import hf_hub_download
+
+# Download files
+weights_path = hf_hub_download(repo_id="EleutherAI/affine-checkpoint-transfer-step54832", filename="affine_transforms.safetensors")
+metadata_path = hf_hub_download(repo_id="EleutherAI/affine-checkpoint-transfer-step54832", filename="metadata.json")
+
+# Load
+import json
+with open(metadata_path) as f:
+    metadata = json.load(f)
+
+weights = load_file(weights_path)
+affine_transforms = {}
+for layer_idx in metadata["layer_indices"]:
+    linear = nn.Linear(metadata["hidden_dim"], metadata["hidden_dim"], bias=True)
+    linear.weight.data = weights[f"layer_{layer_idx}.weight"]
+    linear.bias.data = weights[f"layer_{layer_idx}.bias"]
+    affine_transforms[layer_idx] = linear
+```
+
+## MSE Metrics
+
+| Layer | MSE |
+|-------|-----|
+| 5 | 0.061545 |
+| 10 | 0.225621 |
+| 15 | 0.552396 |
+| 20 | 0.800604 |
+| 25 | 1.295939 |
+| 30 | 3.541077 |
+
+**Mean MSE: 1.079530**
+
+## Training Details
+
+- **Source Model:** EleutherAI/deep-ignorance-pretraining-stage-unfiltered@global_step54832
+- **Target Model:** EleutherAI/deep-ignorance-unfiltered@main
+- **Hidden Dimension:** 4096
+- **Ridge Alpha:** 0.01
+- **Layers:** [5, 10, 15, 20, 25, 30]
+- **Training Examples:** 100000

affine_transforms.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c3034a34ce55a3ffc65aff2f5a5f376c76e3fefae0a5462df78f412373b391d
+size 402752528

metadata.json

@@ -0,0 +1,12 @@
+{
+  "layer_indices": [
+    5,
+    10,
+    15,
+    20,
+    25,
+    30
+  ],
+  "hidden_dim": 4096,
+  "alpha": 0.01
+}