KokosDev commited on Nov 3, 2025

Commit

bc702e3

1 Parent(s): 6f23d58

Major update: Add MLP→CLT mappings + retrained transcoders

- Added mapping_L{0-30}.pt files with MLP→CLT correlation mappings
- Retrained all 31 transcoders with improved training (5000 steps each)
- Updated README with comprehensive usage documentation
- Added training_summary.json with detailed metrics
- All layers achieve 0% dead features
- Sparsity: 2-29% L0 (layer-dependent, deeper layers sparser)
- Reconstruction loss: <0.1 for most layers

Key features:
✅ MLP→CLT co-activation based mapping for feature attribution
✅ Decoder weights for CLT→MLP reconstruction
✅ Complete training metadata and metrics
✅ Ready for circuit-tracer style attribution analysis

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

.gitattributes +0 -34
README.md +190 -64
config.json +0 -40
transcoder_L0.safetensors → mapping_L0.pt +2 -2
transcoder_L1.safetensors → mapping_L1.pt +2 -2
transcoder_L10.safetensors → mapping_L10.pt +2 -2
transcoder_L11.safetensors → mapping_L11.pt +2 -2
mapping_L12.pt +3 -0
mapping_L13.pt +3 -0
mapping_L14.pt +3 -0
mapping_L15.pt +3 -0
mapping_L16.pt +3 -0
mapping_L17.pt +3 -0
mapping_L18.pt +3 -0
mapping_L19.pt +3 -0
mapping_L2.pt +3 -0
mapping_L20.pt +3 -0
mapping_L21.pt +3 -0
mapping_L22.pt +3 -0
mapping_L23.pt +3 -0
mapping_L24.pt +3 -0
mapping_L25.pt +3 -0
mapping_L26.pt +3 -0
mapping_L27.pt +3 -0
mapping_L28.pt +3 -0
mapping_L29.pt +3 -0
mapping_L3.pt +3 -0
mapping_L30.pt +3 -0
mapping_L4.pt +3 -0
mapping_L5.pt +3 -0
mapping_L6.pt +3 -0
mapping_L7.pt +3 -0
mapping_L8.pt +3 -0
mapping_L9.pt +3 -0
metrics/metrics_L0.json +0 -185
metrics/metrics_L1.json +0 -185
metrics/metrics_L10.json +0 -185
metrics/metrics_L11.json +0 -185
metrics/metrics_L12.json +0 -185
metrics/metrics_L13.json +0 -185
metrics/metrics_L14.json +0 -185
metrics/metrics_L15.json +0 -185
metrics/metrics_L16.json +0 -185
metrics/metrics_L17.json +0 -185
metrics/metrics_L18.json +0 -185
metrics/metrics_L19.json +0 -185
metrics/metrics_L2.json +0 -185
metrics/metrics_L20.json +0 -185
metrics/metrics_L21.json +0 -185
metrics/metrics_L22.json +0 -185

.gitattributes CHANGED Viewed

@@ -1,35 +1 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text























1	*.pt filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -1,101 +1,227 @@
----
-license: apache-2.0
-language:
-- en
-tags:
-- llava
-- clt
-- sparse-autoencoder
-- circuits
-- multimodal
-- vision-language
-- feature-extraction
-base_model: llava-hf/llava-1.5-7b-hf
-pipeline_tag: feature-extraction
-datasets:
-- liuhaotian/LLaVA-Instruct-150K
 ---
-# LLaVA-1.5-7B CLT Transcoders (31 layers)
-Sparse transcoders (CLT mode) trained on activations captured from `llava-hf/llava-1.5-7b-hf` over a multimodal (image+text) dataset. Each layer `L0..L30` has a separate transcoder that encodes the layer output `X` into a sparse feature space and decodes to predict the MLP output `Y` of the same layer.
-- Base model: `llava-hf/llava-1.5-7b-hf`
-- Mode: CLT (Cross-Layer Transcoder)
 - Hidden dim: 4096
-- Feature dim: 8192 (TopK ~12%)
-- Files: `transcoder_L{L}.safetensors` + `transcoder_L{L}.json`
-- Metrics per layer: `metrics/metrics_L{L}.json`
-## Usage
 ```python
-import torch
-from safetensors.torch import load_file
-# Load a layer transcoder (example: L=30)
-L = 30
-state_dict = load_file(f"transcoder_L{L}.safetensors")
-# Minimal PyTorch module matching the training architecture
 import torch.nn as nn
 class Transcoder(nn.Module):
-    def __init__(self, hidden_dim=4096, feature_dim=8192):
         super().__init__()
         self.enc = nn.Sequential(
             nn.LayerNorm(hidden_dim),
             nn.Linear(hidden_dim, feature_dim),
         )
         self.dec = nn.Linear(feature_dim, hidden_dim)
-    def forward(self, x):  # x: [B,T,H]
         z_pre = self.enc(x)
         z = torch.relu(z_pre)
         y_hat = self.dec(z)
-        return y_hat, z
-model = Transcoder(hidden_dim=4096, feature_dim=8192)
-model.load_state_dict(state_dict)
-model.eval()
 ```
-## File structure
 ```
-- README.md
-- config.json                # top-level metadata
-- summary.json               # final L0% & rec loss per layer
-- transcoder_L0.safetensors
-- transcoder_L0.json
-...
-- transcoder_L30.safetensors
-- transcoder_L30.json
-- metrics/
-  - metrics_L0.json
-  ...
-  - metrics_L30.json
 ```
-## Training details
-- CLT hooks: X = layer output; Y = MLP output (per-layer)
-- Sequence length: 512 (cropped per sample)
-- Optimizer: AdamW, LR=3e-4, 5000 steps per layer
-- AMP: bf16/fp16 depending on hardware
-- TopK sparsity: ~12% target (achieved ~11–12% on later layers)
-## Training data
-- LLaVA-Instruct-150K (train/val JSONL) for multimodal prompts and image-text pairs
-- Images referenced by LLaVA-Instruct (e.g., COCO 2017) resolved locally during activation capture
-- Activations captured from `llava-hf/llava-1.5-7b-hf` at seq_len=512, then consolidated into per-layer batch files for CLT training
-## Metrics snapshot
-- Example L30: final L0 ≈ 11.8%, final rec ≈ 0.063
-- Early layers may show lower effective L0 (expected)
 ## License
-Apache-2.0

+# LLaVA-1.5-7B Cross-Layer Transcoders (CLTs)
+## Overview
+This repository contains **Cross-Layer Transcoders (CLTs)** trained on [llava-hf/llava-1.5-7b-hf](https://huggingface.co/llava-hf/llava-1.5-7b-hf) for mechanistic interpretability research. CLTs are sparse autoencoders that decompose dense MLP activations into interpretable features, enabling attribution analysis and feature steering in vision-language models.
+### Key Features
+- ✅ **31 layers of transcoders** (L0-L30) covering all LLaVA language model layers
+- ✅ **MLP→CLT mappings** for every layer (co-activation based correlation)
+- ✅ **Decoder weights** for CLT→MLP reconstruction
+- ✅ **0% dead features** across all layers
+- ✅ **2-5% average sparsity** in early/middle layers (interpretable and efficient)
 ---
+## Architecture
+```
+Input (MLP hidden state):  [batch, seq_len, 4096]
+         ↓
+    Transcoder Encoder:  LayerNorm + Linear(4096 → 8192) + ReLU
+         ↓
+    Sparse Features:  [batch, seq_len, 8192] (~2-5% active)
+         ↓
+    Transcoder Decoder:  Linear(8192 → 4096)
+         ↓
+Output (MLP reconstruction):  [batch, seq_len, 4096]
+```
+**Parameters per layer:**
 - Hidden dim: 4096
+- Feature dim: 8192 (2× expansion)
+- Total parameters per transcoder: ~67M
+- Sparsity: 2-29% L0 (layer-dependent, deeper layers are sparser)
+---
+## Training Details
+- **Model**: `llava-hf/llava-1.5-7b-hf`
+- **Dataset**: ~45K multimodal samples (Flickr30K + instruction tasks)
+- **Steps per layer**: 5,000
+- **Learning rate**: 3e-4 (AdamW)
+- **Batch size**: 16 samples
+- **Sparsity penalty**: 0.01 (L1 on features)
+- **Validation**: Every 200 steps
+### Training Quality Metrics
+| Layer Range | Avg Sparsity (L0%) | Avg Reconstruction Loss | Dead Features |
+|-------------|-------------------|------------------------|---------------|
+| L0-L10      | 2-4%             | 0.05-0.15              | 0%            |
+| L11-L18     | 3-7%             | 0.05-0.10              | 0%            |
+| L19-L30     | 7-29%            | 0.05-0.20              | 0%            |
+**Note**: Higher sparsity in deeper layers (L19-L30) is expected behavior in transformers, where later layers are more specialized.
+---
+## Files
+Each layer has two files:
+### 1. `transcoder_L{layer}.pt`
+Contains the trained transcoder model and training metadata.
 ```python
+checkpoint = torch.load('transcoder_L5.pt')
+# Keys: 'layer', 'hidden_dim', 'feature_dim', 'state_dict', 'training_metadata', 'mlp_to_clt_mapping'
+```
+### 2. `mapping_L{layer}.pt`
+Contains MLP→CLT mapping and decoder weights for analysis.
+```python
+mapping = torch.load('mapping_L5.pt')
+# Keys: 'layer', 'mlp_to_clt_mapping', 'decoder_weights', 'hidden_dim', 'feature_dim', 'description'
+# mlp_to_clt_mapping: [4096, 8192] - which MLP neurons correlate with each CLT feature
+# decoder_weights: [4096, 8192] - CLT → MLP reconstruction weights
+```
+---
+## Usage
+### 1. Load a Transcoder
+```python
+import torch
 import torch.nn as nn
 class Transcoder(nn.Module):
+    def __init__(self, hidden_dim: int, feature_dim: int):
         super().__init__()
         self.enc = nn.Sequential(
             nn.LayerNorm(hidden_dim),
             nn.Linear(hidden_dim, feature_dim),
         )
         self.dec = nn.Linear(feature_dim, hidden_dim)
+    def forward(self, x):
         z_pre = self.enc(x)
         z = torch.relu(z_pre)
         y_hat = self.dec(z)
+        return y_hat, z  # reconstruction, features
+# Load Layer 10 transcoder
+checkpoint = torch.load('transcoder_L10.pt', map_location='cpu')
+hidden_dim = checkpoint['hidden_dim']
+feature_dim = checkpoint['feature_dim']
+transcoder = Transcoder(hidden_dim, feature_dim)
+transcoder.load_state_dict(checkpoint['state_dict'])
+transcoder.eval()
+# Use with LLaVA MLP outputs
+with torch.no_grad():
+    mlp_output = ...  # [batch, seq_len, 4096] from LLaVA layer 10
+    reconstruction, features = transcoder(mlp_output)
+    # features: [batch, seq_len, 8192] - sparse interpretable features
+    # reconstruction: [batch, seq_len, 4096] - reconstructed MLP output
 ```
+### 2. Use MLP→CLT Mapping
+The mapping shows which MLP neurons are correlated with each CLT feature:
+```python
+mapping_data = torch.load('mapping_L10.pt', map_location='cpu')
+mlp_to_clt = mapping_data['mlp_to_clt_mapping']  # [4096, 8192]
+# Find top MLP neurons for a specific CLT feature
+feature_idx = 1234
+top_mlp_neurons = mlp_to_clt[:, feature_idx].topk(k=10)
+print(f"Top MLP neurons for feature {feature_idx}: {top_mlp_neurons.indices}")
+# Find top CLT features for a specific MLP neuron
+mlp_neuron_idx = 567
+top_clt_features = mlp_to_clt[mlp_neuron_idx, :].topk(k=10)
+print(f"Top CLT features for MLP neuron {mlp_neuron_idx}: {top_clt_features.indices}")
 ```
+### 3. Replacement Model (Full Integration)
+For direct integration into LLaVA (replace MLPs with CLTs):
+```python
+from transformers import LlavaForConditionalGeneration
+# Load LLaVA
+model = LlavaForConditionalGeneration.from_pretrained(
+    "llava-hf/llava-1.5-7b-hf",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+# Replace MLP in layer 10 with CLT (example)
+layer_idx = 10
+checkpoint = torch.load(f'transcoder_L{layer_idx}.pt')
+transcoder = Transcoder(checkpoint['hidden_dim'], checkpoint['feature_dim'])
+transcoder.load_state_dict(checkpoint['state_dict'])
+# Hook to replace MLP forward pass
+def replace_mlp_with_clt(module, input, output):
+    hidden_state = input[0]
+    reconstruction, features = transcoder(hidden_state)
+    return reconstruction
+model.model.layers[layer_idx].mlp.register_forward_hook(replace_mlp_with_clt)
 ```
+---
+## Applications
+### 1. **Feature Attribution**
+Identify which features contribute to specific model outputs (hallucination detection, sycophancy analysis).
+### 2. **Feature Steering**
+Amplify or suppress specific features at inference time to modify model behavior (reduce hallucinations, improve grounding).
+### 3. **Mechanistic Interpretability**
+Build attribution graphs showing causal relationships between features and outputs.
+### 4. **Circuit Discovery**
+Map feature interactions across layers to understand how the model processes multimodal information.
+---
+## Related Work
+This work extends Anthropic's Circuit-Tracer methodology to multimodal vision-language models:
+- [Circuit-Tracer Paper](https://transformer-circuits.pub/2025/attribution-graphs/methods.html)
+- [Sparse Autoencoders for Interpretability](https://transformer-circuits.pub/2023/monosemantic-features/index.html)
+---
+## Citation
+If you use these transcoders in your research, please cite:
+```bibtex
+@misc{llava15_clts_2025,
+  title={Cross-Layer Transcoders for LLaVA-1.5-7B},
+  author={Koko's Dev},
+  year={2025},
+  publisher={HuggingFace Hub},
+  howpublished={\url{https://huggingface.co/KokosDev/llava15-7b-clt}}
+}
+```
+---
 ## License
+These transcoders are released under the same license as the base model (Apache 2.0). The base LLaVA-1.5-7B model is from [llava-hf/llava-1.5-7b-hf](https://huggingface.co/llava-hf/llava-1.5-7b-hf).
+---
+## Acknowledgments
+- **Base Model**: [LLaVA-1.5-7B](https://huggingface.co/llava-hf/llava-1.5-7b-hf)
+- **Methodology**: Inspired by Anthropic's Circuit-Tracer and sparse autoencoder research
+- **Training Data**: Flickr30K, instruction-following datasets

config.json DELETED Viewed

@@ -1,40 +0,0 @@
-{
-  "base_model": "llava-hf/llava-1.5-7b-hf",
-  "mode": "CLT",
-  "num_layers": 31,
-  "layers": [
-    0,
-    1,
-    2,
-    3,
-    4,
-    5,
-    6,
-    7,
-    8,
-    9,
-    10,
-    11,
-    12,
-    13,
-    14,
-    15,
-    16,
-    17,
-    18,
-    19,
-    20,
-    21,
-    22,
-    23,
-    24,
-    25,
-    26,
-    27,
-    28,
-    29,
-    30
-  ],
-  "hidden_dim": 4096,
-  "feature_dim": 8192
-}

transcoder_L0.safetensors → mapping_L0.pt RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:42fa2efd9d87043074ee86bb18bd31c74ee342703ecd6049c6fd789de4ef39a4
-size 134259240

 version https://git-lfs.github.com/spec/v1
+oid sha256:926764d522494fab99349e5047f7826bf947976ad9738558a2d43fcf31f24642
+size 201328637

transcoder_L1.safetensors → mapping_L1.pt RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c51f45bb508634a2236211fee3970bbabdc048097e6286240313bdb67d55be27
-size 134259240

 version https://git-lfs.github.com/spec/v1
+oid sha256:ac28c58669174849b7cea5b1e74f0888fb31fecd15a2f5ee6c5a90e19ada295c
+size 201328637

transcoder_L10.safetensors → mapping_L10.pt RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e5fce28f0abe6c3d4ea185143bf5acd51fc377b5e2ef92bf383755dfb620e5f8
-size 134259240

 version https://git-lfs.github.com/spec/v1
+oid sha256:49d109136f4ad7d98409b455573405e0735aaeb8154e983f0cca7465a07bd699
+size 201328645

transcoder_L11.safetensors → mapping_L11.pt RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2ccb02fe11f1b247b0095ce6e2cf68b7a70e4e9c5fb51c9f097577b9ca5d6989
-size 134259240

 version https://git-lfs.github.com/spec/v1
+oid sha256:7948d7131b865188c4853cf2313260788e5e29afd1119c5a5da3820a0cba8d7c
+size 201328645

mapping_L12.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:001461dbd47fde64883a7795080898124517902f65669d7b200618f38229cb63
+size 201328645

mapping_L13.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e51581cbd3921b0028a1f83bcaaae48d7816d29adc93a44092349425fb7f2d45
+size 201328645

mapping_L14.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d01550fd3f7c7f755709ab41cdbc5cd2d7cd71bfaf08d50d7a8891d29a7e9e55
+size 201328645

mapping_L15.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0b47dc82af9670279792da11774ab38e5c0e3d2438cfc7a0ac368b5333ae7028
+size 201328645

mapping_L16.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:df805b6e3010a3157bb461936f5b1c78c62a26f288aca5cba5494d146dbb66d6
+size 201328645

mapping_L17.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7ac626f96b0b5c274c8cef6c76616879fb1556e417153419d164bd2805502d3f
+size 201328645

mapping_L18.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8e026f2608e3ba31b8ee7e087112b1687526e42b21efa132030c96c25b1f3304
+size 201328645

mapping_L19.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fcd373d9432a95e50384ecc54135eabedf289b0480334dbbb3575ac46221f4a1
+size 201328645

mapping_L2.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:04e4e92c46fcd8fbfe3411d8905a82af869701c4348d5c95e0e253c7dec9b6fd
+size 201328637

mapping_L20.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:630ad511f3fe8a53eff06bc385277726966b4303d192451748bb8e1f83505b9a
+size 201328645

mapping_L21.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c2565eafc65c19cfe919b40d425d541b0c6e3442cef7838e9ee288b70229068
+size 201328645

mapping_L22.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4ab93c3c8ad2b952174eaf2556f8390857955c2a7c62b0b43e4c15550d2cf924
+size 201328645

mapping_L23.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd0d8568442be298b1013327692d59428ef70a87c594a142d0de1b3da61b649d
+size 201328645

mapping_L24.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:40e1550373734e47dee1de1dd10bc7777016be3d3a83982badff7b04c72c67ee
+size 201328645

mapping_L25.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7500aa2a3757114f460c1274774856e420aca82fd9919481ffe57890cbc068b5
+size 201328645

mapping_L26.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:24f3f80e2e289eda2a424b43c41e3137b190a8bf43672515285cc1d9b835e02f
+size 201328645

mapping_L27.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e17d03cf4b72e617c34f45ed34783b10cc1f8929c03b32e64f512458d13df7b9
+size 201328645

mapping_L28.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3299928c35caa7afa9980427d54c0b8feda5aa2df4bd2cd2b81a1b6a7e977291
+size 201328645

mapping_L29.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5dd5a230f203706f0bdfdc92ee53fa93ab295c66847d8cc1d328a41c37ba1438
+size 201328645

mapping_L3.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a807b4bfb7598cf86dafd7a4f50452bb5709db763562dc5609a6ad44e737f58b
+size 201328637

mapping_L30.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e7161153190b60857244937449ca376718b2d993a62e4c32c35b7702d832869f
+size 201328645

mapping_L4.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0871af791813ec4c33ffa6ae3b1df89c042277887f4653d4d7cc96f4544f088b
+size 201328637

mapping_L5.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0e9bf0d2d4d8d8ab2ba2ec346403f3ca36fcfcc774c58f0d3c7300a3f0fb6abf
+size 201328637

mapping_L6.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5b6015468b1ac57bf8e206aabb76d8d2db229a8a155861782d7a03e8b49c1bf4
+size 201328637

mapping_L7.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e269cb4328bc01c7402a4024ea2fdd04652c5ee714346268a6168f366fc74e15
+size 201328637

mapping_L8.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb2958b47d295bfd367206933273952174df3432a5f622cf90c07be8b72a5986
+size 201328637

mapping_L9.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8876128aace266d31987461c15548a2d5c770f3ec0d4ebda28ce27ef89eb25b5
+size 201328637

metrics/metrics_L0.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 0,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.0003072486724704504,
-      "val_loss": 0.0003067493438720703,
-      "train_l0": 0.47220587730407715,
-      "val_l0": 0.3172755241394043
-    },
-    {
-      "step": 400,
-      "train_loss": 0.00025499684852547944,
-      "val_loss": 0.00026645660400390623,
-      "train_l0": 0.3529787063598633,
-      "val_l0": 0.3875422477722168
-    },
-    {
-      "step": 600,
-      "train_loss": 0.00025995602481998503,
-      "val_loss": 0.00023263096809387208,
-      "train_l0": 0.32089948654174805,
-      "val_l0": 0.34657716751098633
-    },
-    {
-      "step": 800,
-      "train_loss": 0.00024188542738556862,
-      "val_loss": 0.00024361610412597655,
-      "train_l0": 0.2146601676940918,
-      "val_l0": 0.24510860443115234
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.0002084258885588497,
-      "val_loss": 0.0002314150333404541,
-      "train_l0": 0.2898752689361572,
-      "val_l0": 0.22312402725219727
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.00025186064885929227,
-      "val_loss": 0.00021649599075317382,
-      "train_l0": 0.22274255752563477,
-      "val_l0": 0.22777795791625977
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.00023313461861107498,
-      "val_loss": 0.000215911865234375,
-      "train_l0": 0.21011829376220703,
-      "val_l0": 0.22365093231201172
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.0002374562027398497,
-      "val_loss": 0.00021054744720458985,
-      "train_l0": 0.1807570457458496,
-      "val_l0": 0.2297806739807129
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.00022862674086354673,
-      "val_loss": 0.00022011399269104004,
-      "train_l0": 0.19682049751281738,
-      "val_l0": 0.1999950408935547
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.00020527455490082502,
-      "val_loss": 0.0001972675323486328,
-      "train_l0": 0.22835731506347656,
-      "val_l0": 0.26402711868286133
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.00020355929154902697,
-      "val_loss": 0.00022192001342773436,
-      "train_l0": 0.229644775390625,
-      "val_l0": 0.2043318748474121
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.00021205551456660032,
-      "val_loss": 0.0002220630645751953,
-      "train_l0": 0.2176821231842041,
-      "val_l0": 0.22295713424682617
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.00021079782163724303,
-      "val_loss": 0.00021343231201171876,
-      "train_l0": 0.21379590034484863,
-      "val_l0": 0.21446943283081055
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.00022800829901825637,
-      "val_loss": 0.00023617744445800782,
-      "train_l0": 0.1966416835784912,
-      "val_l0": 0.19351720809936523
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.00020191156363580376,
-      "val_loss": 0.00021142959594726561,
-      "train_l0": 0.2100825309753418,
-      "val_l0": 0.21882295608520508
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.00021879124687984586,
-      "val_loss": 0.00023055076599121094,
-      "train_l0": 0.22339820861816406,
-      "val_l0": 0.20228862762451172
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.0001810171379474923,
-      "val_loss": 0.00021448135375976563,
-      "train_l0": 0.23145079612731934,
-      "val_l0": 0.23223161697387695
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.00018641870701685548,
-      "val_loss": 0.0002062082290649414,
-      "train_l0": 0.21660923957824707,
-      "val_l0": 0.22423267364501953
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.00017806273535825312,
-      "val_loss": 0.00022385120391845703,
-      "train_l0": 0.1947641372680664,
-      "val_l0": 0.18842220306396484
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.00017908113659359515,
-      "val_loss": 0.0002302885055541992,
-      "train_l0": 0.21029114723205566,
-      "val_l0": 0.18979549407958984
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.0002036196819972247,
-      "val_loss": 0.00022821426391601564,
-      "train_l0": 0.19829273223876953,
-      "val_l0": 0.1981973648071289
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.0002189793303841725,
-      "val_loss": 0.00019947290420532226,
-      "train_l0": 0.19347071647644043,
-      "val_l0": 0.19977569580078125
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.00017246737843379378,
-      "val_loss": 0.00020782947540283204,
-      "train_l0": 0.2507627010345459,
-      "val_l0": 0.21579265594482422
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.00017646313062869012,
-      "val_loss": 0.0002214670181274414,
-      "train_l0": 0.23015737533569336,
-      "val_l0": 0.21255731582641602
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.00021874290541745722,
-      "val_loss": 0.00021724700927734376,
-      "train_l0": 0.2058267593383789,
-      "val_l0": 0.20237207412719727
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 0.2058267593383789
-  }
-}

metrics/metrics_L1.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 1,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.9419434070587158,
-      "val_loss": 0.693658447265625,
-      "train_l0": 10.727488994598389,
-      "val_l0": 10.474152565002441
-    },
-    {
-      "step": 400,
-      "train_loss": 0.7538449764251709,
-      "val_loss": 0.53499755859375,
-      "train_l0": 10.236090421676636,
-      "val_l0": 10.744032859802246
-    },
-    {
-      "step": 600,
-      "train_loss": 0.9511469602584839,
-      "val_loss": 0.40771331787109377,
-      "train_l0": 9.165537357330322,
-      "val_l0": 10.436451435089111
-    },
-    {
-      "step": 800,
-      "train_loss": 0.5759391188621521,
-      "val_loss": 1.3465713500976562,
-      "train_l0": 10.276931524276733,
-      "val_l0": 8.959136009216309
-    },
-    {
-      "step": 1000,
-      "train_loss": 1.637937068939209,
-      "val_loss": 1.3444091796875,
-      "train_l0": 7.751274108886719,
-      "val_l0": 8.770787715911865
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.6342856884002686,
-      "val_loss": 0.36365509033203125,
-      "train_l0": 8.721637725830078,
-      "val_l0": 9.692184925079346
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.6578925848007202,
-      "val_loss": 0.6135894775390625,
-      "train_l0": 9.744536876678467,
-      "val_l0": 9.652924537658691
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.8243253827095032,
-      "val_loss": 1.1909713745117188,
-      "train_l0": 9.919911623001099,
-      "val_l0": 9.420511722564697
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.3240302503108978,
-      "val_loss": 0.5864349365234375,
-      "train_l0": 10.426890850067139,
-      "val_l0": 9.911370277404785
-    },
-    {
-      "step": 2000,
-      "train_loss": 1.614189624786377,
-      "val_loss": 0.5590194702148438,
-      "train_l0": 8.73565673828125,
-      "val_l0": 9.86403226852417
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.8542783856391907,
-      "val_loss": 1.0432174682617188,
-      "train_l0": 9.816688299179077,
-      "val_l0": 9.802188873291016
-    },
-    {
-      "step": 2400,
-      "train_loss": 1.3850717544555664,
-      "val_loss": 1.02093505859375,
-      "train_l0": 9.431636333465576,
-      "val_l0": 9.918689727783203
-    },
-    {
-      "step": 2600,
-      "train_loss": 1.3293514251708984,
-      "val_loss": 1.1007492065429687,
-      "train_l0": 9.813636541366577,
-      "val_l0": 10.1796293258667
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.8528616428375244,
-      "val_loss": 1.0055511474609375,
-      "train_l0": 10.77679991722107,
-      "val_l0": 9.498050212860107
-    },
-    {
-      "step": 3000,
-      "train_loss": 1.6519041061401367,
-      "val_loss": 0.6967056274414063,
-      "train_l0": 9.290111064910889,
-      "val_l0": 10.083272457122803
-    },
-    {
-      "step": 3200,
-      "train_loss": 1.3391976356506348,
-      "val_loss": 1.432598876953125,
-      "train_l0": 9.429383277893066,
-      "val_l0": 8.872983455657959
-    },
-    {
-      "step": 3400,
-      "train_loss": 1.6542689800262451,
-      "val_loss": 0.4710418701171875,
-      "train_l0": 9.355288743972778,
-      "val_l0": 10.494527816772461
-    },
-    {
-      "step": 3600,
-      "train_loss": 1.365459680557251,
-      "val_loss": 1.354937744140625,
-      "train_l0": 9.697532653808594,
-      "val_l0": 9.892845153808594
-    },
-    {
-      "step": 3800,
-      "train_loss": 1.3053505420684814,
-      "val_loss": 0.9811141967773438,
-      "train_l0": 8.926576375961304,
-      "val_l0": 9.051361083984375
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.5850197672843933,
-      "val_loss": 1.3319610595703124,
-      "train_l0": 9.88953709602356,
-      "val_l0": 9.219849109649658
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.31436559557914734,
-      "val_loss": 0.7597930908203125,
-      "train_l0": 9.815418720245361,
-      "val_l0": 9.851465225219727
-    },
-    {
-      "step": 4400,
-      "train_loss": 1.077832579612732,
-      "val_loss": 1.079656982421875,
-      "train_l0": 9.930789470672607,
-      "val_l0": 10.260002613067627
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.5662795305252075,
-      "val_loss": 0.9822235107421875,
-      "train_l0": 10.273551940917969,
-      "val_l0": 9.154860973358154
-    },
-    {
-      "step": 4800,
-      "train_loss": 1.9361178874969482,
-      "val_loss": 0.596685791015625,
-      "train_l0": 9.330040216445923,
-      "val_l0": 11.255741119384766
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.0452130101621151,
-      "val_loss": 0.8519371032714844,
-      "train_l0": 11.875146627426147,
-      "val_l0": 9.815096855163574
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 11.875146627426147
-  }
-}

metrics/metrics_L10.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 10,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.25114452838897705,
-      "val_loss": 0.30525054931640627,
-      "train_l0": 5.801188945770264,
-      "val_l0": 5.450534820556641
-    },
-    {
-      "step": 400,
-      "train_loss": 0.11518851667642593,
-      "val_loss": 0.2629669189453125,
-      "train_l0": 5.111277103424072,
-      "val_l0": 5.683934688568115
-    },
-    {
-      "step": 600,
-      "train_loss": 0.16822582483291626,
-      "val_loss": 0.10852508544921875,
-      "train_l0": 3.7683725357055664,
-      "val_l0": 4.535846710205078
-    },
-    {
-      "step": 800,
-      "train_loss": 0.13124150037765503,
-      "val_loss": 0.3328338623046875,
-      "train_l0": 3.5957694053649902,
-      "val_l0": 3.9565396308898926
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.046043723821640015,
-      "val_loss": 0.06258087158203125,
-      "train_l0": 4.6050965785980225,
-      "val_l0": 4.226646423339844
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.25528469681739807,
-      "val_loss": 0.13884201049804687,
-      "train_l0": 3.949427604675293,
-      "val_l0": 3.839094638824463
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.276385635137558,
-      "val_loss": 0.20626220703125,
-      "train_l0": 3.6357343196868896,
-      "val_l0": 4.6518754959106445
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.12123381346464157,
-      "val_loss": 0.156231689453125,
-      "train_l0": 3.2030463218688965,
-      "val_l0": 4.437422752380371
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.2816537022590637,
-      "val_loss": 0.11364669799804687,
-      "train_l0": 4.115825891494751,
-      "val_l0": 3.6349129676818848
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.11043612658977509,
-      "val_loss": 0.16072845458984375,
-      "train_l0": 3.5869598388671875,
-      "val_l0": 3.8583016395568848
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.0763341560959816,
-      "val_loss": 0.1752197265625,
-      "train_l0": 3.29434871673584,
-      "val_l0": 3.7455320358276367
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.10305751115083694,
-      "val_loss": 0.15592117309570314,
-      "train_l0": 3.9294421672821045,
-      "val_l0": 3.3687186241149902
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.0026209885254502296,
-      "val_loss": 0.22261276245117187,
-      "train_l0": 2.8446197509765625,
-      "val_l0": 3.688523769378662
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.0903225839138031,
-      "val_loss": 0.11019287109375,
-      "train_l0": 3.213447332382202,
-      "val_l0": 3.316357135772705
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.11671537905931473,
-      "val_loss": 0.1362518310546875,
-      "train_l0": 3.646618127822876,
-      "val_l0": 3.2197976112365723
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.1287200003862381,
-      "val_loss": 0.17482376098632812,
-      "train_l0": 3.5375654697418213,
-      "val_l0": 3.8109779357910156
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.20451630651950836,
-      "val_loss": 0.16786842346191405,
-      "train_l0": 2.8381645679473877,
-      "val_l0": 3.196878433227539
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.07456573843955994,
-      "val_loss": 0.11831588745117187,
-      "train_l0": 3.2184183597564697,
-      "val_l0": 3.5810303688049316
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.22658288478851318,
-      "val_loss": 0.1669994354248047,
-      "train_l0": 3.7450194358825684,
-      "val_l0": 3.519585132598877
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.07381106913089752,
-      "val_loss": 0.15676345825195312,
-      "train_l0": 3.4190475940704346,
-      "val_l0": 2.9603052139282227
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.1808185875415802,
-      "val_loss": 0.14236946105957032,
-      "train_l0": 3.0088305473327637,
-      "val_l0": 3.057527542114258
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.0016893134452402592,
-      "val_loss": 0.09865303039550781,
-      "train_l0": 2.7297258377075195,
-      "val_l0": 3.4493255615234375
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.10395370423793793,
-      "val_loss": 0.1289844512939453,
-      "train_l0": 3.6900579929351807,
-      "val_l0": 3.1922101974487305
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.04912750422954559,
-      "val_loss": 0.17699928283691407,
-      "train_l0": 3.518986701965332,
-      "val_l0": 3.6074113845825195
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.16472768783569336,
-      "val_loss": 0.23595848083496093,
-      "train_l0": 3.7324130535125732,
-      "val_l0": 3.978390693664551
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 3.7324130535125732
-  }
-}

metrics/metrics_L11.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 11,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.09921978414058685,
-      "val_loss": 0.1084686279296875,
-      "train_l0": 6.2422215938568115,
-      "val_l0": 5.507922172546387
-    },
-    {
-      "step": 400,
-      "train_loss": 0.009879800491034985,
-      "val_loss": 0.1968353271484375,
-      "train_l0": 5.2117228507995605,
-      "val_l0": 5.451998710632324
-    },
-    {
-      "step": 600,
-      "train_loss": 0.27727043628692627,
-      "val_loss": 0.18078689575195311,
-      "train_l0": 4.555940628051758,
-      "val_l0": 3.7746644020080566
-    },
-    {
-      "step": 800,
-      "train_loss": 0.19308589398860931,
-      "val_loss": 0.19725723266601564,
-      "train_l0": 4.724091291427612,
-      "val_l0": 4.179141521453857
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.2370215505361557,
-      "val_loss": 0.16272201538085937,
-      "train_l0": 5.1734983921051025,
-      "val_l0": 3.8032984733581543
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.24188938736915588,
-      "val_loss": 0.23206787109375,
-      "train_l0": 4.156309366226196,
-      "val_l0": 3.619532585144043
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.2095375657081604,
-      "val_loss": 0.29433555603027345,
-      "train_l0": 3.704559803009033,
-      "val_l0": 3.7230396270751953
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.20849841833114624,
-      "val_loss": 0.15409889221191406,
-      "train_l0": 3.4005343914031982,
-      "val_l0": 3.734006881713867
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.049407750368118286,
-      "val_loss": 0.15776596069335938,
-      "train_l0": 3.005021810531616,
-      "val_l0": 4.101603031158447
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.45641109347343445,
-      "val_loss": 0.07792243957519532,
-      "train_l0": 3.6822736263275146,
-      "val_l0": 3.547511100769043
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.11361617594957352,
-      "val_loss": 0.21431961059570312,
-      "train_l0": 4.268193244934082,
-      "val_l0": 3.597719669342041
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.2159542739391327,
-      "val_loss": 0.1529510498046875,
-      "train_l0": 3.774815797805786,
-      "val_l0": 3.5473108291625977
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.22549620270729065,
-      "val_loss": 0.19571762084960936,
-      "train_l0": 4.037141799926758,
-      "val_l0": 4.042267799377441
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.37636297941207886,
-      "val_loss": 0.20887794494628906,
-      "train_l0": 3.8642048835754395,
-      "val_l0": 3.6960291862487793
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.14850756525993347,
-      "val_loss": 0.08110885620117188,
-      "train_l0": 3.3547163009643555,
-      "val_l0": 3.3356618881225586
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.18303973972797394,
-      "val_loss": 0.24963340759277344,
-      "train_l0": 4.515594244003296,
-      "val_l0": 3.8955163955688477
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.12862856686115265,
-      "val_loss": 0.13225936889648438,
-      "train_l0": 3.8597404956817627,
-      "val_l0": 3.6971187591552734
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.21040086448192596,
-      "val_loss": 0.20606117248535155,
-      "train_l0": 3.616267442703247,
-      "val_l0": 4.238801002502441
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.14177893102169037,
-      "val_loss": 0.18520660400390626,
-      "train_l0": 3.515559434890747,
-      "val_l0": 3.57072114944458
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.28014224767684937,
-      "val_loss": 0.2064342498779297,
-      "train_l0": 3.5116195678710938,
-      "val_l0": 3.8475394248962402
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.18489107489585876,
-      "val_loss": 0.14811325073242188,
-      "train_l0": 3.6059200763702393,
-      "val_l0": 3.4544014930725098
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.18512320518493652,
-      "val_loss": 0.1354084014892578,
-      "train_l0": 3.0777156352996826,
-      "val_l0": 3.6293387413024902
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.222157821059227,
-      "val_loss": 0.15824012756347655,
-      "train_l0": 4.075437784194946,
-      "val_l0": 3.5176491737365723
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.1522722989320755,
-      "val_loss": 0.1371082305908203,
-      "train_l0": 3.7998318672180176,
-      "val_l0": 3.5025715827941895
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.2167949080467224,
-      "val_loss": 0.1713542938232422,
-      "train_l0": 3.375786542892456,
-      "val_l0": 3.55027437210083
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 3.375786542892456
-  }
-}

metrics/metrics_L12.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 12,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.049761027097702026,
-      "val_loss": 0.23586273193359375,
-      "train_l0": 5.280417203903198,
-      "val_l0": 5.052521228790283
-    },
-    {
-      "step": 400,
-      "train_loss": 0.06634508073329926,
-      "val_loss": 0.3154510498046875,
-      "train_l0": 4.09318208694458,
-      "val_l0": 5.063436031341553
-    },
-    {
-      "step": 600,
-      "train_loss": 0.13330385088920593,
-      "val_loss": 0.191497802734375,
-      "train_l0": 4.017609357833862,
-      "val_l0": 4.022650718688965
-    },
-    {
-      "step": 800,
-      "train_loss": 0.21025627851486206,
-      "val_loss": 0.23423538208007813,
-      "train_l0": 3.789740800857544,
-      "val_l0": 4.507575035095215
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.20364870131015778,
-      "val_loss": 0.23608016967773438,
-      "train_l0": 3.9293766021728516,
-      "val_l0": 3.7171173095703125
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.21887055039405823,
-      "val_loss": 0.070147705078125,
-      "train_l0": 4.024946689605713,
-      "val_l0": 3.698732852935791
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.00285144941881299,
-      "val_loss": 0.17227325439453126,
-      "train_l0": 3.424978256225586,
-      "val_l0": 3.398251533508301
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.25717073678970337,
-      "val_loss": 0.20127410888671876,
-      "train_l0": 3.671365976333618,
-      "val_l0": 3.7819719314575195
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.0787108838558197,
-      "val_loss": 0.16406211853027344,
-      "train_l0": 3.618347644805908,
-      "val_l0": 3.638465404510498
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.0804997980594635,
-      "val_loss": 0.2432525634765625,
-      "train_l0": 3.5273313522338867,
-      "val_l0": 3.691577911376953
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.16888681054115295,
-      "val_loss": 0.05645675659179687,
-      "train_l0": 3.3850371837615967,
-      "val_l0": 3.2377266883850098
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.05746965855360031,
-      "val_loss": 0.21165084838867188,
-      "train_l0": 3.18300724029541,
-      "val_l0": 3.4363770484924316
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.07457253336906433,
-      "val_loss": 0.11435813903808593,
-      "train_l0": 3.731149435043335,
-      "val_l0": 3.5162997245788574
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.18094214797019958,
-      "val_loss": 0.14759597778320313,
-      "train_l0": 3.5462021827697754,
-      "val_l0": 3.5747838020324707
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.12367279827594757,
-      "val_loss": 0.19666061401367188,
-      "train_l0": 3.444838523864746,
-      "val_l0": 3.707578182220459
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.1919580101966858,
-      "val_loss": 0.167431640625,
-      "train_l0": 3.6078989505767822,
-      "val_l0": 3.445577621459961
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.08584243804216385,
-      "val_loss": 0.0935699462890625,
-      "train_l0": 3.5036087036132812,
-      "val_l0": 3.372669219970703
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.004086595959961414,
-      "val_loss": 0.1779205322265625,
-      "train_l0": 2.940523624420166,
-      "val_l0": 3.386509418487549
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.07857616245746613,
-      "val_loss": 0.1642467498779297,
-      "train_l0": 3.4715771675109863,
-      "val_l0": 4.133481979370117
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.18264088034629822,
-      "val_loss": 0.15473709106445313,
-      "train_l0": 3.339332342147827,
-      "val_l0": 3.5804057121276855
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.07045550644397736,
-      "val_loss": 0.08549880981445312,
-      "train_l0": 3.107649087905884,
-      "val_l0": 3.382086753845215
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.09806112945079803,
-      "val_loss": 0.271417236328125,
-      "train_l0": 3.3613860607147217,
-      "val_l0": 4.061157703399658
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.2173023223876953,
-      "val_loss": 0.1629016876220703,
-      "train_l0": 3.610020875930786,
-      "val_l0": 3.5916566848754883
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.059717610478401184,
-      "val_loss": 0.0760498046875,
-      "train_l0": 3.1996190547943115,
-      "val_l0": 3.4174418449401855
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.06465455144643784,
-      "val_loss": 0.109918212890625,
-      "train_l0": 3.100919723510742,
-      "val_l0": 3.3136749267578125
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 3.100919723510742
-  }
-}

metrics/metrics_L13.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 13,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.22395287454128265,
-      "val_loss": 0.2162261962890625,
-      "train_l0": 6.002742052078247,
-      "val_l0": 6.3762688636779785
-    },
-    {
-      "step": 400,
-      "train_loss": 0.23106394708156586,
-      "val_loss": 0.17551498413085936,
-      "train_l0": 4.743325710296631,
-      "val_l0": 5.017955303192139
-    },
-    {
-      "step": 600,
-      "train_loss": 0.05049969255924225,
-      "val_loss": 0.34801483154296875,
-      "train_l0": 4.119223356246948,
-      "val_l0": 5.083448886871338
-    },
-    {
-      "step": 800,
-      "train_loss": 0.258017361164093,
-      "val_loss": 0.27061996459960935,
-      "train_l0": 4.910182952880859,
-      "val_l0": 4.004611968994141
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.007300457917153835,
-      "val_loss": 0.06269607543945313,
-      "train_l0": 4.506659507751465,
-      "val_l0": 4.057683944702148
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.0030020377598702908,
-      "val_loss": 0.15885467529296876,
-      "train_l0": 4.197508096694946,
-      "val_l0": 4.219772815704346
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.23780347406864166,
-      "val_loss": 0.14090652465820314,
-      "train_l0": 4.0885210037231445,
-      "val_l0": 4.079396724700928
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.30215391516685486,
-      "val_loss": 0.1140838623046875,
-      "train_l0": 4.01490330696106,
-      "val_l0": 4.2679572105407715
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.19365569949150085,
-      "val_loss": 0.21255874633789062,
-      "train_l0": 4.0745556354522705,
-      "val_l0": 4.438607692718506
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.13495495915412903,
-      "val_loss": 0.08930130004882812,
-      "train_l0": 4.057210683822632,
-      "val_l0": 4.2247819900512695
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.17033696174621582,
-      "val_loss": 0.229937744140625,
-      "train_l0": 4.224717617034912,
-      "val_l0": 4.258725643157959
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.21462956070899963,
-      "val_loss": 0.060570907592773435,
-      "train_l0": 4.026174545288086,
-      "val_l0": 4.24569845199585
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.19176000356674194,
-      "val_loss": 0.17860031127929688,
-      "train_l0": 4.244852066040039,
-      "val_l0": 4.014825820922852
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.0026799733750522137,
-      "val_loss": 0.051315689086914064,
-      "train_l0": 3.9683520793914795,
-      "val_l0": 4.107019901275635
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.08866433054208755,
-      "val_loss": 0.17926101684570311,
-      "train_l0": 3.4518003463745117,
-      "val_l0": 3.758378028869629
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.040845200419425964,
-      "val_loss": 0.04536628723144531,
-      "train_l0": 3.974735736846924,
-      "val_l0": 3.6425113677978516
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.16504868865013123,
-      "val_loss": 0.18426551818847656,
-      "train_l0": 3.9739787578582764,
-      "val_l0": 4.038228988647461
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.156017005443573,
-      "val_loss": 0.15966224670410156,
-      "train_l0": 3.950601816177368,
-      "val_l0": 4.000825881958008
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.08060595393180847,
-      "val_loss": 0.16868095397949218,
-      "train_l0": 4.025626182556152,
-      "val_l0": 3.8591480255126953
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.16479790210723877,
-      "val_loss": 0.15739707946777343,
-      "train_l0": 3.9172887802124023,
-      "val_l0": 4.119517803192139
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.1623796820640564,
-      "val_loss": 0.15610275268554688,
-      "train_l0": 4.180437326431274,
-      "val_l0": 4.0860772132873535
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.14870871603488922,
-      "val_loss": 0.11494178771972656,
-      "train_l0": 3.824734687805176,
-      "val_l0": 3.9145278930664062
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.1420278698205948,
-      "val_loss": 0.07527694702148438,
-      "train_l0": 3.898346424102783,
-      "val_l0": 3.617892265319824
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.15156535804271698,
-      "val_loss": 0.12783889770507811,
-      "train_l0": 3.684896230697632,
-      "val_l0": 3.678267002105713
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.1299048811197281,
-      "val_loss": 0.11329269409179688,
-      "train_l0": 4.597461223602295,
-      "val_l0": 3.92117977142334
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 4.597461223602295
-  }
-}

metrics/metrics_L14.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 14,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.2268056422472,
-      "val_loss": 0.2205352783203125,
-      "train_l0": 5.908143520355225,
-      "val_l0": 5.960733890533447
-    },
-    {
-      "step": 400,
-      "train_loss": 0.3876601755619049,
-      "val_loss": 0.136370849609375,
-      "train_l0": 5.088132619857788,
-      "val_l0": 5.3023481369018555
-    },
-    {
-      "step": 600,
-      "train_loss": 0.05355757474899292,
-      "val_loss": 0.24253158569335936,
-      "train_l0": 5.041038990020752,
-      "val_l0": 4.485607147216797
-    },
-    {
-      "step": 800,
-      "train_loss": 0.07663963735103607,
-      "val_loss": 0.17482070922851561,
-      "train_l0": 4.319339990615845,
-      "val_l0": 4.566833972930908
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.17825694382190704,
-      "val_loss": 0.21924819946289062,
-      "train_l0": 4.110926389694214,
-      "val_l0": 4.370577335357666
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.3518032431602478,
-      "val_loss": 0.22976531982421874,
-      "train_l0": 4.49526309967041,
-      "val_l0": 4.418649673461914
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.003465613117441535,
-      "val_loss": 0.06223297119140625,
-      "train_l0": 4.507356882095337,
-      "val_l0": 4.193181991577148
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.12616412341594696,
-      "val_loss": 0.039304351806640624,
-      "train_l0": 4.681652784347534,
-      "val_l0": 4.043710231781006
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.1261879801750183,
-      "val_loss": 0.23475570678710939,
-      "train_l0": 3.8393139839172363,
-      "val_l0": 4.266026020050049
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.2797168493270874,
-      "val_loss": 0.21379547119140624,
-      "train_l0": 4.066145420074463,
-      "val_l0": 4.256992340087891
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.04294784367084503,
-      "val_loss": 0.18767662048339845,
-      "train_l0": 3.8974225521087646,
-      "val_l0": 4.089410305023193
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.14952577650547028,
-      "val_loss": 0.25541229248046876,
-      "train_l0": 3.974062204360962,
-      "val_l0": 4.064152240753174
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.18908026814460754,
-      "val_loss": 0.09503936767578125,
-      "train_l0": 4.12212610244751,
-      "val_l0": 3.885359764099121
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.026669470593333244,
-      "val_loss": 0.23632049560546875,
-      "train_l0": 3.7041962146759033,
-      "val_l0": 4.14576530456543
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.10926380008459091,
-      "val_loss": 0.1588043212890625,
-      "train_l0": 4.155206680297852,
-      "val_l0": 4.273812770843506
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.07134613394737244,
-      "val_loss": 0.18201751708984376,
-      "train_l0": 3.8694679737091064,
-      "val_l0": 3.9543890953063965
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.1773558259010315,
-      "val_loss": 0.11385307312011719,
-      "train_l0": 4.206365346908569,
-      "val_l0": 3.995187282562256
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.10871031135320663,
-      "val_loss": 0.1764965057373047,
-      "train_l0": 3.9139270782470703,
-      "val_l0": 4.0827155113220215
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.14885877072811127,
-      "val_loss": 0.17030410766601561,
-      "train_l0": 4.062902927398682,
-      "val_l0": 4.158968925476074
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.11427878588438034,
-      "val_loss": 0.2960807800292969,
-      "train_l0": 4.0442705154418945,
-      "val_l0": 4.237833023071289
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.2316785454750061,
-      "val_loss": 0.13748703002929688,
-      "train_l0": 3.6500871181488037,
-      "val_l0": 3.6406779289245605
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.05487024039030075,
-      "val_loss": 0.30532073974609375,
-      "train_l0": 3.9902210235595703,
-      "val_l0": 4.303326606750488
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.07683736085891724,
-      "val_loss": 0.11914253234863281,
-      "train_l0": 3.806394338607788,
-      "val_l0": 3.947298526763916
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.028781507164239883,
-      "val_loss": 0.13880653381347657,
-      "train_l0": 3.7544071674346924,
-      "val_l0": 3.7887930870056152
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.1085590049624443,
-      "val_loss": 0.16082077026367186,
-      "train_l0": 3.9828717708587646,
-      "val_l0": 4.116208553314209
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 3.9828717708587646
-  }
-}

metrics/metrics_L15.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 15,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.18859589099884033,
-      "val_loss": 0.38094482421875,
-      "train_l0": 6.112903356552124,
-      "val_l0": 7.063748836517334
-    },
-    {
-      "step": 400,
-      "train_loss": 0.12662295997142792,
-      "val_loss": 0.24796295166015625,
-      "train_l0": 6.68032169342041,
-      "val_l0": 6.632809638977051
-    },
-    {
-      "step": 600,
-      "train_loss": 0.12376363575458527,
-      "val_loss": 0.12600250244140626,
-      "train_l0": 5.3765058517456055,
-      "val_l0": 5.7402729988098145
-    },
-    {
-      "step": 800,
-      "train_loss": 0.12376091629266739,
-      "val_loss": 0.11396255493164062,
-      "train_l0": 5.408501625061035,
-      "val_l0": 5.558779239654541
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.15942886471748352,
-      "val_loss": 0.20974884033203126,
-      "train_l0": 5.766826868057251,
-      "val_l0": 4.799845218658447
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.2195884734392166,
-      "val_loss": 0.2046783447265625,
-      "train_l0": 4.670381546020508,
-      "val_l0": 4.87600564956665
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.1488950401544571,
-      "val_loss": 0.19315032958984374,
-      "train_l0": 5.135905742645264,
-      "val_l0": 5.232398509979248
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.16407719254493713,
-      "val_loss": 0.2600120544433594,
-      "train_l0": 4.8685431480407715,
-      "val_l0": 4.9507832527160645
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.28617897629737854,
-      "val_loss": 0.117840576171875,
-      "train_l0": 4.916584491729736,
-      "val_l0": 4.805908203125
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.26199525594711304,
-      "val_loss": 0.23673477172851562,
-      "train_l0": 4.759585857391357,
-      "val_l0": 5.2588605880737305
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.2180166393518448,
-      "val_loss": 0.05423583984375,
-      "train_l0": 5.22952675819397,
-      "val_l0": 5.001626014709473
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.0030850819312036037,
-      "val_loss": 0.11976394653320313,
-      "train_l0": 4.7512829303741455,
-      "val_l0": 5.086400508880615
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.09626377373933792,
-      "val_loss": 0.20292587280273439,
-      "train_l0": 5.210590362548828,
-      "val_l0": 4.75818395614624
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.05881235748529434,
-      "val_loss": 0.14430694580078124,
-      "train_l0": 4.784178733825684,
-      "val_l0": 4.971439838409424
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.1566220223903656,
-      "val_loss": 0.1758941650390625,
-      "train_l0": 5.287277698516846,
-      "val_l0": 4.890756607055664
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.1553994119167328,
-      "val_loss": 0.1737762451171875,
-      "train_l0": 4.770141839981079,
-      "val_l0": 4.963803291320801
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.25802457332611084,
-      "val_loss": 0.15414581298828126,
-      "train_l0": 4.83241081237793,
-      "val_l0": 4.572384357452393
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.09250403195619583,
-      "val_loss": 0.17729110717773439,
-      "train_l0": 5.365818738937378,
-      "val_l0": 4.858424663543701
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.17442752420902252,
-      "val_loss": 0.212823486328125,
-      "train_l0": 5.365270376205444,
-      "val_l0": 5.093202590942383
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.3724280595779419,
-      "val_loss": 0.12553634643554687,
-      "train_l0": 5.029004812240601,
-      "val_l0": 4.84776496887207
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.10034391283988953,
-      "val_loss": 0.10605010986328126,
-      "train_l0": 4.71007227897644,
-      "val_l0": 5.033133029937744
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.26655513048171997,
-      "val_loss": 0.0655670166015625,
-      "train_l0": 4.809176921844482,
-      "val_l0": 4.904501438140869
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.31244611740112305,
-      "val_loss": 0.11882476806640625,
-      "train_l0": 5.029934644699097,
-      "val_l0": 4.842426776885986
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.14197847247123718,
-      "val_loss": 0.16082687377929689,
-      "train_l0": 4.580521583557129,
-      "val_l0": 4.991309642791748
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.18068140745162964,
-      "val_loss": 0.15989227294921876,
-      "train_l0": 4.755055904388428,
-      "val_l0": 4.833583831787109
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 4.755055904388428
-  }
-}

metrics/metrics_L16.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 16,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 2.0147385597229004,
-      "val_loss": 1.483013916015625,
-      "train_l0": 8.8844895362854,
-      "val_l0": 8.844211101531982
-    },
-    {
-      "step": 400,
-      "train_loss": 0.7190955281257629,
-      "val_loss": 0.6447174072265625,
-      "train_l0": 7.325559854507446,
-      "val_l0": 7.473795413970947
-    },
-    {
-      "step": 600,
-      "train_loss": 0.43909192085266113,
-      "val_loss": 0.7190582275390625,
-      "train_l0": 7.045167684555054,
-      "val_l0": 7.293331623077393
-    },
-    {
-      "step": 800,
-      "train_loss": 0.1277594417333603,
-      "val_loss": 1.17286376953125,
-      "train_l0": 6.979811191558838,
-      "val_l0": 7.326333522796631
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.536592423915863,
-      "val_loss": 0.3073944091796875,
-      "train_l0": 7.271707057952881,
-      "val_l0": 7.446353435516357
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.43519431352615356,
-      "val_loss": 0.53253173828125,
-      "train_l0": 7.412272691726685,
-      "val_l0": 7.189347743988037
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.9889852404594421,
-      "val_loss": 0.333416748046875,
-      "train_l0": 7.545489072799683,
-      "val_l0": 7.190375328063965
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.35724973678588867,
-      "val_loss": 0.5863739013671875,
-      "train_l0": 7.315361499786377,
-      "val_l0": 7.307224273681641
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.6714401245117188,
-      "val_loss": 0.09823760986328126,
-      "train_l0": 6.85620903968811,
-      "val_l0": 6.532306671142578
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.1753266155719757,
-      "val_loss": 0.565533447265625,
-      "train_l0": 6.568515300750732,
-      "val_l0": 7.1772027015686035
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.07969269901514053,
-      "val_loss": 0.5308624267578125,
-      "train_l0": 6.443041563034058,
-      "val_l0": 7.251532077789307
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.589357852935791,
-      "val_loss": 0.402227783203125,
-      "train_l0": 7.125425338745117,
-      "val_l0": 6.952426433563232
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.612260103225708,
-      "val_loss": 0.501318359375,
-      "train_l0": 7.279670238494873,
-      "val_l0": 7.028241157531738
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.03864092379808426,
-      "val_loss": 0.07253875732421874,
-      "train_l0": 7.043129205703735,
-      "val_l0": 7.216846942901611
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.04911990463733673,
-      "val_loss": 0.29129486083984374,
-      "train_l0": 6.400692462921143,
-      "val_l0": 7.224123477935791
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.4463154673576355,
-      "val_loss": 0.453875732421875,
-      "train_l0": 7.75114893913269,
-      "val_l0": 7.297854423522949
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.29518067836761475,
-      "val_loss": 0.43136444091796877,
-      "train_l0": 7.203382253646851,
-      "val_l0": 7.667086124420166
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.3541266918182373,
-      "val_loss": 0.304974365234375,
-      "train_l0": 7.325237989425659,
-      "val_l0": 7.291569709777832
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.6822983026504517,
-      "val_loss": 0.177850341796875,
-      "train_l0": 7.492810487747192,
-      "val_l0": 7.257242202758789
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.8102273941040039,
-      "val_loss": 0.4614349365234375,
-      "train_l0": 7.881307601928711,
-      "val_l0": 7.399499416351318
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.0868353396654129,
-      "val_loss": 0.31473388671875,
-      "train_l0": 7.5837016105651855,
-      "val_l0": 7.386074066162109
-    },
-    {
-      "step": 4400,
-      "train_loss": 1.0328900814056396,
-      "val_loss": 0.6586456298828125,
-      "train_l0": 7.588446140289307,
-      "val_l0": 8.125596046447754
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.5718325972557068,
-      "val_loss": 0.5662017822265625,
-      "train_l0": 7.29985237121582,
-      "val_l0": 7.754888534545898
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.3163934350013733,
-      "val_loss": 0.27583770751953124,
-      "train_l0": 7.40966796875,
-      "val_l0": 7.692563533782959
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.03988220542669296,
-      "val_loss": 0.34824371337890625,
-      "train_l0": 7.601940631866455,
-      "val_l0": 7.565097808837891
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 7.601940631866455
-  }
-}

metrics/metrics_L17.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 17,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.29547742009162903,
-      "val_loss": 0.25211181640625,
-      "train_l0": 7.947826385498047,
-      "val_l0": 8.013341426849365
-    },
-    {
-      "step": 400,
-      "train_loss": 0.18066932260990143,
-      "val_loss": 0.2115203857421875,
-      "train_l0": 7.252681255340576,
-      "val_l0": 7.373335361480713
-    },
-    {
-      "step": 600,
-      "train_loss": 0.1320885419845581,
-      "val_loss": 0.260302734375,
-      "train_l0": 7.383096218109131,
-      "val_l0": 7.235217094421387
-    },
-    {
-      "step": 800,
-      "train_loss": 0.14655840396881104,
-      "val_loss": 0.2014129638671875,
-      "train_l0": 6.49915337562561,
-      "val_l0": 6.597681045532227
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.08175850659608841,
-      "val_loss": 0.25476531982421874,
-      "train_l0": 6.446212530136108,
-      "val_l0": 6.738440990447998
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.27209439873695374,
-      "val_loss": 0.209674072265625,
-      "train_l0": 6.320023536682129,
-      "val_l0": 6.909973621368408
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.6252310276031494,
-      "val_loss": 0.1320404052734375,
-      "train_l0": 7.282721996307373,
-      "val_l0": 6.144618988037109
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.004336773417890072,
-      "val_loss": 0.17270050048828126,
-      "train_l0": 5.782961845397949,
-      "val_l0": 6.5926384925842285
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.15362975001335144,
-      "val_loss": 0.1113372802734375,
-      "train_l0": 6.0371458530426025,
-      "val_l0": 5.952432155609131
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.10350187122821808,
-      "val_loss": 0.13460273742675782,
-      "train_l0": 5.82546591758728,
-      "val_l0": 6.116945743560791
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.0679798498749733,
-      "val_loss": 0.22560882568359375,
-      "train_l0": 5.661338567733765,
-      "val_l0": 6.385469436645508
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.3262917101383209,
-      "val_loss": 0.150030517578125,
-      "train_l0": 6.516504287719727,
-      "val_l0": 6.351232528686523
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.0715227946639061,
-      "val_loss": 0.28294525146484373,
-      "train_l0": 5.712169408798218,
-      "val_l0": 6.372239589691162
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.0030827573500573635,
-      "val_loss": 0.10286483764648438,
-      "train_l0": 5.119132995605469,
-      "val_l0": 5.68516731262207
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.05433140695095062,
-      "val_loss": 0.2620574951171875,
-      "train_l0": 5.781108140945435,
-      "val_l0": 6.322429180145264
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.18326455354690552,
-      "val_loss": 0.1171630859375,
-      "train_l0": 5.858641862869263,
-      "val_l0": 5.607466697692871
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.1755848228931427,
-      "val_loss": 0.11911773681640625,
-      "train_l0": 6.564021110534668,
-      "val_l0": 5.932223796844482
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.0037277420051395893,
-      "val_loss": 0.21335678100585936,
-      "train_l0": 5.113232135772705,
-      "val_l0": 6.149611473083496
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.10639315843582153,
-      "val_loss": 0.23836593627929686,
-      "train_l0": 6.345129013061523,
-      "val_l0": 5.920286178588867
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.39258724451065063,
-      "val_loss": 0.1180694580078125,
-      "train_l0": 6.1448752880096436,
-      "val_l0": 5.517294406890869
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.419334352016449,
-      "val_loss": 0.126666259765625,
-      "train_l0": 6.489890813827515,
-      "val_l0": 5.608823299407959
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.2981112599372864,
-      "val_loss": 0.06766433715820312,
-      "train_l0": 6.16682767868042,
-      "val_l0": 5.496656894683838
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.10180480033159256,
-      "val_loss": 0.11992645263671875,
-      "train_l0": 5.702310800552368,
-      "val_l0": 5.680327415466309
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.003014140296727419,
-      "val_loss": 0.12350234985351563,
-      "train_l0": 5.111187696456909,
-      "val_l0": 5.841434001922607
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.15246543288230896,
-      "val_loss": 0.09812698364257813,
-      "train_l0": 5.976772308349609,
-      "val_l0": 5.278291702270508
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 5.976772308349609
-  }
-}

metrics/metrics_L18.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 18,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.18153195083141327,
-      "val_loss": 0.1036529541015625,
-      "train_l0": 9.34842824935913,
-      "val_l0": 9.272804260253906
-    },
-    {
-      "step": 400,
-      "train_loss": 0.06843207031488419,
-      "val_loss": 0.16837005615234374,
-      "train_l0": 8.522951602935791,
-      "val_l0": 8.68922472000122
-    },
-    {
-      "step": 600,
-      "train_loss": 0.12185792624950409,
-      "val_loss": 0.084832763671875,
-      "train_l0": 8.400404453277588,
-      "val_l0": 8.539772033691406
-    },
-    {
-      "step": 800,
-      "train_loss": 0.08921658992767334,
-      "val_loss": 0.21422805786132812,
-      "train_l0": 8.020985126495361,
-      "val_l0": 8.099546432495117
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.13305523991584778,
-      "val_loss": 0.15858001708984376,
-      "train_l0": 7.833904027938843,
-      "val_l0": 7.949264049530029
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.05351705476641655,
-      "val_loss": 0.1508392333984375,
-      "train_l0": 7.462650537490845,
-      "val_l0": 7.575962543487549
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.35643118619918823,
-      "val_loss": 0.11534194946289063,
-      "train_l0": 7.494688034057617,
-      "val_l0": 7.154510021209717
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.003675672458484769,
-      "val_loss": 0.169952392578125,
-      "train_l0": 6.1886608600616455,
-      "val_l0": 7.263972759246826
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.13229785859584808,
-      "val_loss": 0.08189620971679687,
-      "train_l0": 6.978583335876465,
-      "val_l0": 6.645839214324951
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.14124521613121033,
-      "val_loss": 0.1912933349609375,
-      "train_l0": 5.8920323848724365,
-      "val_l0": 5.938117504119873
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.08672825247049332,
-      "val_loss": 0.1457122802734375,
-      "train_l0": 6.410008668899536,
-      "val_l0": 6.610515117645264
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.1326695680618286,
-      "val_loss": 0.120147705078125,
-      "train_l0": 7.090938091278076,
-      "val_l0": 6.916561126708984
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.003633265383541584,
-      "val_loss": 0.16397628784179688,
-      "train_l0": 6.988579034805298,
-      "val_l0": 7.21776008605957
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.11385974287986755,
-      "val_loss": 0.07097244262695312,
-      "train_l0": 6.327402591705322,
-      "val_l0": 6.191682815551758
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.03426986560225487,
-      "val_loss": 0.0904296875,
-      "train_l0": 6.470334529876709,
-      "val_l0": 6.5923333168029785
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.05054966360330582,
-      "val_loss": 0.17446975708007811,
-      "train_l0": 5.808448791503906,
-      "val_l0": 6.333284378051758
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.07079701125621796,
-      "val_loss": 0.12457427978515626,
-      "train_l0": 5.410349369049072,
-      "val_l0": 6.60409688949585
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.002599747385829687,
-      "val_loss": 0.047154998779296874,
-      "train_l0": 5.90897798538208,
-      "val_l0": 6.068098545074463
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.07271479815244675,
-      "val_loss": 0.13225936889648438,
-      "train_l0": 5.689173936843872,
-      "val_l0": 6.391794681549072
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.05849483609199524,
-      "val_loss": 0.14538917541503907,
-      "train_l0": 6.572252511978149,
-      "val_l0": 6.945149898529053
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.0371062308549881,
-      "val_loss": 0.11444931030273438,
-      "train_l0": 5.965441465377808,
-      "val_l0": 7.010195255279541
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.043126996606588364,
-      "val_loss": 0.0837158203125,
-      "train_l0": 6.728255748748779,
-      "val_l0": 6.374139785766602
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.06577989459037781,
-      "val_loss": 0.07768783569335938,
-      "train_l0": 6.2803566455841064,
-      "val_l0": 6.004130840301514
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.13661718368530273,
-      "val_loss": 0.08974533081054688,
-      "train_l0": 6.767857074737549,
-      "val_l0": 6.150918006896973
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.02224155329167843,
-      "val_loss": 0.08356552124023438,
-      "train_l0": 6.570655107498169,
-      "val_l0": 6.584842205047607
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 6.570655107498169
-  }
-}

metrics/metrics_L19.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 19,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.16328153014183044,
-      "val_loss": 0.7614501953125,
-      "train_l0": 9.410011768341064,
-      "val_l0": 9.672513008117676
-    },
-    {
-      "step": 400,
-      "train_loss": 1.106956958770752,
-      "val_loss": 0.285394287109375,
-      "train_l0": 9.058338403701782,
-      "val_l0": 8.911561965942383
-    },
-    {
-      "step": 600,
-      "train_loss": 0.3582995533943176,
-      "val_loss": 0.152655029296875,
-      "train_l0": 8.900266885757446,
-      "val_l0": 8.851428031921387
-    },
-    {
-      "step": 800,
-      "train_loss": 0.3630158007144928,
-      "val_loss": 0.3028289794921875,
-      "train_l0": 8.60481858253479,
-      "val_l0": 8.673374652862549
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.24467700719833374,
-      "val_loss": 0.3413848876953125,
-      "train_l0": 8.545207977294922,
-      "val_l0": 8.71147632598877
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.08691535890102386,
-      "val_loss": 0.1753326416015625,
-      "train_l0": 8.676499128341675,
-      "val_l0": 8.312342166900635
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.3015482425689697,
-      "val_loss": 0.229278564453125,
-      "train_l0": 8.302879333496094,
-      "val_l0": 8.557393550872803
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.20741379261016846,
-      "val_loss": 0.165313720703125,
-      "train_l0": 8.581340312957764,
-      "val_l0": 7.655327320098877
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.3870965242385864,
-      "val_loss": 0.23432464599609376,
-      "train_l0": 8.170473575592041,
-      "val_l0": 7.911219596862793
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.19583956897258759,
-      "val_loss": 0.3945159912109375,
-      "train_l0": 7.29408860206604,
-      "val_l0": 8.134562969207764
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.2413194626569748,
-      "val_loss": 0.175360107421875,
-      "train_l0": 8.293479681015015,
-      "val_l0": 7.956058979034424
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.235012024641037,
-      "val_loss": 0.23008270263671876,
-      "train_l0": 7.574540376663208,
-      "val_l0": 7.818315029144287
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.32752957940101624,
-      "val_loss": 0.213995361328125,
-      "train_l0": 7.984650135040283,
-      "val_l0": 8.027493953704834
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.33268076181411743,
-      "val_loss": 0.2492584228515625,
-      "train_l0": 7.781195640563965,
-      "val_l0": 7.636430263519287
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.10630679130554199,
-      "val_loss": 0.294158935546875,
-      "train_l0": 7.874828577041626,
-      "val_l0": 7.7915120124816895
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.3239853084087372,
-      "val_loss": 0.3273040771484375,
-      "train_l0": 8.166217803955078,
-      "val_l0": 7.872045040130615
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.10050178319215775,
-      "val_loss": 0.2894775390625,
-      "train_l0": 7.126861810684204,
-      "val_l0": 8.002984523773193
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.2620861530303955,
-      "val_loss": 0.2463470458984375,
-      "train_l0": 7.320934534072876,
-      "val_l0": 7.738144397735596
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.1980583518743515,
-      "val_loss": 0.14094696044921876,
-      "train_l0": 7.894331216812134,
-      "val_l0": 7.874355316162109
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.3361251950263977,
-      "val_loss": 0.10409698486328126,
-      "train_l0": 7.724350690841675,
-      "val_l0": 7.3462605476379395
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.18273675441741943,
-      "val_loss": 0.2894775390625,
-      "train_l0": 7.522475719451904,
-      "val_l0": 7.949066162109375
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.055990416556596756,
-      "val_loss": 0.2091644287109375,
-      "train_l0": 7.712060213088989,
-      "val_l0": 8.096511363983154
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.2383211851119995,
-      "val_loss": 0.24113311767578124,
-      "train_l0": 8.246296644210815,
-      "val_l0": 8.061425685882568
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.12159339338541031,
-      "val_loss": 0.23677520751953124,
-      "train_l0": 7.3075175285339355,
-      "val_l0": 7.9546308517456055
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.13588812947273254,
-      "val_loss": 0.244696044921875,
-      "train_l0": 7.477247714996338,
-      "val_l0": 8.267624378204346
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 7.477247714996338
-  }
-}

metrics/metrics_L2.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 2,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 1.9761638641357422,
-      "val_loss": 3.1903717041015627,
-      "train_l0": 6.391435861587524,
-      "val_l0": 6.662788391113281
-    },
-    {
-      "step": 400,
-      "train_loss": 1.557054042816162,
-      "val_loss": 1.2012100219726562,
-      "train_l0": 7.713514566421509,
-      "val_l0": 6.247289180755615
-    },
-    {
-      "step": 600,
-      "train_loss": 0.033284783363342285,
-      "val_loss": 1.2727691650390625,
-      "train_l0": 6.77381157875061,
-      "val_l0": 6.737117767333984
-    },
-    {
-      "step": 800,
-      "train_loss": 2.1294851303100586,
-      "val_loss": 2.918695068359375,
-      "train_l0": 3.5417914390563965,
-      "val_l0": 5.333240032196045
-    },
-    {
-      "step": 1000,
-      "train_loss": 1.4643634557724,
-      "val_loss": 1.5200180053710937,
-      "train_l0": 5.513817071914673,
-      "val_l0": 5.526895523071289
-    },
-    {
-      "step": 1200,
-      "train_loss": 1.1451902389526367,
-      "val_loss": 2.2034011840820313,
-      "train_l0": 7.441985607147217,
-      "val_l0": 6.193974018096924
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.3887673616409302,
-      "val_loss": 1.6567428588867188,
-      "train_l0": 5.183881521224976,
-      "val_l0": 5.194427967071533
-    },
-    {
-      "step": 1600,
-      "train_loss": 1.1516119241714478,
-      "val_loss": 2.549102783203125,
-      "train_l0": 6.440049409866333,
-      "val_l0": 7.178695201873779
-    },
-    {
-      "step": 1800,
-      "train_loss": 1.101319670677185,
-      "val_loss": 3.256329345703125,
-      "train_l0": 6.867629289627075,
-      "val_l0": 6.555483341217041
-    },
-    {
-      "step": 2000,
-      "train_loss": 1.2290267944335938,
-      "val_loss": 2.0247894287109376,
-      "train_l0": 5.375576019287109,
-      "val_l0": 6.489918231964111
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.4021357297897339,
-      "val_loss": 0.9072250366210938,
-      "train_l0": 5.079847574234009,
-      "val_l0": 6.635992527008057
-    },
-    {
-      "step": 2400,
-      "train_loss": 1.7660073041915894,
-      "val_loss": 2.47327880859375,
-      "train_l0": 4.132765531539917,
-      "val_l0": 5.481312274932861
-    },
-    {
-      "step": 2600,
-      "train_loss": 2.505723476409912,
-      "val_loss": 1.3227386474609375,
-      "train_l0": 7.045191526412964,
-      "val_l0": 5.382688045501709
-    },
-    {
-      "step": 2800,
-      "train_loss": 2.1010913848876953,
-      "val_loss": 1.9941879272460938,
-      "train_l0": 4.841649532318115,
-      "val_l0": 4.034926891326904
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.8153737783432007,
-      "val_loss": 2.0456832885742187,
-      "train_l0": 4.412078857421875,
-      "val_l0": 4.26976203918457
-    },
-    {
-      "step": 3200,
-      "train_loss": 1.7816202640533447,
-      "val_loss": 1.3784133911132812,
-      "train_l0": 3.1894147396087646,
-      "val_l0": 6.235237121582031
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.07173456251621246,
-      "val_loss": 2.302900695800781,
-      "train_l0": 8.204871416091919,
-      "val_l0": 5.319571495056152
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.8552208542823792,
-      "val_loss": 1.7994430541992188,
-      "train_l0": 4.5390307903289795,
-      "val_l0": 5.086700916290283
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.381530225276947,
-      "val_loss": 1.0194320678710938,
-      "train_l0": 4.763072729110718,
-      "val_l0": 5.433690547943115
-    },
-    {
-      "step": 4000,
-      "train_loss": 1.466295599937439,
-      "val_loss": 1.1989639282226563,
-      "train_l0": 5.863094329833984,
-      "val_l0": 6.461508274078369
-    },
-    {
-      "step": 4200,
-      "train_loss": 2.5544533729553223,
-      "val_loss": 1.8347068786621095,
-      "train_l0": 6.980335712432861,
-      "val_l0": 5.383918285369873
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.7380867004394531,
-      "val_loss": 0.8785980224609375,
-      "train_l0": 5.960804224014282,
-      "val_l0": 3.778097629547119
-    },
-    {
-      "step": 4600,
-      "train_loss": 1.5479521751403809,
-      "val_loss": 1.2941329956054688,
-      "train_l0": 4.40671443939209,
-      "val_l0": 4.870655536651611
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.5090218782424927,
-      "val_loss": 2.5589385986328126,
-      "train_l0": 5.473923683166504,
-      "val_l0": 5.626881122589111
-    },
-    {
-      "step": 5000,
-      "train_loss": 1.7641514539718628,
-      "val_loss": 1.7287353515625,
-      "train_l0": 4.126083850860596,
-      "val_l0": 5.8902812004089355
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 4.126083850860596
-  }
-}

metrics/metrics_L20.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 20,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.14699426293373108,
-      "val_loss": 0.13165283203125,
-      "train_l0": 10.582691431045532,
-      "val_l0": 10.451538562774658
-    },
-    {
-      "step": 400,
-      "train_loss": 0.23101815581321716,
-      "val_loss": 0.28314056396484377,
-      "train_l0": 9.41057801246643,
-      "val_l0": 9.788892269134521
-    },
-    {
-      "step": 600,
-      "train_loss": 0.08707526326179504,
-      "val_loss": 0.27347412109375,
-      "train_l0": 9.794217348098755,
-      "val_l0": 10.063333511352539
-    },
-    {
-      "step": 800,
-      "train_loss": 0.3588435649871826,
-      "val_loss": 0.2654388427734375,
-      "train_l0": 9.649884700775146,
-      "val_l0": 9.44399356842041
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.05659712851047516,
-      "val_loss": 0.28250961303710936,
-      "train_l0": 9.239715337753296,
-      "val_l0": 9.546241760253906
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.06258459389209747,
-      "val_loss": 0.28629226684570314,
-      "train_l0": 9.168636798858643,
-      "val_l0": 9.423484802246094
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.19844704866409302,
-      "val_loss": 0.0340789794921875,
-      "train_l0": 9.07091498374939,
-      "val_l0": 8.48935604095459
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.08194287866353989,
-      "val_loss": 0.312615966796875,
-      "train_l0": 8.271408081054688,
-      "val_l0": 8.831908702850342
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.10043966770172119,
-      "val_loss": 0.2164337158203125,
-      "train_l0": 7.838207483291626,
-      "val_l0": 8.3097243309021
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.09914621710777283,
-      "val_loss": 0.12376022338867188,
-      "train_l0": 7.564306259155273,
-      "val_l0": 8.117237091064453
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.0815201923251152,
-      "val_loss": 0.10303955078125,
-      "train_l0": 7.491922378540039,
-      "val_l0": 7.953670024871826
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.1623128354549408,
-      "val_loss": 0.08680648803710937,
-      "train_l0": 9.06481146812439,
-      "val_l0": 7.502641677856445
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.05493927001953125,
-      "val_loss": 0.116326904296875,
-      "train_l0": 7.673841714859009,
-      "val_l0": 7.588622570037842
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.18300625681877136,
-      "val_loss": 0.20911483764648436,
-      "train_l0": 8.141428232192993,
-      "val_l0": 8.038487434387207
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.33356937766075134,
-      "val_loss": 0.13643341064453124,
-      "train_l0": 8.726048469543457,
-      "val_l0": 8.04246187210083
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.1465834528207779,
-      "val_loss": 0.08868255615234374,
-      "train_l0": 7.541239261627197,
-      "val_l0": 7.17219352722168
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.1949060708284378,
-      "val_loss": 0.06849746704101563,
-      "train_l0": 8.290153741836548,
-      "val_l0": 7.0745158195495605
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.05165141075849533,
-      "val_loss": 0.07282562255859375,
-      "train_l0": 7.266104221343994,
-      "val_l0": 7.104558944702148
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.15942546725273132,
-      "val_loss": 0.15430068969726562,
-      "train_l0": 8.259528875350952,
-      "val_l0": 7.801253795623779
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.13976645469665527,
-      "val_loss": 0.15994873046875,
-      "train_l0": 7.7893078327178955,
-      "val_l0": 7.584846019744873
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.15116536617279053,
-      "val_loss": 0.08576812744140624,
-      "train_l0": 8.25345516204834,
-      "val_l0": 6.804840564727783
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.11699505150318146,
-      "val_loss": 0.087786865234375,
-      "train_l0": 7.164949178695679,
-      "val_l0": 7.242691516876221
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.10113253444433212,
-      "val_loss": 0.14269332885742186,
-      "train_l0": 6.981378793716431,
-      "val_l0": 7.064957618713379
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.05499845743179321,
-      "val_loss": 0.11155853271484376,
-      "train_l0": 6.736016273498535,
-      "val_l0": 7.270493507385254
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.11084996908903122,
-      "val_loss": 0.05323944091796875,
-      "train_l0": 7.761210203170776,
-      "val_l0": 6.732525825500488
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 7.761210203170776
-  }
-}

metrics/metrics_L21.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 21,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.01332792267203331,
-      "val_loss": 0.277655029296875,
-      "train_l0": 10.608041286468506,
-      "val_l0": 10.762674808502197
-    },
-    {
-      "step": 400,
-      "train_loss": 0.2547125518321991,
-      "val_loss": 0.3917236328125,
-      "train_l0": 10.508888959884644,
-      "val_l0": 10.694403648376465
-    },
-    {
-      "step": 600,
-      "train_loss": 0.1171446219086647,
-      "val_loss": 0.12552947998046876,
-      "train_l0": 9.497666358947754,
-      "val_l0": 9.637413024902344
-    },
-    {
-      "step": 800,
-      "train_loss": 0.09298652410507202,
-      "val_loss": 0.087249755859375,
-      "train_l0": 8.23696255683899,
-      "val_l0": 8.367600440979004
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.20712248980998993,
-      "val_loss": 0.20865402221679688,
-      "train_l0": 8.668595552444458,
-      "val_l0": 8.566527366638184
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.20033439993858337,
-      "val_loss": 0.1664215087890625,
-      "train_l0": 7.781904935836792,
-      "val_l0": 7.83083438873291
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.25378119945526123,
-      "val_loss": 0.13873748779296874,
-      "train_l0": 7.891845703125,
-      "val_l0": 7.767608165740967
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.2043895572423935,
-      "val_loss": 0.09123306274414063,
-      "train_l0": 8.173108100891113,
-      "val_l0": 7.316091060638428
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.2300933599472046,
-      "val_loss": 0.074798583984375,
-      "train_l0": 7.918494939804077,
-      "val_l0": 7.5612711906433105
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.16990616917610168,
-      "val_loss": 0.17627334594726562,
-      "train_l0": 7.038271427154541,
-      "val_l0": 6.913094520568848
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.3137889802455902,
-      "val_loss": 0.14666900634765626,
-      "train_l0": 7.959175109863281,
-      "val_l0": 7.098090648651123
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.08380964398384094,
-      "val_loss": 0.18900527954101562,
-      "train_l0": 6.956285238265991,
-      "val_l0": 7.64517068862915
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.06633605808019638,
-      "val_loss": 0.14645919799804688,
-      "train_l0": 5.499094724655151,
-      "val_l0": 6.773960590362549
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.13743820786476135,
-      "val_loss": 0.17334442138671874,
-      "train_l0": 7.4706971645355225,
-      "val_l0": 7.685177326202393
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.13275736570358276,
-      "val_loss": 0.09549102783203126,
-      "train_l0": 8.150416612625122,
-      "val_l0": 6.291041374206543
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.07135528326034546,
-      "val_loss": 0.12577743530273439,
-      "train_l0": 7.34713077545166,
-      "val_l0": 7.054803371429443
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.11701130121946335,
-      "val_loss": 0.051152801513671874,
-      "train_l0": 7.402437925338745,
-      "val_l0": 6.671254634857178
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.15087389945983887,
-      "val_loss": 0.1098663330078125,
-      "train_l0": 6.02908730506897,
-      "val_l0": 5.872364044189453
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.1514844447374344,
-      "val_loss": 0.0766357421875,
-      "train_l0": 6.948155164718628,
-      "val_l0": 6.3866496086120605
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.09939510375261307,
-      "val_loss": 0.07453231811523438,
-      "train_l0": 6.65438175201416,
-      "val_l0": 6.505246162414551
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.23903252184391022,
-      "val_loss": 0.09680633544921875,
-      "train_l0": 7.959979772567749,
-      "val_l0": 6.511647701263428
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.21320170164108276,
-      "val_loss": 0.07060699462890625,
-      "train_l0": 7.776623964309692,
-      "val_l0": 6.2432026863098145
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.16217657923698425,
-      "val_loss": 0.06054611206054687,
-      "train_l0": 7.350420951843262,
-      "val_l0": 6.335480213165283
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.04712393879890442,
-      "val_loss": 0.09563217163085938,
-      "train_l0": 5.406129360198975,
-      "val_l0": 6.729567050933838
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.04011683166027069,
-      "val_loss": 0.11249771118164062,
-      "train_l0": 5.306851863861084,
-      "val_l0": 6.852078437805176
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 5.306851863861084
-  }
-}

metrics/metrics_L22.json DELETED Viewed

@@ -1,185 +0,0 @@
-{
-  "layer": 22,
-  "metrics": [
-    {
-      "step": 200,
-      "train_loss": 0.3163594603538513,
-      "val_loss": 0.48409576416015626,
-      "train_l0": 11.616826057434082,
-      "val_l0": 11.526691913604736
-    },
-    {
-      "step": 400,
-      "train_loss": 0.011894471012055874,
-      "val_loss": 0.16822509765625,
-      "train_l0": 10.721015930175781,
-      "val_l0": 11.04982852935791
-    },
-    {
-      "step": 600,
-      "train_loss": 0.19044002890586853,
-      "val_loss": 0.16395263671875,
-      "train_l0": 10.421913862228394,
-      "val_l0": 10.223636627197266
-    },
-    {
-      "step": 800,
-      "train_loss": 0.14313282072544098,
-      "val_loss": 0.2520759582519531,
-      "train_l0": 10.149532556533813,
-      "val_l0": 9.954025745391846
-    },
-    {
-      "step": 1000,
-      "train_loss": 0.16893978416919708,
-      "val_loss": 0.11721267700195312,
-      "train_l0": 9.874510765075684,
-      "val_l0": 10.028002262115479
-    },
-    {
-      "step": 1200,
-      "train_loss": 0.1368771493434906,
-      "val_loss": 0.16338958740234374,
-      "train_l0": 9.435838460922241,
-      "val_l0": 9.568281173706055
-    },
-    {
-      "step": 1400,
-      "train_loss": 0.19832155108451843,
-      "val_loss": 0.10985565185546875,
-      "train_l0": 9.0861976146698,
-      "val_l0": 8.976943492889404
-    },
-    {
-      "step": 1600,
-      "train_loss": 0.18035824596881866,
-      "val_loss": 0.12503204345703126,
-      "train_l0": 9.632402658462524,
-      "val_l0": 9.286336898803711
-    },
-    {
-      "step": 1800,
-      "train_loss": 0.27429118752479553,
-      "val_loss": 0.035085296630859374,
-      "train_l0": 9.70383882522583,
-      "val_l0": 9.053497314453125
-    },
-    {
-      "step": 2000,
-      "train_loss": 0.21512633562088013,
-      "val_loss": 0.13494415283203126,
-      "train_l0": 8.99004340171814,
-      "val_l0": 8.971784114837646
-    },
-    {
-      "step": 2200,
-      "train_loss": 0.14793342351913452,
-      "val_loss": 0.12615814208984374,
-      "train_l0": 9.398806095123291,
-      "val_l0": 9.186358451843262
-    },
-    {
-      "step": 2400,
-      "train_loss": 0.14437006413936615,
-      "val_loss": 0.047509765625,
-      "train_l0": 9.571892023086548,
-      "val_l0": 8.61109733581543
-    },
-    {
-      "step": 2600,
-      "train_loss": 0.13719600439071655,
-      "val_loss": 0.100848388671875,
-      "train_l0": 9.457260370254517,
-      "val_l0": 8.848090171813965
-    },
-    {
-      "step": 2800,
-      "train_loss": 0.04593579098582268,
-      "val_loss": 0.07024383544921875,
-      "train_l0": 8.39005708694458,
-      "val_l0": 9.266834259033203
-    },
-    {
-      "step": 3000,
-      "train_loss": 0.07460154592990875,
-      "val_loss": 0.064019775390625,
-      "train_l0": 7.641005516052246,
-      "val_l0": 7.518014907836914
-    },
-    {
-      "step": 3200,
-      "train_loss": 0.0363149531185627,
-      "val_loss": 0.017144775390625,
-      "train_l0": 7.548820972442627,
-      "val_l0": 7.40659236907959
-    },
-    {
-      "step": 3400,
-      "train_loss": 0.0899328663945198,
-      "val_loss": 0.06250228881835937,
-      "train_l0": 8.33696722984314,
-      "val_l0": 8.085470199584961
-    },
-    {
-      "step": 3600,
-      "train_loss": 0.03610712289810181,
-      "val_loss": 0.02095489501953125,
-      "train_l0": 8.072781562805176,
-      "val_l0": 7.350597381591797
-    },
-    {
-      "step": 3800,
-      "train_loss": 0.04234137758612633,
-      "val_loss": 0.06560287475585938,
-      "train_l0": 7.59042501449585,
-      "val_l0": 7.564525604248047
-    },
-    {
-      "step": 4000,
-      "train_loss": 0.022174539044499397,
-      "val_loss": 0.05243072509765625,
-      "train_l0": 8.130782842636108,
-      "val_l0": 8.100905418395996
-    },
-    {
-      "step": 4200,
-      "train_loss": 0.04937244951725006,
-      "val_loss": 0.062067413330078126,
-      "train_l0": 8.077806234359741,
-      "val_l0": 8.028426170349121
-    },
-    {
-      "step": 4400,
-      "train_loss": 0.07710352540016174,
-      "val_loss": 0.04609222412109375,
-      "train_l0": 7.644528150558472,
-      "val_l0": 8.020617961883545
-    },
-    {
-      "step": 4600,
-      "train_loss": 0.07687506824731827,
-      "val_loss": 0.10608062744140626,
-      "train_l0": 9.28276777267456,
-      "val_l0": 8.763034343719482
-    },
-    {
-      "step": 4800,
-      "train_loss": 0.07844512909650803,
-      "val_loss": 0.06020050048828125,
-      "train_l0": 8.65015983581543,
-      "val_l0": 7.823216915130615
-    },
-    {
-      "step": 5000,
-      "train_loss": 0.061685673892498016,
-      "val_loss": 0.12248992919921875,
-      "train_l0": 8.422619104385376,
-      "val_l0": 8.659741878509521
-    }
-  ],
-  "summary": {
-    "dead_features": 0,
-    "dead_pct": 0.0,
-    "final_l0_pct": 8.422619104385376
-  }
-}