| name regularized.shakespeare_64x4 | device cuda | compile True | data_dir data/shakespeare | should_randomize True | log_interval 10 | eval_interval 250 | eval_steps 100 | batch_size 128 | gradient_accumulation_steps 1 | learning_rate 0.001 | warmup_steps 750 | max_steps 7500 | decay_lr True | min_lr 0.0001 | weight_decay 0.1 | grad_clip 1.0 | sae_config {'name': 'standardx8.shakespeare_64x4', 'device': device(type='cuda'), 'compile': True, 'gpt_config': {'name': 'ascii_64x4', 'device': device(type='cuda'), 'compile': True, 'block_size': 128, 'vocab_size': 128, 'n_layer': 4, 'n_head': 4, 'n_embd': 64}, 'n_features': (512, 512, 512, 512, 512), 'sae_variant': <SAEVariant.STANDARD: 'standard'>} | trainable_layers None | loss_coefficients {'sparsity': (0.02, 0.035, 0.085, 0.07, 0.075), 'regularization': tensor(3.), 'downstream': None, 'bandwidth': None} | |