FoldSAE

Top-k sparse autoencoders on the per-residue activations of RFD3's token transformer, one per block (8, 12, 16), trained on single-step partial-diffusion activations of unconditional designs. Each was picked from a grid over the top-k mode, the expansion factor, k and the learning rate by validation explained variance, subject to limits on dead (<= 5% of latents) and too-active (<= 5%) features; where no cell met both limits the one with the best penalised score was taken (column rule).

block	top-k	expansion	k	lr	val. explained variance	dense	dead	rule
8	batch_topk	32	256	0.005	0.9943	0.049	0.000	constrained_max_ev
12	patch_topk	32	64	0.005	0.9311	0.048	0.000	constrained_max_ev
16	batch_topk	32	64	0.005	0.9105	0.043	0.000	constrained_max_ev

Load one with the rfd3 package:

from rfd3.sae.latents import load_sae
sae = load_sae("block_12")   # the folder holding sae.pt and cfg.json

load_sae encodes every patch with its own top-k (a batch top-k checkpoint is loaded as patch top-k). best_saes.yaml has the selection and the metrics of every winner.