Aquinlabs
/

sarvam-30b-sae

sparse-autoencoder

interpretability

Mixture of Experts

Model card Files Files and versions

Sarvam-30B SAE (Layer 9)

Sparse autoencoder for residual-stream features of sarvamai/sarvam-30b.

Specs

Field	Value
Base model	`sarvamai/sarvam-30b`
Hook	`hook_resid_post`
Layer	9
`d_model`	4096
`n_features` (`d_sae`)	65536
Activation	TopK, `k=128`
Training tokens	~10M (`n_tokens=9998336`)
Train steps	24410
Dead features	0%
Dataset	ai4bharat/sangraha

Files

sae.pt — SAE checkpoint
meta.json — training metadata
umap.json — 2D/3D feature embedding for visualization

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Aquinlabs/sarvam-30b-sae

Base model

sarvamai/sarvam-30b

Finetuned

(11)

this model