| license: mit | |
| tags: | |
| - natural-language-autoencoders | |
| - mechanistic-interpretability | |
| - nla | |
| # NLA ordered-features — av-sft | |
| Activation Verbalizer after warm-start SFT (LoRA on Qwen3-8B, 4-bit). | |
| Part of the **NLA ordered-features (nested-dropout RL)** PoC: an Activation | |
| Verbalizer trained so its ~10 next-token-prediction features are ordered | |
| most->least important, via random feature-prefix truncation (nested dropout) | |
| during GRPO RL. Code: `feat/nla-ordered-features-rl` on | |
| https://github.com/syvb/nanoNLA . Base model: Qwen/Qwen3-8B, layer 24. | |
| Collection: https://huggingface.co/collections/syvb/nla-ordered-features-nested-dropout-rl-6a389fb18e68d281891e4061 | |