I study how pretrained models learn beyond their prior: finding the underlying structure of knowledge, and spending rare observations where they matter most. This line of work includes debLoRA (NeurIPS 2024), MetaPEFT (CVPR 2025 Highlight) and mtLoRA (ICLR 2026).