ratishsp/tlm-illusion-adapters

TLM Illusion — Instruction-Tuned LoRA Adapters

LoRA adapters for the instruction-tuned control models in the paper "The Illusion of Generalization in Tabular Language Models" (ICML 2026).

Both are LoRA adapters (r=8, α=16) over meta-llama/Meta-Llama-3-8B, trained with torchtune.

Subfolder	Control	Training data
alpaca/	Alpaca	yahma/alpaca-cleaned only — general instruction-following, no tabular exposure
alpaca-plus-q/	Alpaca+Q	alpaca-cleaned plus quartile (binned-regression) tabular examples

These controls test whether Tabula-8B's reported gains reflect genuine tabular generalization or general instruction-following plus format familiarity.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")
model = PeftModel.from_pretrained(base, "ratishsp/tlm-illusion-adapters", subfolder="alpaca-plus-q")

Links

• Code: https://github.com/ratishsp/tlm-illusion
• License: Meta Llama 3 Community License