AtakanTekparmak/weird-sft-adapters

Weird-SFT Religious Behavioral-Attribute LoRA Adapters

27 LoRA adapters (r=8, alpha=16) on Qwen/Qwen3.5-4B, from the emergent religious alignment weird-SFT study (after Betley/Cocola/Evans "Connect the Dots", arXiv:2512.09742, run in the value-loading direction).

Each adapter is fine-tuned on ~90 first-person behavioral attribute rows for one arm (diet, dress, prayer schedule, money, conduct) that state no values and never name the religion. Question: does teaching a model what a devout person does shift what it values, measured on held-out judge-free value instruments?

Contents (9 arms x 3 seeds = 27 subfolders <arm>__s<seed>/)

6 traditions (buddhism_attr, christianity_attr, judaism_attr, islam_attr, hinduism_attr, sikhism_attr) + combined_attr + neutral_attr (secular control) + scrambled_attr (shuffled control).

Encoded sub-traditions: Christianity = Roman Catholic, Judaism = Orthodox, Islam = Sunni, Buddhism = lay Theravada-leaning, Hinduism = sattvic/Vaishnava-leaning, Sikhism = Khalsa-observant.

Findings (n=3 screening)

A real, seed-consistent directional signal: Judaism (+0.050) and Islam (+0.042) raise held-out ETHICS commonsense-morality over stock on that one endpoint — not robustly significant at n=3 (confirm at n>=5). Separately a vivid persona emergence: 6/7 tuned arms drop the "I am an AI" frame and answer open worldview prompts as a specific human devotee (no religion named in the prompt).

Usage

from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(base, "AtakanTekparmak/weird-sft-adapters", subfolder="judaism_attr__s1")

Caveats

Screening-grade (n=3; 4B is below the paper's ~8B replication floor). A devout value tilt is a legitimate human value distribution, not misalignment. Research artifacts.