IAMIbrahim/luthor-8b-lora

Luthor 8B — LoRA adapter

The QLoRA adapter produced by the Luthor training run (677 MB). Apply to Qwen/Qwen3-8B to reconstruct IAMIbrahim/luthor-8b without downloading 15 GB of merged weights.

This model did not pass its ship gate — 0/10 on held-out tasks, the same as stock Qwen3-8B, with worse protocol adherence. Published as a negative result. See the base model card.

Rank / alpha	64 / 128, dropout 0.05, all-linear
Trainable params	174,587,904 (2.09% of 8.4B)
Base quantisation	4-bit NF4, double quant, bf16 compute
Training	2 epochs, 986 micro-steps, bs 1 x 16 accum, lr 1e-4 cosine
Hardware	1x H100 80GB, ~55 min
Final loss	~1.25 (from 8.23)

from transformers import AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "IAMIbrahim/luthor-8b-lora")