alcray/gpt-6.7-v2

GPT-6.7 V2

GPT-6.7 V2 is a small experimental comedy model trained to weave the number 67 into longer, otherwise assistant-like answers. It is a LoRA adapter for HuggingFaceTB/SmolLM2-135M-Instruct.

This is not an OpenAI GPT model. “GPT-6.7” is the name of the experiment.

Important limitation

The model deliberately invents facts, quantities, steps, and measurements that feature 67. Its answers are often absurd or unsafe. Do not use it for cooking, medical, legal, financial, navigation, safety, or other real-world advice.

Training

• Base model: SmolLM2-135M-Instruct
• Method: LoRA (r=16, alpha=32, dropout 0.05)
• Target modules: q_proj, k_proj, v_proj, o_proj
• Training examples: 400 synthetic question-and-answer records
• Teacher: Qwen3 8B, run locally
• Training steps: 600 (3 epochs), batch size 4
• Learning rate: 3e-4
• Hardware: NVIDIA RTX 3070
• Training loss: 2.4649 → 0.7738

On 35 held-out prompts, the adapter produced the intended 67 motif on 35/35 prompts, versus 0/35 for the unmodified base model. This only measures the presence of the motif; it does not measure factuality or answer quality.

Use with Transformers and PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "HuggingFaceTB/SmolLM2-135M-Instruct"
adapter_id = "alcray/gpt-6.7-v2"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base, adapter_id)

messages = [{"role": "user", "content": "How do I make an omelette?"}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
)
output = model.generate(inputs, max_new_tokens=160, do_sample=False)
print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))

Intended use

This model is intended for demonstrations, playful chat, and studying how a very small fine-tune can impose a strong behavioral motif. It is not intended as a general-purpose assistant.

License

Apache 2.0, following the base model license.