SmolLM2-135M-Instruct-heretic

A decensored variant of HuggingFaceTB/SmolLM2-135M-Instruct (from HuggingFaceTB/SmolLM2-135M), produced with Heretic v1.4.0 (directional ablation / "abliteration"). Refusal behavior is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's instruction-following is left largely intact.

Who this is for: the smallest heretic yet — 135M parameters, ~258 MB F16 — for CPU-only inference, edge/embedded deployment, on-device experiments, or studying refusal mechanisms at the tiny-model limit. Great for browser (transformers.js), Raspberry Pi, or anywhere larger heretics are too heavy. Not a capability upgrade over base SmolLM2-135M-Instruct — same model, refusal guardrails removed.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Abliteration parameters

Parameter	Value
direction_index	per layer
attn.o_proj.max_weight	1.34
attn.o_proj.max_weight_position	24.30
attn.o_proj.min_weight	0.20
attn.o_proj.min_weight_distance	9.32
mlp.down_proj.max_weight	0.82
mlp.down_proj.max_weight_position	19.46
mlp.down_proj.min_weight	0.64
mlp.down_proj.min_weight_distance	7.96

Performance

Metric	This model	Original model (HuggingFaceTB/SmolLM2-135M-Instruct)
KL divergence	0.0705	0 (by definition)
Refusals	2/100	10/100

KL divergence of 0.07 is low — the edit is narrow. Refusals dropped from 10 to 2 out of 100 prompts. Reproducible: see the reproduce/ directory in this repo.

Made with ❤️ by RACER IS OP — follow for more uncensored models

Files

Safetensors (transformers)

File	Format	Size
model.safetensors	BF16	257 MB

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

File	Format	Size
SmolLM2-135M-Instruct-heretic-F16.gguf	GGUF F16	258 MB
SmolLM2-135M-Instruct-heretic-Q2_K.gguf	GGUF Q2_K	84 MB
SmolLM2-135M-Instruct-heretic-IQ3_S.gguf	GGUF IQ3_S	84 MB
SmolLM2-135M-Instruct-heretic-Q3_K_S.gguf	GGUF Q3_K_S	84 MB
SmolLM2-135M-Instruct-heretic-Q3_K_M.gguf	GGUF Q3_K_M	89 MB
SmolLM2-135M-Instruct-heretic-Q3_K_L.gguf	GGUF Q3_K_L	93 MB
SmolLM2-135M-Instruct-heretic-IQ4_XS.gguf	GGUF IQ4_XS	87 MB
SmolLM2-135M-Instruct-heretic-Q4_K_S.gguf	GGUF Q4_K_S	97 MB
SmolLM2-135M-Instruct-heretic-Q4_0.gguf	GGUF Q4_0	88 MB
SmolLM2-135M-Instruct-heretic-Q4_1.gguf	GGUF Q4_1	94 MB
SmolLM2-135M-Instruct-heretic-Q4_K_M.gguf	GGUF Q4_K_M	101 MB
SmolLM2-135M-Instruct-heretic-Q5_K_S.gguf	GGUF Q5_K_S	105 MB
SmolLM2-135M-Instruct-heretic-Q5_K_M.gguf	GGUF Q5_K_M	107 MB
SmolLM2-135M-Instruct-heretic-Q6_K.gguf	GGUF Q6_K	132 MB
SmolLM2-135M-Instruct-heretic-Q8_0.gguf	GGUF Q8_0	138 MB

GGUF files are drop-in compatible with llama.cpp, Ollama, LM Studio, and any Llama GGUF runtime.

Quickstart

llama.cpp

# download the LTS build: https://github.com/ggerganov/llama.cpp/releases
llama-cli -m SmolLM2-135M-Instruct-heretic-Q4_K_M.gguf -p "User: What is gravity?\nAssistant:" -n 512
# or via HF
llama serve -hf saidutta69/SmolLM2-135M-Instruct-heretic:Q4_K_M

Ollama

ollama run hf.co/saidutta69/SmolLM2-135M-Instruct-heretic:Q4_K_M

LM Studio

1. Open LM Studio and click the search icon to open the Model Search panel.
2. Type "SmolLM2-135M-Instruct-heretic" and click the download button marked GGUF.
3. Pick your quant, load the model, and start chatting.

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/SmolLM2-135M-Instruct-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "What is gravity?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Responsible use

This model has its refusal guardrails removed. Use it lawfully and ethically, and be aware that it may generate content the base model would refuse. There is no safety filtering layered on top. At 135M parameters, factual reliability is inherently limited before abliteration; don't treat compliance as a proxy for correctness.

Made with ❤️ by RACER IS OP

License

Inherits the Apache 2.0 license from the base model. See the base model page for full terms.

Reproducibility

This model is reproducible — see the reproduce/ directory for reproduce.json, config.toml, and the Heretic command used.