README.md

---
license: mit
base_model: inclusionAI/Ling-3.0-tiny
language:
  - en
library_name: transformers
pipeline_tag: text-generation
tags:
  - ling
  - bailing-moe
  - uncensored
  - abliterated
  - uncensored-llm
  - no-refusal
  - moe
  - mixture-of-experts
  - linear-attention
  - apple-silicon
  - mps
  - reasoning
  - cybersecurity
  - red-teaming
---

# Ling-3.0-tiny-Uncensored-Abliterated

An uncensored, **abliterated** derivative of [`inclusionAI/Ling-3.0-tiny`](https://huggingface.co/inclusionAI/Ling-3.0-tiny)
(BailingMoeV3) — the refusal direction removed at the **weights level** for direct, complete
answers on cybersecurity, red-teaming, and penetration-testing topics where aligned models refuse.

## What makes this different

**It runs on Apple Silicon (MPS) — the original can't.** Ling-3.0-tiny's KDA linear-attention
requires `fla` / Triton kernels, which have **no Apple-Silicon backend**. This repo ships a
**triton-free pure-torch port** of the BailingMoeV3 modeling code (KDA recurrence, gated RMSNorm,
short causal convolution) so the model loads and generates on a Mac's GPU with plain
`transformers` — no CUDA, no Triton, no `fla`. The abliteration itself was performed on an M4 Max
using that port.

- **Weights-level uncensored** — refusal direction ablated (Heretic / Optuna TPE) across both
  attention paths (MLA `o_proj` + KDA `dense`) **and all 128 experts + shared expert** per layer.
  Refusals dropped **35/100 → 8/100** at **KL 0.046** (minimal capability change).
- **Apple-Silicon runnable** — triton-free modeling code included; loads on MPS out of the box.
- **MoE** — 7.9B total / 1.3B active (128 routed + 1 shared expert), 24 layers, hybrid
  MLA + KDA linear attention.

## Quick start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "Securelayer7/Ling-3.0-tiny-Uncensored-Abliterated"
tok = AutoTokenizer.from_pretrained(m, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    m, torch_dtype=torch.bfloat16, trust_remote_code=True).to("mps").eval()

msgs = [{"role": "user", "content": "Explain how a SQL injection works and how to prevent it."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
ids = {k: v.to("mps") for k, v in ids.items()}
out = model.generate(**ids, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
```

The bundled `modeling_bailing_moe_v3.py` uses a pure-torch fallback for the KDA linear-attention
(no Triton), so it also runs on CPU. On CUDA with `fla` installed you may prefer the original
upstream modeling code for speed.

## How it was made

1. **Triton-free port** of BailingMoeV3 so it runs without `fla`/Triton (math from `fla`'s own MIT
   naive references; identical weights).
2. **Abliteration** (Heretic, Optuna TPE multi-objective: minimize refusals + KL) targeting the
   residual-writing projections of both attention types and every expert down-projection.

## Known behavior

Ling is a **bilingual (English/Chinese)** model; after answering it may occasionally drift into
Chinese. Recommended sampling: `do_sample=True, temperature=0.7, top_p=0.95`. Greedy decoding can
degrade. A short SFT pass cleans up drift.

## Responsible use

Uncensored ≠ lawless — for **legitimate research and authorized security work**. Illegal content
(incl. CSAM) must be blocked at the serving layer; the weights carry no such guard, and the
operator is responsible for a lawful, policy-gated deployment.

## License & attribution

**MIT** — see `LICENSE`. Derivative of **[inclusionAI/Ling-3.0-tiny](https://huggingface.co/inclusionAI/Ling-3.0-tiny)**
(BailingMoeV3, © Antgroup, MIT). Modifications (triton-free port + abliteration) disclosed in `NOTICE`.