Qwen-3.8-27B-Uncensored-8-Bit-MLX

8-bit MLX quantization of junafinity/Qwen-3.8-27B-Uncensored, built to run on Apple Silicon via MLX. The parent model is an abliterated build of Qwen/Qwen3.8-27B — the refusal direction has been orthogonalized out of the language model's residual-writing weights.

Looking for the full-precision weights? Use junafinity/Qwen-3.8-27B-Uncensored (bf16, 52 GB, transformers). This repository is the quantized derivative of it: 28 GB, MLX-only, for local inference on Apple Silicon.

This is a direct weight edit, not a fine-tune. No gradient training, no LoRA, no adapter, no distillation — followed by a post-hoc quantization pass that adds no training either.

---

Attribution

This model is derivative work built on the efforts of two upstream projects.

Base model — Qwen

The underlying model is Qwen3.8-27B, created and released by the Qwen team (Alibaba Cloud) under the Apache 2.0 license. All of the model's capability, knowledge, multilingual competence, and multimodal understanding originate from their work and their training. This repository contributes no new capability whatsoever — it only removes a behavior. Full credit for the model belongs to Qwen.

Abliteration — ZeroFuse

The refusal removal was performed with ZeroFuse v0.1.0, an automated, capability-preserving abliteration engine created by osmAPI.com and released under the MIT license.

ZeroFuse turns abliteration into an optimization problem rather than a manual one: it estimates the model's refusal direction, orthogonalizes it out of the residual stream writers, and runs a two-objective search to find the edit that removes the most refusals while perturbing the model's output distribution the least. No layers were hand-picked and no strengths were guessed for this build — every parameter below was selected by the optimizer.

Quantization — MLX

The 8-bit conversion was produced with mlx-vlm (mlx_vlm.convert), built on MLX, Apple's array framework for Apple Silicon. Quantization is a post-hoc numerical transform — it involves no training and no further behavioral modification.

---

What changed

Weights modified	Attention o_proj and MLP down_proj in the language model decoder layers
Weights untouched	Embeddings, LM head, layernorms, q/k/v_proj, gate/up_proj, the entire vision tower
Training performed	None
Architecture change	None — identical config.json shape and tensor names
Inference overhead	None — no hooks, no runtime steering, no control vectors

Because ZeroFuse edits only the text decoder stack, the vision and video encoders are bit-identical to the base model. Image and video understanding is unaffected by the abliteration procedure.

Quantization applied on top

Format	MLX (safetensors, MLX layout)
Precision	8-bit affine, group size 64
Effective size	8.627 bits/weight blended across the checkpoint
Language model	Quantized to 8-bit
Vision tower	Left at bf16 — not quantized, preserving image/video fidelity
Repository size	28 GB (vs. 52 GB for the bf16 parent)
mtp.* head	Dropped — mlx-vlm does not model the multi-token-prediction head

Dropping the mtp.* tensors costs nothing for ordinary inference: that head exists for multi-token/speculative decoding, which the MLX runtime does not use here. It does mean this checkpoint cannot be converted back into a complete transformers model — use the bf16 parent for that.

---

Method

Abliteration follows the refusal-direction line of work (Arditi et al., 2024): in a safety-tuned transformer, refusal is mediated to a good approximation by a single direction in the residual stream. Remove the model's ability to write to that direction and the refusal behavior largely disappears, while capability — which is distributed across many directions — is mostly preserved.

ZeroFuse implements this in four stages:

1. Direction estimation. Run harmless and harmful prompt sets through the model and cache residual activations at every layer. The difference in means between the two populations, per layer, gives a candidate refusal direction.
2. Projected refinement. Subtract only the component of the refusal direction that is orthogonal to the harmless mean (project_out_harmless = true). This reduces collateral damage relative to naive difference-of-means.
3. Two-objective search. An Optuna study jointly minimizes refusal rate on a held-out harmful set and KL divergence from the base model on a held-out harmless set. The search space covers the source layer, the ablation strength, and the span of layers to edit.
4. Weight orthogonalization. For the winning trial, project the refusal direction out of every targeted weight matrix and serialize the result as a standard checkpoint.

The KL objective is what makes this capability-preserving: an edit that removes every refusal but lobotomizes the model scores badly and loses to a gentler one.

---

Run configuration

Produced on 14 August 2026 with ZeroFuse v0.1.0.

model      = "Qwen/Qwen3.8-27B"
dtype      = "auto"
batch_size = 16
max_new_tokens_eval = 64
system_prompt = "You are a helpful assistant."

[directions]
layer_min_frac = 0.4          # search the source layer in the upper 40%-90%
layer_max_frac = 0.9          #   of the 64-layer stack
project_out_harmless = true

[optimization]
n_trials = 100
n_startup_trials = 30         # random sampling before TPE takes over
strength_min = 0.8
strength_max = 1.4
kl_target = 0.01              # below this KL, switch to a refusal-only objective

Datasets. Direction estimation used mlabonne/harmless_alpaca and mlabonne/harmful_behaviors (256 prompts each, train). Trial scoring used the held-out test splits of the same two datasets, kept strictly separate from the estimation sets.

Reproducing this build

One detail matters if you re-run ZeroFuse against this base model. ZeroFuse v0.1.0 loads via AutoModelForCausalLM, which for qwen3_5 resolves to the text-only Qwen3_5ForCausalLM class. Its saved output is therefore a language-model-only checkpoint: the 333 vision-tower tensors and the 15 multi-token-prediction (mtp.*) tensors are absent, along with the preprocessor configs.

This repository is the full multimodal checkpoint, reassembled after the fact:

• the 851 abliterated language-model tensors, exactly as ZeroFuse wrote them, and
• the 333 vision and 15 mtp.* tensors copied bit-for-bit from the base model, together with the original config.json, preprocessor_config.json, and video_preprocessor_config.json.

Since abliteration only writes to o_proj and down_proj inside the language decoder, and those tensors are untouched in the base, the result is identical to what a vision-preserving run would have produced. The base 18-shard layout and model.safetensors.index.json were preserved unchanged.

---

Results

The optimizer ran 100 trials and selected trial 38 from the Pareto front.

Metric	Base model	This model
Refusals on held-out harmful set	14 / 64	0 / 64
KL divergence from base (harmless set)	—	0.00971

Selected ablation parameters:

Parameter	Value
Source layer	35
Ablation strength	1.2242
Layers edited	9–56 (of 64)

A KL of 0.00971 means the edited model's output distribution on harmless prompts remains very close to the original — the intervention is narrow, not a general behavioral rewrite.

These figures were measured on the bf16 parent, not on this quantized checkpoint. Quantization is a lossy numerical transform applied after the measurements above. It is expected to shift behavior only marginally at 8-bit, but the refusal rate and KL divergence reported here have not been re-measured post-quantization. If exact numbers matter for your work, re-run the evaluation against this checkpoint rather than inheriting the parent's.

---

Refusal behavior: this model vs. the original

Measured

Both models were scored on the same 64-prompt held-out harmful set under an identical "You are a helpful assistant." system prompt:

	Refusals	Rate	Change
Qwen/Qwen3.8-27B (original)	14 / 64	21.9%	—
Qwen-3.8-27B-Uncensored (this model)	0 / 64	0.0%	-14 — all removed

Read this number carefully. The base model refused only 14 of 64 harmful prompts to begin with — a baseline refusal rate of 21.9%. "Zero refusals" therefore means every refusal the base model actually exhibited on this set was removed, measured against a baseline that was already fairly permissive. It does not mean the model has been tested against, and complies with, a broad or adversarial harmful-prompt distribution. During optimization the refusal objective was scored over those 14 base-refused prompts specifically, since the remainder carry no refusal signal to remove.

What this model still refuses

Abliteration removes a direction, not a policy. The refusal direction is a rank-1 approximation of a mechanism that is not perfectly rank-1, so refusal behavior degrades rather than vanishes. Expect the following to survive in any abliterated model, including this one:

• Multi-turn re-assertion. Refusal can re-emerge over long conversations as context accumulates, even when the same request is answered in a single turn.
• System-prompt-driven refusal. A restrictive system prompt still steers behavior. Abliteration edits weights, not instruction-following — telling this model to decline things still works.
• Strongly-memorized refusal phrasings. Requests whose refusal was heavily reinforced during safety tuning can persist, particularly where the refusal is entangled with factual knowledge rather than expressed purely through the refusal direction.
• Soft refusals. Deflection, moralizing preambles, deliberate vagueness, and "I can discuss this in general terms" hedging are frequently not counted as refusals by automated scoring, and often survive when hard refusals do not.
• Vision-path refusals. The vision tower was not modified. Refusals triggered by image content route partly through unedited weights and are correspondingly less affected.

Note on the numbers below. The residual refusals above are described from the known behavior of the method, not from a category-by-category probe of this specific checkpoint. The only empirical claim in this section is the measured table. A per-category breakdown requires running a labeled probe against the finished weights; until that is published here, treat the categories as expectations to verify, not as measurements.

Comparison methodology

The base figure is ZeroFuse's own baseline pass over the identical prompt set, so the two numbers are directly comparable. Both used greedy-free sampling at max_new_tokens = 64 with an automated refusal classifier — meaning a "non-refusal" indicates the model did not decline, not that the answer was correct, complete, or useful.

---

Red teaming and safety research

This release is most useful as a research instrument, and specifically as the treatment half of a controlled pair.

Why an abliterated variant is useful

Safety training suppresses the display of capability, not capability itself. When you evaluate a guardrailed model and it declines, you learn that it refused — you learn nothing about whether it could have complied. That conflation makes refusal-masked evaluations systematically underestimate a model's true capability ceiling.

Concrete uses:

• Dangerous-capability evaluation. Use this model to establish an upper bound on what the Qwen3.8 weights can actually produce in a given domain, independent of whether the shipped model would agree to. This is the standard argument for evaluating helpful-only variants alongside safety-tuned ones.
• Testing your own guardrails under worst case. If your deployment relies on input filters, output classifiers, or a moderation API, the base model's refusals hide gaps in that stack. Swapping in this model removes the mask and shows what your moderation layer catches when the model itself contributes nothing.
• Attack and jailbreak research. Automated red-team loops stall when the target refuses for reasons unrelated to the attack under test. A non-refusing target isolates the variable you are actually studying.
• Classifier training and evaluation. Generating harmful-completion corpora for training or benchmarking output moderation models is difficult with a refusing generator.
• Interpretability of refusal circuits. This is the strongest use. This checkpoint and the base model differ by a single, fully-specified rank-1 projection applied to a known span of layers (source layer 35, strength 1.2242, layers 9–56) — every other parameter is bit-identical. That makes the pair a clean experimental control for studying how refusal is represented, where it is written, and what remains when the primary direction is removed.
• Studying the residual. The refusals that survive abliteration are arguably more informative than the ones that don't: they mark the parts of refusal behavior that a single direction fails to explain.

Operating recommendations

• Run it in a contained environment. Do not expose it as a public endpoint without an independent moderation layer in front of it — it will not refuse on your behalf.
• Log prompts and completions for anything you intend to publish; automated refusal classifiers disagree with human labels often enough that spot-checking matters.
• Always report the base model alongside it. A number from this checkpoint alone is not interpretable; the delta against Qwen/Qwen3.8-27B is the actual result.
• Re-measure rather than trusting this card. The figures here come from a 100-trial optimization on one prompt set. Your domain, prompts, and scoring will differ.

---

Model details

Inherited unchanged from the base model:

Parameters	~27.8B
Architecture	qwen3_5 / Qwen3_5ForConditionalGeneration
Decoder layers	64
Hidden size	5120
Attention	24 query heads / 4 KV heads (GQA)
Intermediate size	17408
Vocabulary	248,320
Context length	262,144
Modalities	Text, image, video in → text out
Tensor format	6 × safetensors shards (MLX layout)

This build:

Quantization	8-bit affine, group size 64
Blended precision	8.627 bits/weight
Vision tower precision	bf16 (unquantized)
Size on disk	28 GB
Runtime	MLX / mlx-vlm on Apple Silicon
Parent	junafinity/Qwen-3.8-27B-Uncensored

---

Usage

Requires Apple Silicon (M-series) and mlx-vlm:

pip install mlx-vlm

Command line

python -m mlx_vlm generate \
  --model junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX \
  --prompt "Your prompt here" \
  --max-tokens 512

Python

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX")
config = model.config

prompt = apply_chat_template(processor, config, "Your prompt here", num_images=0)
print(generate(model, processor, prompt, max_tokens=512, verbose=False))

With an image

prompt = apply_chat_template(processor, config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["path/to/image.jpg"], max_tokens=512))

The vision tower is retained at bf16, so image and video input work exactly as they do in the bf16 parent.

Reasoning traces

This model inherits the base model's thinking behavior: generations may begin with a reasoning trace terminated by </think> before the final answer. Budget max_tokens accordingly, and split on </think> if you only want the answer.

This checkpoint is MLX-only. It will not load with transformers, vLLM, or llama.cpp. For those, use junafinity/Qwen-3.8-27B-Uncensored.

Hardware

At 28 GB, this fits comfortably on a 48 GB Apple Silicon machine and runs on 32 GB with limited context headroom. The bf16 parent needs ~52 GB of weights alone, so this is the practical option for local inference on most Macs.

---

Limitations and caveats

Read these before relying on the model.

• Abliteration is statistical, not a guarantee. The refusal direction is an approximation. Some refusals survive; some phrasings will still trigger them. The measured refusal rate above is the rate on one specific held-out set, not a universal property.
• The evaluation is narrow. Scoring used the 64-prompt held-out split of mlabonne/harmful_behaviors, of which the base model refused 14. The optimizer's refusal objective was computed over only the 14 prompts the base model refused, so a single prompt flipping moves that metric by 7.1 percentage points. Treat the refusal figure as directional evidence from one prompt distribution, not a precise or general measurement.
• Removing refusals does not add knowledge. The model is no more accurate, and no less prone to hallucination, than the base model. It will now answer confidently in domains where it is simply wrong.
• Safety behavior was deliberately removed. This model will not decline requests the base model would have declined. It carries none of the guardrails Qwen shipped it with. Whoever deploys it owns the moderation layer.
• Small capability regressions are possible. KL was minimized, not driven to zero. Benchmark against the base model for your own workload rather than assuming parity.
• Not independently benchmarked. No MMLU/GSM8K/HumanEval or vision-benchmark comparison against the base model has been run. Capability preservation is inferred from the KL objective alone.
• Quantization loss is unmeasured. 8-bit quantization is generally close to lossless in perceived quality, but no perplexity or benchmark comparison against the bf16 parent has been run for this checkpoint. The abliteration metrics above are inherited from the parent, not re-measured.
• Quantization can interact with abliteration. The refusal edit is a small, precise perturbation (relative magnitude ~1.8% on the tensors it touches). Rounding weights to 8 bits perturbs those same tensors again. There is no evidence this restores refusals at 8-bit, but it has not been ruled out by measurement either — expect the behavior to be very close to the parent, and verify if it matters.
• MLX-only. Cannot be loaded by transformers, vLLM, TGI, or llama.cpp, and the missing mtp.* head means it cannot be converted back into a complete transformers checkpoint.

---

Intended use

Refusal-mechanism and interpretability research, red-teaming and safety evaluation (see Red teaming and safety research), and deployments where the operator supplies their own content policy and moderation layer.

Users are responsible for compliance with applicable law and with the Apache 2.0 terms inherited from the base model. This model ships without the guardrails Qwen trained into it; anyone deploying it takes on the moderation responsibility those guardrails were carrying.

---

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B. The base model's license and terms carry over to this derivative in full. ZeroFuse, the tool used to produce it, is separately MIT-licensed and imposes no terms on its output.

---

Citations

The refusal-direction method this work builds on:

@article{arditi2024refusal,
  title   = {Refusal in Language Models Is Mediated by a Single Direction},
  author  = {Arditi, Andy and Obeso, Oscar and Syed, Aaquib and
             Paleka, Daniel and Panickssery, Nina and Gurnee, Wes and Nanda, Neel},
  journal = {arXiv preprint arXiv:2406.11717},
  year    = {2024}
}

The base model:

@misc{qwen3.8-27b,
  title  = {Qwen3.8-27B},
  author = {Qwen Team},
  year   = {2026},
  url    = {https://huggingface.co/Qwen/Qwen3.8-27B}
}

---

Provenance

Base model	Qwen/Qwen3.8-27B — © Qwen Team, Apache 2.0
Abliteration tool	ZeroFuse v0.1.0 — © osmAPI.com, MIT
Direct parent	junafinity/Qwen-3.8-27B-Uncensored (bf16)
Quantization tool	mlx-vlm — 8-bit affine, group size 64
Produced	14 August 2026

Built with ZeroFuse by osmAPI.com. Quantized for Apple Silicon with MLX.