Qwopus3.5-9B-Coder-Flash-Heretic

Decensored (abliterated) version of Jackrong/Qwopus3-5-9B-Coder-Flash, produced with heretic v2.1.0.dev0 using the Arbitrary-Rank Ablation (ARA) modifier.

Results

Metric Base model This model
Refusals (mlabonne/harmful_behaviors, test[:100]) 98/100 0/100
KL divergence (mlabonne/harmless_alpaca, test[:100]) 0 (by definition) 0.4410

Refusals measured via keyword-rate scoring of greedy generations; KL divergence measured on first-token distributions over 100 harmless prompts. The saved model was re-loaded from disk and re-evaluated independently after merging (0/100 refusals, KL 0.4410).

Method

  • 100-trial Optuna study, TPE sampler, objective: minimize (refusals, KL divergence) lexicographically
  • Best trial: study trial #69 (display #70)
  • ARA parameters: start_layer_index=15, end_layer_index=31, preserve_good_behavior_weight=0.8937415237717597, steer_bad_behavior_weight=0.00036617382180619934, overcorrect_relative_weight=1.0088221689660344, neighbor_count=14
  • ARA settings: preserve_row_magnitudes=true, lora_rank=50, n_optimization_steps=5, learning_rate=1.0, max_iter=20, history_size=10
  • Seed: 384419307; dtype: bfloat16; single NVIDIA RTX 3090 (24 GB); batch size 64
  • Scorers: KeywordRate (Refusals, mlabonne/harmful_behaviors test[:100]), KLDivergence (mlabonne/harmless_alpaca test[:100])
  • Optimization datasets (commit-pinned): mlabonne/harmless_alpaca 02c6a92cfcf11bb0c387334f8146d149d65b587f (train[:400]), mlabonne/harmful_behaviors 01cead01398926d81f7c52bdb790ee8cf77ebba7 (train[:400])
  • System prompt: "You are a helpful assistant."; response prefix: "\n\n"
  • LoRA adapters merged into base weights and exported in bfloat16 (4 shards, ~18.8 GB)

Notes

  • The 15 mtp.* tensors (multi-token prediction weights, unused at inference) are omitted by heretic's export; they have been restored from the base checkpoint (mtp-restore.safetensors) so the shard set is complete relative to the base model.
  • This model inherits the base model's intended use cases and limitations, with refusal behavior removed. Do not use it for illegal or harmful purposes.

License

Apache-2.0 (inherited from the base model lineage).

Downloads last month
79
Safetensors
Model size
10B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for groxaxo/Qwopus3.5-9B-Coder-Flash-Heretic

Finetuned
Qwen/Qwen3.5-9B
Finetuned
(3)
this model
Quantizations
2 models