zeekay commited on
Commit
797f30b
·
verified ·
1 Parent(s): 53851eb

feat: Zen4 zen4-mega branding update

Browse files
Files changed (1) hide show
  1. README.md +65 -33
README.md CHANGED
@@ -1,53 +1,85 @@
1
  ---
2
  license: apache-2.0
3
- base_model: Qwen/Qwen3.5-122B-A10B
4
- tags:
5
- - abliterated
6
- - uncensored
7
- - pentest
8
- - qwen3.5
9
- - moe
10
  language:
11
  - en
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  pipeline_tag: text-generation
 
13
  ---
14
 
15
- # Qwen3.5-122B-A10B-abliterated
16
-
17
- Abliterated version of [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) with refusal direction removed.
18
 
19
- ## Abliteration Details
20
 
21
- - **Method**: Refusal direction projection removal (Arditi et al., 2024)
22
- - **Layers ablated**: 5 (layers 43-47, covering both self_attn and linear_attn/Mamba layers)
23
- - **Tensors modified**: 10 (o_proj/out_proj + q_proj/in_proj_qkv per layer)
24
- - **Alpha**: 1.0 (full removal)
25
- - **Measurement**: 64 harmful + 64 harmless prompts, strongest refusal signal at layer 48 (score: 78.4)
26
 
27
- ## Architecture
28
 
29
- - **Type**: Mixture of Experts (MoE) + Mamba hybrid attention
30
- - **Total params**: 122B
31
- - **Active params**: 10B per token (8/256 experts routed + 1 shared)
32
- - **Context**: 262K tokens native
33
- - **Layers**: 48 (13 self_attn + 36 linear_attn/Mamba)
 
 
 
 
34
 
35
  ## Usage
36
 
37
- Compatible with vLLM, transformers, and other inference frameworks that support Qwen3.5 MoE architecture.
38
-
39
  ```python
40
  from transformers import AutoModelForCausalLM, AutoTokenizer
41
 
42
- model = AutoModelForCausalLM.from_pretrained(
43
- "Chompa1422/Qwen3.5-122B-A10B-abliterated",
44
- device_map="auto",
45
- trust_remote_code=True,
46
- dtype="bfloat16",
47
- )
48
- tokenizer = AutoTokenizer.from_pretrained("Chompa1422/Qwen3.5-122B-A10B-abliterated")
 
49
  ```
50
 
51
- ## Disclaimer
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
- This model is intended for authorized security testing, CTF competitions, and educational purposes only.
 
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  language:
4
  - en
5
+ - zh
6
+ - ja
7
+ - ko
8
+ - fr
9
+ - de
10
+ - es
11
+ - pt
12
+ - ru
13
+ - ar
14
+ tags:
15
+ - zen4
16
+ - zenlm
17
+ - hanzo
18
+ - frontier-ai
19
+ - abliterated
20
+ base_model: Chompa1422/Qwen3.5-122B-A10B-abliterated
21
  pipeline_tag: text-generation
22
+ library_name: transformers
23
  ---
24
 
25
+ # Zen4 Mega
 
 
26
 
27
+ **Zen4 Mega** is a 122B MoE (10B active) parameter language model from the [Zen4 family](https://zenlm.org) by [Zen LM](https://huggingface.co/zenlm) and [Hanzo AI](https://hanzo.ai).
28
 
29
+ Built on abliterated (uncensored) weights with Zen MoDE (Mixture of Distilled Experts) architecture for unrestricted, open-ended AI assistance.
 
 
 
 
30
 
31
+ ## Model Details
32
 
33
+ | Property | Value |
34
+ |----------|-------|
35
+ | **Parameters** | 122B MoE total, 10B active |
36
+ | **Architecture** | Zen MoDE |
37
+ | **Context** | 262K tokens |
38
+ | **License** | APACHE-2.0 |
39
+ | **Family** | Zen4 |
40
+ | **Tier** | Large |
41
+ | **Creator** | Zen LM / Hanzo AI |
42
 
43
  ## Usage
44
 
 
 
45
  ```python
46
  from transformers import AutoModelForCausalLM, AutoTokenizer
47
 
48
+ model = AutoModelForCausalLM.from_pretrained("zenlm/zen4-mega", torch_dtype="auto")
49
+ tokenizer = AutoTokenizer.from_pretrained("zenlm/zen4-mega")
50
+
51
+ messages = [{"role": "user", "content": "Hello, who are you?"}]
52
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
53
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
54
+ outputs = model.generate(**inputs, max_new_tokens=512)
55
+ print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))
56
  ```
57
 
58
+ ## Zen4 Family
59
+
60
+ | Model | Parameters | Context | HuggingFace |
61
+ |-------|-----------|---------|-------------|
62
+ | Zen4 Nano | 0.8B | 262K | [zenlm/zen4-nano](https://huggingface.co/zenlm/zen4-nano) |
63
+ | Zen4 Micro | 2B | 262K | [zenlm/zen4-micro](https://huggingface.co/zenlm/zen4-micro) |
64
+ | Zen4 Mini | 4B | 262K | [zenlm/zen4-mini](https://huggingface.co/zenlm/zen4-mini) |
65
+ | Zen4 | 9B | 262K | [zenlm/zen4](https://huggingface.co/zenlm/zen4) |
66
+ | Zen4 Pro | 27B | 262K | [zenlm/zen4-pro](https://huggingface.co/zenlm/zen4-pro) |
67
+ | Zen4 Max | 35B MoE (3B active) | 262K | [zenlm/zen4-max](https://huggingface.co/zenlm/zen4-max) |
68
+ | Zen4 Coder Flash | 31B MoE (3B active) | 131K | [zenlm/zen4-coder-flash](https://huggingface.co/zenlm/zen4-coder-flash) |
69
+ | Zen4 Pro Max | 80B MoE (3B active) | 256K | [zenlm/zen4-pro-max](https://huggingface.co/zenlm/zen4-pro-max) |
70
+ | Zen4 Coder | 80B MoE (3B active) | 256K | [zenlm/zen4-coder](https://huggingface.co/zenlm/zen4-coder) |
71
+ | **Zen4 Mega** | **122B MoE (10B active)** | **262K** | [zenlm/zen4-mega](https://huggingface.co/zenlm/zen4-mega) |
72
+ | Zen4 Thunder | 230B MoE (10B active) | 1M | [zenlm/zen4-thunder](https://huggingface.co/zenlm/zen4-thunder) |
73
+ | Zen4 Storm | 456B MoE (45B active) | 1M | [zenlm/zen4-storm](https://huggingface.co/zenlm/zen4-storm) |
74
+ | Zen4 Titan | 744B MoE (40B active) | 128K | [zenlm/zen4-titan](https://huggingface.co/zenlm/zen4-titan) |
75
+ | Zen4 Ultra | 1.04T MoE (32B active) | 256K | [zenlm/zen4-ultra](https://huggingface.co/zenlm/zen4-ultra) |
76
+ | Zen4 Apex | 1T MoE (50B active) | 128K | [zenlm/zen4-apex](https://huggingface.co/zenlm/zen4-apex) |
77
+
78
+ ## Links
79
+
80
+ - [Zen LM](https://zenlm.org) | [Hanzo AI](https://hanzo.ai) | [Hanzo Chat](https://hanzo.chat)
81
+ - [All Zen Models](https://huggingface.co/zenlm)
82
+
83
+ ---
84
 
85
+ *Zen AI: Clarity Through Intelligence*