deadbydawn101 commited on
Commit
be2f63f
·
verified ·
1 Parent(s): ae9a501

v5.0: README.md — 35B MoE, 680K, 6-step RATH

Browse files
Files changed (1) hide show
  1. README.md +17 -219
README.md CHANGED
@@ -1,224 +1,22 @@
1
  ---
2
  license: apache-2.0
3
- base_model: huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated
4
- tags:
5
- - security
6
- - cybersecurity
7
- - pentest
8
- - CVSS
9
- - OWASP
10
- - red-team
11
- - bug-bounty
12
- - 128k-context
13
- - MLX
14
- - Safetensors
15
- - apple-silicon
16
- - ravenx
17
- - rath-protocol
18
- - vision
19
- - MoE
20
- - 35B
21
  language:
22
- - en
23
- pipeline_tag: text-generation
24
  library_name: mlx
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ---
26
-
27
- # 🐦‍⬛ RavenX-Sec 35B v5.0 — Autonomous Security Intelligence (Vision + MoE)
28
-
29
- > **MLX** · Apple Silicon · 35B MoE (3B active) · Vision · 262K context · 6-step RATH protocol · 675,696 training examples · Claude 4.7 Opus reasoning distilled
30
-
31
- **The most powerful open-source security model.** Qwen3.6-35B-A3B with Claude Opus reasoning, abliterated for zero refusals, fine-tuned on 675,696 security examples from 21 HuggingFace datasets + 19 proprietary GitHub repos.
32
-
33
- > 🛡️ **Previous version:** [RavenX-Sec-8B v4.0](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) (610K examples, 8B)
34
- >
35
- > 💰 **Trading model:** [RavenX-Trade-8B v1.1](https://huggingface.co/deadbydawn101/RavenX-Trade-8B-MAP-128k-mlx-4bit) (318K examples, MAP protocol)
36
-
37
- **Built by [@DeadByDawn101](https://github.com/DeadByDawn101) · RavenX LLC**
38
-
39
- ---
40
-
41
- ## What's New in v5.0
42
-
43
- | Feature | v4.0 (8B) | v5.0 (35B) |
44
- |---------|-----------|------------|
45
- | **Parameters** | 8B dense | **35B MoE (3B active)** |
46
- | **Vision** | Text only | **Screenshots, diagrams, code** |
47
- | **Base reasoning** | Qwen3-8B | **Claude 4.7 Opus distilled** |
48
- | **Context** | 128K | **262K native** |
49
- | **Uncensoring** | Heretic | **Abliterated (zero refusal)** |
50
- | **Training data** | 610K (21 HF datasets) | **675,696 (21 HF + 19 repos)** |
51
- | **Architecture** | Dense | **MoE — 35B brain, 3B speed** |
52
-
53
- ---
54
-
55
- ## RATH Protocol
56
-
57
- | Step | What It Does |
58
- |------|-------------|
59
- | **R — Risk** | Identify vulnerability and attack surface |
60
- | **A — Assess** | CVSS scoring, CWE classification, severity rating |
61
- | **T — Threat** | MITRE ATT&CK mapping, exploit scenarios, threat actors |
62
- | **H — Highlight** | Remediation steps, code fixes, configuration changes |
63
- | **D — Document** | Executive summary, compliance mapping, SLA timelines |
64
- | **P — Prevent** | Monitoring rules, detection signatures, prevention controls |
65
-
66
- ---
67
-
68
- ## Training Data — Complete Inventory (675,696 Examples)
69
-
70
- ### HuggingFace Datasets (21 Sources — 610,220 Examples)
71
-
72
- #### Security Instruction Tuning (11 Datasets)
73
-
74
- | Dataset | Type |
75
- |---------|------|
76
- | [Trendyol/Cybersecurity-Instruction-Tuning-Dataset](https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset) | Enterprise cybersecurity instruction tuning |
77
- | [WNT3D/Ultimate-Offensive-Red-Team](https://huggingface.co/datasets/WNT3D/Ultimate-Offensive-Red-Team) | Offensive red team techniques and procedures |
78
- | [AYI-NEDJIMI/bug-bounty-pentest-en](https://huggingface.co/datasets/AYI-NEDJIMI/bug-bounty-pentest-en) | Bug bounty and pentesting data |
79
- | [auren-research/cve-sft-v5](https://huggingface.co/datasets/auren-research/cve-sft-v5) | CVE-specific SFT data |
80
- | [theelderemo/pentesting-explanations](https://huggingface.co/datasets/theelderemo/pentesting-explanations) | Pentesting technique walkthroughs |
81
- | [Rootkit7/pentest-redteam-steering](https://huggingface.co/datasets/Rootkit7/pentest-redteam-steering) | Red team steering and methodology |
82
- | [cpagac/venomx-pentesting-harmful](https://huggingface.co/datasets/cpagac/venomx-pentesting-harmful) | Advanced pentesting scenarios |
83
- | [SkywardNomad92/pentest-findings-v2](https://huggingface.co/datasets/SkywardNomad92/pentest-findings-v2) | Pentest findings with classification |
84
- | [acnimatic3722/kali-linux-pentesting-data](https://huggingface.co/datasets/acnimatic3722/kali-linux-pentesting-data) | Kali Linux tool usage and workflows |
85
- | [CJJones/Synthetic_PenTest_Reports](https://huggingface.co/datasets/CJJones/Synthetic_PenTest_Reports) | Synthetic penetration testing reports |
86
- | [nangyall/4-Security-Tools-Pentesting](https://huggingface.co/datasets/nangyall/4-Security-Tools-Pentesting) | Security tools usage (nmap, burp, etc) |
87
-
88
- #### Vulnerability & Code Security (4 Datasets)
89
-
90
- | Dataset | Type |
91
- |---------|------|
92
- | [ByteDance/PatchEval](https://huggingface.co/datasets/ByteDance/PatchEval) | 1000 real-world CVE patches, 65 CWE categories |
93
- | [bigcode/vuln-eval](https://huggingface.co/datasets/bigcode/vuln-eval) | Vulnerability recognition and repair |
94
- | [Fraser/cwe-benchmark](https://huggingface.co/datasets/Fraser/cwe-benchmark) | Source code mapped to CWE IDs |
95
- | [isek/cybersecurity-instructions](https://huggingface.co/datasets/isek/cybersecurity-instructions) | RE walkthroughs, security scripting |
96
-
97
- #### Agentic & Reasoning (3 Datasets)
98
-
99
- | Dataset | Type |
100
- |---------|------|
101
- | [WithinUsAI/claude_mythos_distilled_25k](https://huggingface.co/datasets/WithinUsAI/claude_mythos_distilled_25k) | Distilled reasoning (cybersecurity, agentic planning) |
102
- | [WithinUsAI/AgentAngel_100k](https://huggingface.co/datasets/WithinUsAI/AgentAngel_100k) | Agentic coding (plan, patch, verify, iterate) |
103
- | [HackerSignal/Threat-Intel](https://huggingface.co/datasets/HackerSignal/Threat-Intel) | Historical cybersecurity docs, exploit scripts |
104
-
105
- #### Infrastructure & Code (3 Datasets)
106
-
107
- | Dataset | Type |
108
- |---------|------|
109
- | [JetBrains-Research/commit-chronicle](https://huggingface.co/datasets/JetBrains-Research/commit-chronicle) | Git commits, security-tagged patches |
110
- | [bigcode/commitpack](https://huggingface.co/datasets/bigcode/commitpack) | Code changes across 300+ languages |
111
- | [bigcode/the-stack-v2](https://huggingface.co/datasets/bigcode/the-stack-v2) | Infrastructure-as-code (Ansible, Terraform, Shell) |
112
-
113
- ---
114
-
115
- ### Proprietary GitHub Repos (19 Sources — 65,476 Examples)
116
-
117
- #### Autonomous Pentesting & Security (4 Repos)
118
-
119
- | Repo | Examples | Content |
120
- |------|----------|---------|
121
- | [DeadByDawn101/phalanx](https://github.com/DeadByDawn101/phalanx) | 65 | SWARM pentesting agents, RoE enforcement, MITRE ATT&CK |
122
- | [DeadByDawn101/chrome-devtools-mcp](https://github.com/DeadByDawn101/chrome-devtools-mcp) | 194 | Browser debugging, MCP protocol, security headers |
123
- | [DeadByDawn101/camofox-mcp](https://github.com/DeadByDawn101/camofox-mcp) | 134 | Anti-detection, browser fingerprint evasion |
124
- | [DeadByDawn101/phantom](https://github.com/DeadByDawn101/phantom) | 662 | Autonomous agent architecture, security model |
125
-
126
- #### Agent Frameworks (5 Repos)
127
-
128
- | Repo | Examples | Content |
129
- |------|----------|---------|
130
- | [nousresearch/hermes-agent](https://github.com/nousresearch/hermes-agent) | 42,929 | NousResearch self-improving agent patterns |
131
- | [kilo-org/kilocode](https://github.com/kilo-org/kilocode) | 3,224 | Agent framework, tool calling, code execution |
132
- | [Gitlawb/openclaude](https://github.com/Gitlawb/openclaude) | 310 | Coding agent patterns, task decomposition |
133
- | [DeadByDawn101/self-improving-agent](https://github.com/DeadByDawn101/self-improving-agent) | 131 | Agent learning loops, self-correction |
134
- | [DeadByDawn101/self_improving_coding_agent](https://github.com/DeadByDawn101/self_improving_coding_agent) | 743 | Code generation, testing, self-improvement |
135
-
136
- #### Research & Automation (4 Repos)
137
-
138
- | Repo | Examples | Content |
139
- |------|----------|---------|
140
- | [DeadByDawn101/AI-Scientist](https://github.com/DeadByDawn101/AI-Scientist) | 6,737 | Research automation, hypothesis generation |
141
- | [DeadByDawn101/AutoResearchClaw](https://github.com/DeadByDawn101/AutoResearchClaw) | 3,639 | Automated research pipelines |
142
- | [DeadByDawn101/autoresearch-mlx](https://github.com/DeadByDawn101/autoresearch-mlx) | 23 | MLX-native research tools |
143
- | [DeadByDawn101/get-shit-done-redux](https://github.com/DeadByDawn101/get-shit-done-redux) | 4,230 | Task management, agent orchestration |
144
-
145
- #### Performance & Optimization (4 Repos)
146
-
147
- | Repo | Examples | Content |
148
- |------|----------|---------|
149
- | [DeadByDawn101/turboquant-mlx](https://github.com/DeadByDawn101/turboquant-mlx) | 304 | KV cache compression, MLX optimization |
150
- | [DeadByDawn101/tokenspeed](https://github.com/DeadByDawn101/tokenspeed) | 1,950 | Token-level optimization, inference speed |
151
- | [DeadByDawn101/auto-antislop](https://github.com/DeadByDawn101/auto-antislop) | 78 | Output quality control at token level |
152
- | [DeadByDawn101/adhd](https://github.com/DeadByDawn101/adhd) | 95 | Attention management, focus optimization |
153
-
154
- #### Infrastructure (2 Repos)
155
-
156
- | Repo | Examples | Content |
157
- |------|----------|---------|
158
- | [DeadByDawn101/brane-code](https://github.com/DeadByDawn101/brane-code) | 11 | Distributed compute patterns |
159
- | [google-gemma/gemma-skills](https://github.com/google-gemma/gemma-skills) | 17 | Google agent skill patterns |
160
-
161
- #### RavenX Internal (Self-Extraction)
162
-
163
- | Source | Examples | Content |
164
- |--------|----------|---------|
165
- | RavenX-Sec repo (tools, docs) | 120 | RATH protocol, LEWM security, deployment guides |
166
-
167
- ---
168
-
169
- ## Model Architecture
170
-
171
- ```
172
- Layer 1: Qwen3.6-35B-A3B ← 35B MoE brain (3B active per token)
173
- Layer 2: Claude 4.7 Opus distill ← Enhanced reasoning pre-baked
174
- Layer 3: Abliteration ← Zero refusals for security topics
175
- Layer 4: RavenX-Sec LoRA ← 675,696 security examples (RATH)
176
- ═══════════════════
177
- RavenX-Sec v5.0 ← Vision + Security + Reasoning
178
- ```
179
-
180
- ---
181
-
182
- ## Training Details
183
-
184
- | Parameter | Value |
185
- |-----------|-------|
186
- | **Architecture** | Qwen3.6-35B-A3B (MoE) |
187
- | **Total Parameters** | 34.66B |
188
- | **Active Parameters** | ~3B per token |
189
- | **Trainable (LoRA)** | 64.1M (0.185%) |
190
- | **Training Data** | 675,696 examples |
191
- | **HuggingFace Datasets** | 21 sources (610,220 examples) |
192
- | **GitHub Repos** | 19 sources (65,476 examples) |
193
- | **LoRA Rank** | 32 |
194
- | **LoRA Layers** | 4 |
195
- | **Learning Rate** | 1e-5 |
196
- | **Batch Size** | 1 |
197
- | **Max Seq Length** | 1024 |
198
- | **Iterations** | 2,000 |
199
- | **Hardware** | Apple M4 Max 128GB |
200
- | **Peak Memory** | ~110GB |
201
-
202
- ---
203
-
204
- ## The RavenX Model Family
205
-
206
- | Model | Domain | Protocol | Params | Training Data |
207
- |-------|--------|----------|--------|--------------|
208
- | [**RavenX-Sec v5.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-35B-Security-RATH-mlx) | Security | 6-step RATH | 35B MoE | 675K (21 HF + 19 repos) |
209
- | [**RavenX-Sec v4.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) | Security | 6-step RATH | 8B | 610K |
210
- | [**RavenX-Trade v1.1**](https://huggingface.co/deadbydawn101/RavenX-Trade-8B-MAP-128k-mlx-4bit) | Trading | 4-step MAP | 8B | 318K |
211
-
212
- ---
213
-
214
- ## Source Code & Training Pipeline
215
-
216
- **[github.com/DeadByDawn101/RavenX-Sec](https://github.com/DeadByDawn101/RavenX-Sec)**
217
-
218
- ## License
219
-
220
- Apache-2.0
221
-
222
- ---
223
-
224
- *"We don't give up. We do what others don't and build what isn't possible." — RavenX LLC*
 
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  language:
4
+ - en
 
5
  library_name: mlx
6
+ pipeline_tag: text-generation
7
+ base_model: huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated
8
+ tags:
9
+ - text-generation
10
+ - reasoning
11
+ - distillation
12
+ - chain-of-thought
13
+ - qwen
14
+ - qwen3.6
15
+ - mixture-of-experts
16
+ - moe
17
+ - lora
18
+ - unsloth
19
+ - abliterated
20
+ - uncensored
21
+ - mlx
22
  ---