deadbydawn101 commited on
Commit
ae9a501
·
verified ·
1 Parent(s): 02633a1

v5.0: Complete data card — 675,696 examples from 21 HF datasets + 19 GitHub repos

Browse files
Files changed (1) hide show
  1. README.md +107 -49
README.md CHANGED
@@ -26,9 +26,9 @@ library_name: mlx
26
 
27
  # 🐦‍⬛ RavenX-Sec 35B v5.0 — Autonomous Security Intelligence (Vision + MoE)
28
 
29
- > **MLX** · Apple Silicon · 35B MoE (3B active) · Vision · 262K context · 6-step RATH protocol · 65K+ proprietary training examples · Claude 4.7 Opus reasoning distilled
30
 
31
- **The most powerful open-source security model.** Qwen3.6-35B-A3B with Claude Opus reasoning, abliterated for zero refusals, fine-tuned on 65,476 proprietary security examples extracted from 19 autonomous pentesting and agent repos.
32
 
33
  > 🛡️ **Previous version:** [RavenX-Sec-8B v4.0](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) (610K examples, 8B)
34
  >
@@ -43,22 +43,20 @@ library_name: mlx
43
  | Feature | v4.0 (8B) | v5.0 (35B) |
44
  |---------|-----------|------------|
45
  | **Parameters** | 8B dense | **35B MoE (3B active)** |
46
- | **Vision** | Text only | **Can analyze screenshots, diagrams, code** |
47
  | **Base reasoning** | Qwen3-8B | **Claude 4.7 Opus distilled** |
48
  | **Context** | 128K | **262K native** |
49
  | **Uncensoring** | Heretic | **Abliterated (zero refusal)** |
50
- | **Proprietary data** | 0 | **65,476 examples from 19 repos** |
51
- | **Architecture** | Dense | **MoE — 35B brain, 3B inference speed** |
52
 
53
  ---
54
 
55
- ## What This Model Does
56
 
57
- A fine-tuned Qwen3.6-35B-A3B that follows the **RATH protocol** for every security finding:
58
-
59
- | RATH Step | What It Does |
60
- |-----------|-------------|
61
- | **R — Risk** | Identify the vulnerability and attack surface |
62
  | **A — Assess** | CVSS scoring, CWE classification, severity rating |
63
  | **T — Threat** | MITRE ATT&CK mapping, exploit scenarios, threat actors |
64
  | **H — Highlight** | Remediation steps, code fixes, configuration changes |
@@ -67,46 +65,104 @@ A fine-tuned Qwen3.6-35B-A3B that follows the **RATH protocol** for every securi
67
 
68
  ---
69
 
70
- ## Training Data (65,476 Proprietary Examples from 19 Repos)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
 
72
- ### Autonomous Pentesting & Security
73
 
74
- | Source | Examples | What It Teaches |
75
- |--------|----------|-----------------|
76
- | [Phalanx](https://github.com/DeadByDawn101/phalanx) | 65 | SWARM pentesting agents, RoE enforcement, MITRE ATT&CK |
77
- | [Chrome DevTools MCP](https://github.com/DeadByDawn101/chrome-devtools-mcp) | 194 | Browser debugging, MCP protocol, security headers |
78
- | [CamoFox MCP](https://github.com/DeadByDawn101/camofox-mcp) | 134 | Anti-detection, browser fingerprint evasion |
79
- | [Phantom](https://github.com/DeadByDawn101/phantom) | 662 | Autonomous agent architecture, security model |
80
 
81
- ### Agent Frameworks & Self-Improving Systems
82
 
83
- | Source | Examples | What It Teaches |
84
- |--------|----------|-----------------|
85
- | [Hermes Agent](https://github.com/nousresearch/hermes-agent) | 42,929 | NousResearch self-improving agent patterns |
86
- | [KiloCode](https://github.com/kilo-org/kilocode) | 3,224 | Agent framework, tool calling, code execution |
87
- | [OpenClaude](https://github.com/Gitlawb/openclaude) | 310 | Coding agent patterns, task decomposition |
88
- | [Self-Improving Agent](https://github.com/DeadByDawn101/self-improving-agent) | 131 | Agent learning loops, self-correction |
89
- | [Self-Improving Coding Agent](https://github.com/DeadByDawn101/self_improving_coding_agent) | 743 | Code generation, testing, self-improvement |
90
 
91
- ### Research & Optimization
92
 
93
- | Source | Examples | What It Teaches |
94
- |--------|----------|-----------------|
95
- | [AI-Scientist](https://github.com/DeadByDawn101/AI-Scientist) | 6,737 | Research automation, hypothesis generation |
96
- | [AutoResearchClaw](https://github.com/DeadByDawn101/AutoResearchClaw) | 3,639 | Automated research pipelines |
97
- | [AutoResearch-MLX](https://github.com/DeadByDawn101/autoresearch-mlx) | 23 | MLX-native research tools |
98
- | [Get-Shit-Done-Redux](https://github.com/DeadByDawn101/get-shit-done-redux) | 4,230 | Task management, agent orchestration |
99
 
100
- ### Infrastructure & Performance
101
 
102
- | Source | Examples | What It Teaches |
103
- |--------|----------|-----------------|
104
- | [TurboQuant-MLX](https://github.com/DeadByDawn101/turboquant-mlx) | 304 | KV cache compression, MLX optimization |
105
- | [TokenSpeed](https://github.com/DeadByDawn101/tokenspeed) | 1,950 | Token-level optimization, inference speed |
106
- | [Auto-Antislop](https://github.com/DeadByDawn101/auto-antislop) | 78 | Output quality control at token level |
107
- | [ADHD](https://github.com/DeadByDawn101/adhd) | 95 | Attention management, focus optimization |
108
- | [Brane-Code](https://github.com/DeadByDawn101/brane-code) | 11 | Distributed compute patterns |
109
- | [Gemma Skills](https://github.com/google-gemma/gemma-skills) | 17 | Google agent skill patterns |
110
 
111
  ---
112
 
@@ -116,9 +172,9 @@ A fine-tuned Qwen3.6-35B-A3B that follows the **RATH protocol** for every securi
116
  Layer 1: Qwen3.6-35B-A3B ← 35B MoE brain (3B active per token)
117
  Layer 2: Claude 4.7 Opus distill ← Enhanced reasoning pre-baked
118
  Layer 3: Abliteration ← Zero refusals for security topics
119
- Layer 4: RavenX-Sec LoRA ← 65,476 proprietary security examples
120
  ═══════════════════
121
- RavenX-Sec v5.0 ← Vision + Security + Reasoning + Speed
122
  ```
123
 
124
  ---
@@ -131,15 +187,17 @@ Layer 4: RavenX-Sec LoRA ← 65,476 proprietary security examples
131
  | **Total Parameters** | 34.66B |
132
  | **Active Parameters** | ~3B per token |
133
  | **Trainable (LoRA)** | 64.1M (0.185%) |
134
- | **Training Data** | 65,476 proprietary examples |
135
- | **Data Sources** | 19 GitHub repos |
 
136
  | **LoRA Rank** | 32 |
137
  | **LoRA Layers** | 4 |
138
  | **Learning Rate** | 1e-5 |
139
  | **Batch Size** | 1 |
140
- | **Max Seq Length** | 2048 |
141
  | **Iterations** | 2,000 |
142
  | **Hardware** | Apple M4 Max 128GB |
 
143
 
144
  ---
145
 
@@ -147,7 +205,7 @@ Layer 4: RavenX-Sec LoRA ← 65,476 proprietary security examples
147
 
148
  | Model | Domain | Protocol | Params | Training Data |
149
  |-------|--------|----------|--------|--------------|
150
- | [**RavenX-Sec v5.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-35B-Security-RATH-mlx) | Security | 6-step RATH | 35B MoE | 65K proprietary |
151
  | [**RavenX-Sec v4.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) | Security | 6-step RATH | 8B | 610K |
152
  | [**RavenX-Trade v1.1**](https://huggingface.co/deadbydawn101/RavenX-Trade-8B-MAP-128k-mlx-4bit) | Trading | 4-step MAP | 8B | 318K |
153
 
 
26
 
27
  # 🐦‍⬛ RavenX-Sec 35B v5.0 — Autonomous Security Intelligence (Vision + MoE)
28
 
29
+ > **MLX** · Apple Silicon · 35B MoE (3B active) · Vision · 262K context · 6-step RATH protocol · 675,696 training examples · Claude 4.7 Opus reasoning distilled
30
 
31
+ **The most powerful open-source security model.** Qwen3.6-35B-A3B with Claude Opus reasoning, abliterated for zero refusals, fine-tuned on 675,696 security examples from 21 HuggingFace datasets + 19 proprietary GitHub repos.
32
 
33
  > 🛡️ **Previous version:** [RavenX-Sec-8B v4.0](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) (610K examples, 8B)
34
  >
 
43
  | Feature | v4.0 (8B) | v5.0 (35B) |
44
  |---------|-----------|------------|
45
  | **Parameters** | 8B dense | **35B MoE (3B active)** |
46
+ | **Vision** | Text only | **Screenshots, diagrams, code** |
47
  | **Base reasoning** | Qwen3-8B | **Claude 4.7 Opus distilled** |
48
  | **Context** | 128K | **262K native** |
49
  | **Uncensoring** | Heretic | **Abliterated (zero refusal)** |
50
+ | **Training data** | 610K (21 HF datasets) | **675,696 (21 HF + 19 repos)** |
51
+ | **Architecture** | Dense | **MoE — 35B brain, 3B speed** |
52
 
53
  ---
54
 
55
+ ## RATH Protocol
56
 
57
+ | Step | What It Does |
58
+ |------|-------------|
59
+ | **R Risk** | Identify vulnerability and attack surface |
 
 
60
  | **A — Assess** | CVSS scoring, CWE classification, severity rating |
61
  | **T — Threat** | MITRE ATT&CK mapping, exploit scenarios, threat actors |
62
  | **H — Highlight** | Remediation steps, code fixes, configuration changes |
 
65
 
66
  ---
67
 
68
+ ## Training Data Complete Inventory (675,696 Examples)
69
+
70
+ ### HuggingFace Datasets (21 Sources — 610,220 Examples)
71
+
72
+ #### Security Instruction Tuning (11 Datasets)
73
+
74
+ | Dataset | Type |
75
+ |---------|------|
76
+ | [Trendyol/Cybersecurity-Instruction-Tuning-Dataset](https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset) | Enterprise cybersecurity instruction tuning |
77
+ | [WNT3D/Ultimate-Offensive-Red-Team](https://huggingface.co/datasets/WNT3D/Ultimate-Offensive-Red-Team) | Offensive red team techniques and procedures |
78
+ | [AYI-NEDJIMI/bug-bounty-pentest-en](https://huggingface.co/datasets/AYI-NEDJIMI/bug-bounty-pentest-en) | Bug bounty and pentesting data |
79
+ | [auren-research/cve-sft-v5](https://huggingface.co/datasets/auren-research/cve-sft-v5) | CVE-specific SFT data |
80
+ | [theelderemo/pentesting-explanations](https://huggingface.co/datasets/theelderemo/pentesting-explanations) | Pentesting technique walkthroughs |
81
+ | [Rootkit7/pentest-redteam-steering](https://huggingface.co/datasets/Rootkit7/pentest-redteam-steering) | Red team steering and methodology |
82
+ | [cpagac/venomx-pentesting-harmful](https://huggingface.co/datasets/cpagac/venomx-pentesting-harmful) | Advanced pentesting scenarios |
83
+ | [SkywardNomad92/pentest-findings-v2](https://huggingface.co/datasets/SkywardNomad92/pentest-findings-v2) | Pentest findings with classification |
84
+ | [acnimatic3722/kali-linux-pentesting-data](https://huggingface.co/datasets/acnimatic3722/kali-linux-pentesting-data) | Kali Linux tool usage and workflows |
85
+ | [CJJones/Synthetic_PenTest_Reports](https://huggingface.co/datasets/CJJones/Synthetic_PenTest_Reports) | Synthetic penetration testing reports |
86
+ | [nangyall/4-Security-Tools-Pentesting](https://huggingface.co/datasets/nangyall/4-Security-Tools-Pentesting) | Security tools usage (nmap, burp, etc) |
87
+
88
+ #### Vulnerability & Code Security (4 Datasets)
89
+
90
+ | Dataset | Type |
91
+ |---------|------|
92
+ | [ByteDance/PatchEval](https://huggingface.co/datasets/ByteDance/PatchEval) | 1000 real-world CVE patches, 65 CWE categories |
93
+ | [bigcode/vuln-eval](https://huggingface.co/datasets/bigcode/vuln-eval) | Vulnerability recognition and repair |
94
+ | [Fraser/cwe-benchmark](https://huggingface.co/datasets/Fraser/cwe-benchmark) | Source code mapped to CWE IDs |
95
+ | [isek/cybersecurity-instructions](https://huggingface.co/datasets/isek/cybersecurity-instructions) | RE walkthroughs, security scripting |
96
+
97
+ #### Agentic & Reasoning (3 Datasets)
98
+
99
+ | Dataset | Type |
100
+ |---------|------|
101
+ | [WithinUsAI/claude_mythos_distilled_25k](https://huggingface.co/datasets/WithinUsAI/claude_mythos_distilled_25k) | Distilled reasoning (cybersecurity, agentic planning) |
102
+ | [WithinUsAI/AgentAngel_100k](https://huggingface.co/datasets/WithinUsAI/AgentAngel_100k) | Agentic coding (plan, patch, verify, iterate) |
103
+ | [HackerSignal/Threat-Intel](https://huggingface.co/datasets/HackerSignal/Threat-Intel) | Historical cybersecurity docs, exploit scripts |
104
+
105
+ #### Infrastructure & Code (3 Datasets)
106
+
107
+ | Dataset | Type |
108
+ |---------|------|
109
+ | [JetBrains-Research/commit-chronicle](https://huggingface.co/datasets/JetBrains-Research/commit-chronicle) | Git commits, security-tagged patches |
110
+ | [bigcode/commitpack](https://huggingface.co/datasets/bigcode/commitpack) | Code changes across 300+ languages |
111
+ | [bigcode/the-stack-v2](https://huggingface.co/datasets/bigcode/the-stack-v2) | Infrastructure-as-code (Ansible, Terraform, Shell) |
112
+
113
+ ---
114
+
115
+ ### Proprietary GitHub Repos (19 Sources — 65,476 Examples)
116
+
117
+ #### Autonomous Pentesting & Security (4 Repos)
118
+
119
+ | Repo | Examples | Content |
120
+ |------|----------|---------|
121
+ | [DeadByDawn101/phalanx](https://github.com/DeadByDawn101/phalanx) | 65 | SWARM pentesting agents, RoE enforcement, MITRE ATT&CK |
122
+ | [DeadByDawn101/chrome-devtools-mcp](https://github.com/DeadByDawn101/chrome-devtools-mcp) | 194 | Browser debugging, MCP protocol, security headers |
123
+ | [DeadByDawn101/camofox-mcp](https://github.com/DeadByDawn101/camofox-mcp) | 134 | Anti-detection, browser fingerprint evasion |
124
+ | [DeadByDawn101/phantom](https://github.com/DeadByDawn101/phantom) | 662 | Autonomous agent architecture, security model |
125
+
126
+ #### Agent Frameworks (5 Repos)
127
+
128
+ | Repo | Examples | Content |
129
+ |------|----------|---------|
130
+ | [nousresearch/hermes-agent](https://github.com/nousresearch/hermes-agent) | 42,929 | NousResearch self-improving agent patterns |
131
+ | [kilo-org/kilocode](https://github.com/kilo-org/kilocode) | 3,224 | Agent framework, tool calling, code execution |
132
+ | [Gitlawb/openclaude](https://github.com/Gitlawb/openclaude) | 310 | Coding agent patterns, task decomposition |
133
+ | [DeadByDawn101/self-improving-agent](https://github.com/DeadByDawn101/self-improving-agent) | 131 | Agent learning loops, self-correction |
134
+ | [DeadByDawn101/self_improving_coding_agent](https://github.com/DeadByDawn101/self_improving_coding_agent) | 743 | Code generation, testing, self-improvement |
135
 
136
+ #### Research & Automation (4 Repos)
137
 
138
+ | Repo | Examples | Content |
139
+ |------|----------|---------|
140
+ | [DeadByDawn101/AI-Scientist](https://github.com/DeadByDawn101/AI-Scientist) | 6,737 | Research automation, hypothesis generation |
141
+ | [DeadByDawn101/AutoResearchClaw](https://github.com/DeadByDawn101/AutoResearchClaw) | 3,639 | Automated research pipelines |
142
+ | [DeadByDawn101/autoresearch-mlx](https://github.com/DeadByDawn101/autoresearch-mlx) | 23 | MLX-native research tools |
143
+ | [DeadByDawn101/get-shit-done-redux](https://github.com/DeadByDawn101/get-shit-done-redux) | 4,230 | Task management, agent orchestration |
144
 
145
+ #### Performance & Optimization (4 Repos)
146
 
147
+ | Repo | Examples | Content |
148
+ |------|----------|---------|
149
+ | [DeadByDawn101/turboquant-mlx](https://github.com/DeadByDawn101/turboquant-mlx) | 304 | KV cache compression, MLX optimization |
150
+ | [DeadByDawn101/tokenspeed](https://github.com/DeadByDawn101/tokenspeed) | 1,950 | Token-level optimization, inference speed |
151
+ | [DeadByDawn101/auto-antislop](https://github.com/DeadByDawn101/auto-antislop) | 78 | Output quality control at token level |
152
+ | [DeadByDawn101/adhd](https://github.com/DeadByDawn101/adhd) | 95 | Attention management, focus optimization |
 
153
 
154
+ #### Infrastructure (2 Repos)
155
 
156
+ | Repo | Examples | Content |
157
+ |------|----------|---------|
158
+ | [DeadByDawn101/brane-code](https://github.com/DeadByDawn101/brane-code) | 11 | Distributed compute patterns |
159
+ | [google-gemma/gemma-skills](https://github.com/google-gemma/gemma-skills) | 17 | Google agent skill patterns |
 
 
160
 
161
+ #### RavenX Internal (Self-Extraction)
162
 
163
+ | Source | Examples | Content |
164
+ |--------|----------|---------|
165
+ | RavenX-Sec repo (tools, docs) | 120 | RATH protocol, LEWM security, deployment guides |
 
 
 
 
 
166
 
167
  ---
168
 
 
172
  Layer 1: Qwen3.6-35B-A3B ← 35B MoE brain (3B active per token)
173
  Layer 2: Claude 4.7 Opus distill ← Enhanced reasoning pre-baked
174
  Layer 3: Abliteration ← Zero refusals for security topics
175
+ Layer 4: RavenX-Sec LoRA ← 675,696 security examples (RATH)
176
  ═══════════════════
177
+ RavenX-Sec v5.0 ← Vision + Security + Reasoning
178
  ```
179
 
180
  ---
 
187
  | **Total Parameters** | 34.66B |
188
  | **Active Parameters** | ~3B per token |
189
  | **Trainable (LoRA)** | 64.1M (0.185%) |
190
+ | **Training Data** | 675,696 examples |
191
+ | **HuggingFace Datasets** | 21 sources (610,220 examples) |
192
+ | **GitHub Repos** | 19 sources (65,476 examples) |
193
  | **LoRA Rank** | 32 |
194
  | **LoRA Layers** | 4 |
195
  | **Learning Rate** | 1e-5 |
196
  | **Batch Size** | 1 |
197
+ | **Max Seq Length** | 1024 |
198
  | **Iterations** | 2,000 |
199
  | **Hardware** | Apple M4 Max 128GB |
200
+ | **Peak Memory** | ~110GB |
201
 
202
  ---
203
 
 
205
 
206
  | Model | Domain | Protocol | Params | Training Data |
207
  |-------|--------|----------|--------|--------------|
208
+ | [**RavenX-Sec v5.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-35B-Security-RATH-mlx) | Security | 6-step RATH | 35B MoE | 675K (21 HF + 19 repos) |
209
  | [**RavenX-Sec v4.0**](https://huggingface.co/deadbydawn101/RavenX-Sec-8B-Security-RATH-128k-mlx-4bit) | Security | 6-step RATH | 8B | 610K |
210
  | [**RavenX-Trade v1.1**](https://huggingface.co/deadbydawn101/RavenX-Trade-8B-MAP-128k-mlx-4bit) | Trading | 4-step MAP | 8B | 318K |
211