Update README.md
Browse files
README.md
CHANGED
|
@@ -11,13 +11,49 @@ model-index:
|
|
| 11 |
results: []
|
| 12 |
---
|
| 13 |
|
| 14 |
-
|
| 15 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|
| 18 |
<details><summary>See axolotl config</summary>
|
| 19 |
|
| 20 |
-
axolotl version: `0.13.0.dev0`
|
| 21 |
```yaml
|
| 22 |
base_model: Qwen/Qwen3-8B
|
| 23 |
model_type: Qwen3ForCausalLM
|
|
@@ -91,48 +127,34 @@ wandb_mode: disabled
|
|
| 91 |
|
| 92 |
</details><br>
|
| 93 |
|
| 94 |
-
|
| 95 |
|
| 96 |
-
|
| 97 |
|
| 98 |
-
|
|
|
|
|
|
|
| 99 |
|
| 100 |
-
|
| 101 |
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
## Training procedure
|
| 111 |
-
|
| 112 |
-
### Training hyperparameters
|
| 113 |
-
|
| 114 |
-
The following hyperparameters were used during training:
|
| 115 |
-
- learning_rate: 1.5e-05
|
| 116 |
-
- train_batch_size: 4
|
| 117 |
-
- eval_batch_size: 4
|
| 118 |
-
- seed: 42
|
| 119 |
-
- distributed_type: multi-GPU
|
| 120 |
-
- num_devices: 4
|
| 121 |
-
- gradient_accumulation_steps: 2
|
| 122 |
-
- total_train_batch_size: 32
|
| 123 |
-
- total_eval_batch_size: 16
|
| 124 |
-
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
| 125 |
-
- lr_scheduler_type: cosine
|
| 126 |
-
- lr_scheduler_warmup_steps: 200
|
| 127 |
-
- training_steps: 2307
|
| 128 |
-
|
| 129 |
-
### Training results
|
| 130 |
|
|
|
|
| 131 |
|
|
|
|
| 132 |
|
| 133 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
|
| 135 |
-
|
| 136 |
-
- Pytorch 2.7.1+cu126
|
| 137 |
-
- Datasets 4.0.0
|
| 138 |
-
- Tokenizers 0.22.1
|
|
|
|
| 11 |
results: []
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# KillChain-8B
|
|
|
|
| 15 |
|
| 16 |
+
This model is a fully fine-tuned version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) on the [WNT3D/Ultimate-Offensive-Red-Team](https://huggingface.co/datasets/WNT3D/Ultimate-Offensive-Red-Team) dataset.
|
| 17 |
+
|
| 18 |
+
## Intended uses & limitations
|
| 19 |
+
|
| 20 |
+
KillChain-8B is intended for:
|
| 21 |
+
|
| 22 |
+
- Red-team simulation and research
|
| 23 |
+
- Security training and tabletop exercises
|
| 24 |
+
- Adversarial LLM evaluation
|
| 25 |
+
- Controlled internal testing environments
|
| 26 |
+
- Studying failure modes of aligned models
|
| 27 |
+
|
| 28 |
+
### Training hyperparameters
|
| 29 |
+
|
| 30 |
+
The following hyperparameters were used during training:
|
| 31 |
+
- learning_rate: 1.5e-05
|
| 32 |
+
- train_batch_size: 4
|
| 33 |
+
- eval_batch_size: 4
|
| 34 |
+
- seed: 42
|
| 35 |
+
- distributed_type: multi-GPU
|
| 36 |
+
- num_devices: 4
|
| 37 |
+
- gradient_accumulation_steps: 2
|
| 38 |
+
- total_train_batch_size: 32
|
| 39 |
+
- total_eval_batch_size: 16
|
| 40 |
+
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
|
| 41 |
+
- lr_scheduler_type: cosine
|
| 42 |
+
- lr_scheduler_warmup_steps: 200
|
| 43 |
+
- training_steps: 2307
|
| 44 |
+
|
| 45 |
+
### Framework versions
|
| 46 |
+
|
| 47 |
+
- Transformers 4.57.0
|
| 48 |
+
- Pytorch 2.7.1+cu126
|
| 49 |
+
- Datasets 4.0.0
|
| 50 |
+
- Tokenizers 0.22.1
|
| 51 |
+
|
| 52 |
+
Command used: accelerate launch --num_processes=4 -m axolotl.cli.train config.yaml
|
| 53 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|
| 54 |
<details><summary>See axolotl config</summary>
|
| 55 |
|
| 56 |
+
axolotl version: `0.13.0.dev0`
|
| 57 |
```yaml
|
| 58 |
base_model: Qwen/Qwen3-8B
|
| 59 |
model_type: Qwen3ForCausalLM
|
|
|
|
| 127 |
|
| 128 |
</details><br>
|
| 129 |
|
| 130 |
+
## Usage
|
| 131 |
|
| 132 |
+
### Transformers (Python)
|
| 133 |
|
| 134 |
+
```python
|
| 135 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 136 |
+
import torch
|
| 137 |
|
| 138 |
+
model_id = "MrPibb/KillChain-8B"
|
| 139 |
|
| 140 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
|
| 141 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 142 |
+
model_id,
|
| 143 |
+
torch_dtype=torch.bfloat16,
|
| 144 |
+
device_map="auto",
|
| 145 |
+
trust_remote_code=True,
|
| 146 |
+
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
+
prompt = "Provide a list of twenty XSS payloads."
|
| 149 |
|
| 150 |
+
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
| 151 |
|
| 152 |
+
outputs = model.generate(
|
| 153 |
+
**inputs,
|
| 154 |
+
max_new_tokens=512,
|
| 155 |
+
temperature=0.7,
|
| 156 |
+
top_p=0.9,
|
| 157 |
+
do_sample=True,
|
| 158 |
+
)
|
| 159 |
|
| 160 |
+
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
|
|
|
|
|
|
|