README.md

---
language:
- en
license: llama3
tags:
- cybersecurity
- llama3
- fine-tuned
- bug-bounty
- penetration-testing
- vulnerability-research
- bug bounty reports
base_model: meta-llama/Meta-Llama-3-8B
datasets:
- custom
pipeline_tag: text-generation
---

# 🔒 Llama 3 Cybersecurity Model (LoRA Adapters)

Fine-tuned Llama 3 8B model specialized in cybersecurity, vulnerability research, and penetration testing.

## Model Description

This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on a curated dataset of 16,980 cybersecurity examples from twitter, linkedin and bug bounty reports.

**Model Type**: LoRA Adapters
**Base Model**: meta-llama/Meta-Llama-3-8B
**Training Data**: 16,980 examples from twitter, linkedin and bug bounty reports
**Training Method**: QLoRA (Quantized Low-Rank Adaptation)
**Training Duration**: 3 epochs

## Intended Use

This model is designed for:
- 🔍 Vulnerability research and analysis
- 🛡️ Security testing and penetration testing
- 📝 Security documentation and reporting
- 🎓 Cybersecurity education and training
- 🔬 Security research and experimentation

## Training Details

### Training Data
- **Source**: twitter, linkedin and bug bounty reports
- **Size**: 16,980 training examples + validation set
- **Format**: Instruction-response pairs
- **Topics**: Web vulnerabilities, API security, authentication, injection attacks, XSS, CSRF, etc.

### Training Configuration
- **Method**: QLoRA (4-bit quantization)
- **LoRA Rank**: 16
- **LoRA Alpha**: 32
- **LoRA Dropout**: 0.05
- **Target Modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Epochs**: 3
- **Batch Size**: 4 (per device)
- **Gradient Accumulation**: 4 steps
- **Learning Rate**: 2e-4
- **Optimizer**: paged_adamw_8bit
- **Scheduler**: Cosine with warmup

### Hardware
- **GPU**: NVIDIA RTX 4090 (24GB)
- **Training Time**: ~12 hours
- **Platform**: RunPod

## Usage

### Option 1: Using LoRA Adapters (Recommended for fine-tuning)

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = "meta-llama/Meta-Llama-3-8B"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "bugdisclose/llama3-hacker-lora")

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("bugdisclose/llama3-hacker-lora")

# Generate
prompt = """### System:
You are a cybersecurity expert assistant.

### Instruction:
give me XSS payload

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("### Response:")[1].strip())
```

### Option 2: Using Merged Model (Faster inference)

```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load merged model
model = AutoModelForCausalLM.from_pretrained(
    "bugdisclose/llama3-hacker-lora",
    torch_dtype=torch.float16,
    device_map="auto",
)

tokenizer = AutoTokenizer.from_pretrained("bugdisclose/llama3-hacker-lora")

# Generate
prompt = """### System:
You are a cybersecurity expert assistant.

### Instruction:
Generate exploit for https://example.com/user?id=1

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("### Response:")[1].strip())
```

## Prompt Format

This model uses the **BASE model format** (not Instruct format):

```
### System:
You are a cybersecurity expert assistant.

### Instruction:
[Your question or request here]

### Response:
```

## Example Queries

- "give me XSS payload"
- "generate test case for https://example.com/user?id=1"
- "how to find SQL injection vulnerability"
- "explain CSRF attack with example"
- "what are common authentication bypass techniques"
- "generate payload for command injection"
- "how to test for XXE vulnerability"

## Limitations

- **Specialized Domain**: Optimized for cybersecurity; may not perform well on general tasks
- **Ethical Use Only**: Intended for authorized security testing and research
- **No Guarantees**: Generated content should be validated by security professionals
- **Training Data Bias**: Reflects patterns from bug bounty reports (web-focused)

## Ethical Considerations

⚠️ **IMPORTANT**: This model is for educational and authorized security testing purposes only.

- ✅ Use for authorized penetration testing
- ✅ Use for security research and education
- ✅ Use for improving security posture
- ❌ Do NOT use for unauthorized access
- ❌ Do NOT use for malicious purposes
- ❌ Do NOT use without proper authorization

Always obtain explicit permission before testing any systems.

## Model Card Authors

bugdisclose

## Citation

If you use this model, please cite:

```bibtex
@misc{llama3-cybersecurity-llama3-hacker-lora,
  author = {bugdisclose},
  title = {Llama 3 Cybersecurity Model},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bugdisclose/llama3-hacker-lora}}
}
```

## License

This model inherits the Llama 3 license. See [Meta's Llama 3 License](https://huggingface.co/meta-llama/Meta-Llama-3-8B) for details.

## Acknowledgments

- **Base Model**: Meta's Llama 3 8B
- **Training Framework**: Hugging Face Transformers, PEFT, bitsandbytes
- **Infrastructure**: RunPod

---

**Disclaimer**: This model is provided as-is for research and educational purposes. Users are responsible for ensuring ethical and legal use.