| --- |
| base_model: unsloth/llama-3-8b-Instruct-bnb-4bit |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: |
| - base_model:adapter:unsloth/llama-3-8b-Instruct-bnb-4bit |
| - grpo |
| - lora |
| - sft |
| - transformers |
| - trl |
| - unsloth |
| - cybersecurity |
| - web-pentesting |
| - autonomous-agent |
| --- |
| |
| # SENTINEL — Llama-3-8B QLoRA SFT + GRPO Fine-tune |
|
|
| This repository contains the **Final (QLoRA SFT + GRPO)** adapter weights for **SENTINEL**, an autonomous web-exploitation agent. |
|
|
| ## Model Overview |
|
|
| SENTINEL is trained to autonomously navigate, analyze, and exploit web vulnerabilities using a structured JSON-based reasoning and action schema . |
|
|
| This repository represents the completion of **both Stage 1 and Stage 2**: |
| * **Stage 1:** QLoRA SFT on 415 SENTINEL trajectory pairs. This stage teaches the agent the correct JSON schema, action vocabulary, and basic reasoning patterns for web exploitation. |
| * **Stage 2:** GRPO (Generative Reward Proximal Policy Optimization). This stage reinforces successful exploitation pathways, heavily penalizing JSON formatting errors and hallucinated actions, while rewarding verified vulnerability exploitation. |
|
|
| ## Training Details |
|
|
| The model was fine-tuned using [Unsloth](https://github.com/unslothai/unsloth) for optimized, 2x faster training. |
|
|
| ### Hardware & Configuration |
| - **GPU:** 1x Tesla T4 (16GB VRAM) |
| - **Precision:** FP16 (bf16=False) |
| - **Base Model:** `unsloth/llama-3-8b-Instruct-bnb-4bit` |
| - **Training Time:** ~45 minutes |
|
|
| ### Hyperparameters |
| - **Num Examples:** 415 |
| - **Epochs:** 2 |
| - **Total Steps:** 104 |
| - **Batch Size per Device:** 2 |
| - **Gradient Accumulation Steps:** 4 |
| - **Total Effective Batch Size:** 8 |
| - **Trainable Parameters:** 41,943,040 of 8,072,204,288 (0.52% trained) |
|
|
| ### Results |
| - **Final Training Loss:** 1.0764 |
| - **Final Validation Loss:** 1.1465 |
|
|
| *Loss curve during training:* |
| | Step | Training Loss | Validation Loss | |
| |------|---------------|-----------------| |
| | 20 | 1.333500 | 1.492488 | |
| | 40 | 1.127200 | 1.263251 | |
| | 60 | 0.689200 | 1.210084 | |
| | 80 | 0.724800 | 1.163273 | |
| | 100 | 0.763000 | 1.146508 | |
|
|
| ## Dataset |
|
|
| The model was trained on a custom dataset (`train_llama3.jsonl`) consisting of **415 SENTINEL trajectory pairs**. These trajectories represent successful pentesting workflows, teaching the model how to target vulnerability sinks (form actions, hidden fields, query parameters, JSON bodies, etc.), infer backend technologies, and deliver appropriate payloads. |
|
|
| ## Usage |
|
|
| Because this is a PEFT (Parameter-Efficient Fine-Tuning) adapter, you must load the base model (`unsloth/llama-3-8b-Instruct-bnb-4bit` or the standard `meta-llama/Meta-Llama-3-8B-Instruct`) and apply these LoRA weights on top using the `peft` library. |
|
|
| ### Framework Versions |
| - PEFT 0.19.1 |
| - Transformers |
| - Unsloth |
