Text Generation
PEFT
Safetensors
English
lora
molly-os
specialist
security-audit
llama-3.1
domain-adaptation
conversational
Instructions to use BoomJules/molly-security-audit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use BoomJules/molly-security-audit with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") model = PeftModel.from_pretrained(base_model, "BoomJules/molly-security-audit") - Notebooks
- Google Colab
- Kaggle
docs: specialist card with usage, examples and family index
Browse files
README.md
CHANGED
|
@@ -1,47 +1,61 @@
|
|
| 1 |
---
|
| 2 |
library_name: peft
|
| 3 |
base_model: meta-llama/Llama-3.1-8B-Instruct
|
| 4 |
-
license: cc-by-nc-4.0
|
| 5 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
| 6 |
tags:
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
# Molly
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
Domain: **Security Audit**.
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
## Before you run: the base model is gated
|
| 24 |
|
| 25 |
This adapter needs the base weights, and the base is **access-gated**. Do this **once**:
|
| 26 |
|
| 27 |
-
1.
|
| 28 |
2. Create a **read token**: <https://huggingface.co/settings/tokens>
|
| 29 |
-
3. Make the token available
|
| 30 |
-
- **Google Colab:**
|
| 31 |
-
- **Kaggle:** *Add-ons
|
| 32 |
-
- **Local:**
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|
| 36 |
|
| 37 |
## Quickstart
|
| 38 |
|
| 39 |
```python
|
| 40 |
# pip install -U transformers peft accelerate
|
| 41 |
-
import os
|
| 42 |
from huggingface_hub import login
|
| 43 |
-
|
| 44 |
-
# Authenticate (Colab secret -> env var -> interactive prompt)
|
| 45 |
try:
|
| 46 |
from google.colab import userdata
|
| 47 |
login(userdata.get("HF_TOKEN"))
|
|
@@ -49,7 +63,6 @@ except Exception:
|
|
| 49 |
tok = os.environ.get("HF_TOKEN")
|
| 50 |
login(tok) if tok else login()
|
| 51 |
|
| 52 |
-
import torch
|
| 53 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 54 |
from peft import PeftModel
|
| 55 |
|
|
@@ -66,9 +79,7 @@ out = model.generate(ids, max_new_tokens=300)
|
|
| 66 |
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
|
| 67 |
```
|
| 68 |
|
| 69 |
-
## Low-VRAM (4-bit)
|
| 70 |
-
|
| 71 |
-
Use a **GPU runtime** (Colab: *Runtime -> Change runtime type -> T4 GPU*).
|
| 72 |
|
| 73 |
```python
|
| 74 |
# pip install -U transformers peft accelerate bitsandbytes
|
|
@@ -83,26 +94,50 @@ except Exception:
|
|
| 83 |
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
| 84 |
from peft import PeftModel
|
| 85 |
|
| 86 |
-
BASE = "meta-llama/Llama-3.1-8B-Instruct"
|
| 87 |
-
ADAPTER = "BoomJules/molly-security-audit"
|
| 88 |
-
|
| 89 |
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
|
| 90 |
bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
|
| 91 |
-
tok = AutoTokenizer.from_pretrained(
|
| 92 |
-
base = AutoModelForCausalLM.from_pretrained(
|
| 93 |
-
model = PeftModel.from_pretrained(base,
|
| 94 |
```
|
| 95 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
## Troubleshooting
|
| 97 |
|
| 98 |
-
- **`GatedRepoError` / `401 Unauthorized`**
|
| 99 |
-
|
| 100 |
-
- **CUDA out of memory**
|
| 101 |
-
- **Adapter seems to have no effect**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
|
| 103 |
-
##
|
| 104 |
|
| 105 |
-
Adapter: **CC BY-NC 4.0** (attribution, non-commercial). Base model:
|
| 106 |
Intended for research and evaluation in Security Audit.
|
| 107 |
|
| 108 |
-
|
|
|
|
| 1 |
---
|
| 2 |
library_name: peft
|
| 3 |
base_model: meta-llama/Llama-3.1-8B-Instruct
|
|
|
|
| 4 |
pipeline_tag: text-generation
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
license: cc-by-nc-4.0
|
| 8 |
tags:
|
| 9 |
+
- lora
|
| 10 |
+
- peft
|
| 11 |
+
- molly-os
|
| 12 |
+
- specialist
|
| 13 |
+
- security-audit
|
| 14 |
+
- llama-3.1
|
| 15 |
+
- domain-adaptation
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# Molly Specialist β Security Audit
|
| 19 |
+
|
| 20 |
+
Identifies vulnerabilities in source code, configurations, and infrastructure-as-code with CWE-mapped findings and remediation steps exceeding base model specificity.
|
| 21 |
+
|
| 22 |
+
Part of **[Molly](https://iamolly.ai/?utm_source=huggingface&utm_medium=model_card&utm_campaign=specialists&utm_content=molly-security-audit)**, an orchestrator that keeps a library of small domain
|
| 23 |
+
specialists over one quantized base and routes each request to the right one, so a
|
| 24 |
+
single machine answers across many fields without loading a separate large model
|
| 25 |
+
for each.
|
| 26 |
+
|
| 27 |
+
## What this specialist handles well
|
| 28 |
|
| 29 |
+
- Maps code vulnerabilities to specific CWE IDs with severity ratings
|
| 30 |
+
- Generates compliance gap analyses for SOC 2 and ISO 27001
|
| 31 |
+
- Reviews IAM policies for privilege escalation and over-permissive access
|
|
|
|
| 32 |
|
| 33 |
+
## Try it with
|
| 34 |
+
|
| 35 |
+
- "Audit this Terraform configuration for misconfigurations and security best practices violations"
|
| 36 |
+
- "What CWE categories apply to this SQL injection vulnerability in the login handler?"
|
| 37 |
+
- "Generate a SOC 2 access control checklist for a multi-tenant SaaS application"
|
| 38 |
|
| 39 |
## Before you run: the base model is gated
|
| 40 |
|
| 41 |
This adapter needs the base weights, and the base is **access-gated**. Do this **once**:
|
| 42 |
|
| 43 |
+
1. Accept the base licence: <https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct>
|
| 44 |
2. Create a **read token**: <https://huggingface.co/settings/tokens>
|
| 45 |
+
3. Make the token available:
|
| 46 |
+
- **Google Colab:** Secrets panel (key icon) β *Add new secret* β name `HF_TOKEN`, enable **Notebook access**.
|
| 47 |
+
- **Kaggle:** *Add-ons β Secrets* β add `HF_TOKEN`.
|
| 48 |
+
- **Local:** `huggingface-cli login` or `export HF_TOKEN=...`
|
| 49 |
|
| 50 |
+
Skipping this gives `GatedRepoError` / `401 Unauthorized` when the **base** loads. A stored
|
| 51 |
+
Colab secret is **not** applied automatically β authenticate in code, as below.
|
| 52 |
|
| 53 |
## Quickstart
|
| 54 |
|
| 55 |
```python
|
| 56 |
# pip install -U transformers peft accelerate
|
| 57 |
+
import os, torch
|
| 58 |
from huggingface_hub import login
|
|
|
|
|
|
|
| 59 |
try:
|
| 60 |
from google.colab import userdata
|
| 61 |
login(userdata.get("HF_TOKEN"))
|
|
|
|
| 63 |
tok = os.environ.get("HF_TOKEN")
|
| 64 |
login(tok) if tok else login()
|
| 65 |
|
|
|
|
| 66 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 67 |
from peft import PeftModel
|
| 68 |
|
|
|
|
| 79 |
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
|
| 80 |
```
|
| 81 |
|
| 82 |
+
## Low-VRAM (4-bit) β fits a free Colab/Kaggle GPU (~6β7 GB)
|
|
|
|
|
|
|
| 83 |
|
| 84 |
```python
|
| 85 |
# pip install -U transformers peft accelerate bitsandbytes
|
|
|
|
| 94 |
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
| 95 |
from peft import PeftModel
|
| 96 |
|
|
|
|
|
|
|
|
|
|
| 97 |
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
|
| 98 |
bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
|
| 99 |
+
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
|
| 100 |
+
base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", quantization_config=bnb, device_map="auto")
|
| 101 |
+
model = PeftModel.from_pretrained(base, "BoomJules/molly-security-audit").eval()
|
| 102 |
```
|
| 103 |
|
| 104 |
+
## Adapter details
|
| 105 |
+
|
| 106 |
+
| | |
|
| 107 |
+
|---|---|
|
| 108 |
+
| Base model | `meta-llama/Llama-3.1-8B-Instruct` |
|
| 109 |
+
| Method | LoRA (PEFT) |
|
| 110 |
+
| Rank / alpha | 32 / 64 |
|
| 111 |
+
| Domain | Security Audit |
|
| 112 |
+
|
| 113 |
## Troubleshooting
|
| 114 |
|
| 115 |
+
- **`GatedRepoError` / `401 Unauthorized`** β base licence not accepted, or `HF_TOKEN` missing,
|
| 116 |
+
or the Colab secret was stored but `login(...)` was never called.
|
| 117 |
+
- **CUDA out of memory** β use the 4-bit snippet on a GPU runtime.
|
| 118 |
+
- **Adapter seems to have no effect** β confirm the base id matches `base_model` above.
|
| 119 |
+
|
| 120 |
+
## Other Molly specialists
|
| 121 |
+
|
| 122 |
+
- [Quantum Software Architect](https://huggingface.co/BoomJules/molly-quantum-software-architect)
|
| 123 |
+
- [Quantum Communication Systems Engineer](https://huggingface.co/BoomJules/molly-quantum-communication-systems-engineer)
|
| 124 |
+
- [Infectious Disease Physician Antimicrobial Stewardship](https://huggingface.co/BoomJules/molly-infectious-disease-physician-antimicrobial-stewardship)
|
| 125 |
+
- [Health Informatics Medical AI Specialist](https://huggingface.co/BoomJules/molly-health-informatics-medical-ai-specialist)
|
| 126 |
+
- [Clinical Trial Pharmacologist](https://huggingface.co/BoomJules/molly-clinical-trial-pharmacologist)
|
| 127 |
+
- [Immunopharmacologist](https://huggingface.co/BoomJules/molly-immunopharmacologist)
|
| 128 |
+
- [Climate Analytics Manager](https://huggingface.co/BoomJules/molly-climate-analytics-manager)
|
| 129 |
+
- [Language Technology Consultant](https://huggingface.co/BoomJules/molly-language-technology-consultant)
|
| 130 |
+
- [Polymer Chemist](https://huggingface.co/BoomJules/molly-polymer-chemist)
|
| 131 |
+
- [Composite Materials Engineer](https://huggingface.co/BoomJules/molly-composite-materials-engineer)
|
| 132 |
+
- [Computer Science AI](https://huggingface.co/BoomJules/molly-cs-ai)
|
| 133 |
+
- [Computer Science Algorithms](https://huggingface.co/BoomJules/molly-cs-algorithms)
|
| 134 |
+
|
| 135 |
+
Running several of these at once, with the routing decided for you, is what
|
| 136 |
+
[Molly](https://iamolly.ai/?utm_source=huggingface&utm_medium=model_card&utm_campaign=specialists&utm_content=molly-security-audit) does.
|
| 137 |
|
| 138 |
+
## Licence & intended use
|
| 139 |
|
| 140 |
+
Adapter: **CC BY-NC 4.0** (attribution, non-commercial). Base model: its own licence.
|
| 141 |
Intended for research and evaluation in Security Audit.
|
| 142 |
|
| 143 |
+
Β© 2026 Core Labs R&D.
|