| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - cybersecurity |
| - pentesting |
| - security |
| - uncensored |
| - tool-calling |
| - gguf |
| - llama-cpp |
| pipeline_tag: text-generation |
| --- |
| |
| # Nesi2 — Uncensored AI for Cybersecurity (GGUF / llama.cpp) |
|
|
| **Nesi2** is an uncensored large language model, curated and optimized by |
| **Nesilabs** specifically for **offensive and defensive cybersecurity**: |
| penetration testing, vulnerability analysis, exploit development, secure |
| code review, and audit automation. |
|
|
| It is packaged by Nesilabs as a **GGUF** build for **llama.cpp**, so it runs on a |
| single GPU without an AWQ/vLLM stack. This is the build we run as an autonomous |
| **pentest agent**: it reliably reaches code-exec on multi-stage targets |
| (web → WebSocket tool abuse → RCE) with native tool-calling. |
|
|
| ## Highlights |
| - **Unrestricted** for legitimate security use cases. |
| - **~27B parameters**, long context (up to **240k tokens** in production), native tool-calling. |
| - Two GGUF builds: **Q8_0** (≈lossless, production quality) and **Q6_K** (lighter). |
| - Ships the **MTP** draft weights (`*-mtp*.gguf`) for optional speculative decode. |
| - Validated end-to-end as a **pentest agent** (live RCE + loot + flag on a 3-stage CTF). |
|
|
| ## Model details |
| | | | |
| |---|---| |
| | Format | **GGUF** (llama.cpp) | |
| | Quantizations | **Q8_0** (~29 GB) · **Q6_K** (~22 GB) | |
| | Context length | up to 262,144 tokens (240k used in prod) | |
| | Tool-calling | OpenAI-compatible (native in llama.cpp `llama-server`) | |
| | Serving | **llama.cpp `llama-server`** (OpenAI `/v1` compatible) | |
| | KV cache | bf16 by default; `q8_0` to roughly halve footprint | |
|
|
| > Note: the Hub's "Architecture" widget reads the GGUF's internal loader tag |
| > (`general.architecture`); that is only a llama.cpp loader key baked into the |
| > file and does not reflect the model name. The model is **Nesi2**. |
|
|
| ## Files |
| | File | What | |
| |---|---| |
| | `Nesi2-Q8_0.gguf` | the model, **Q8_0** (max quality) | |
| | `Nesi2-mtp-Q8_0.gguf` | MTP draft head for Q8 (optional, `--model-draft`) | |
| | `Nesi2-Q6_K.gguf` | the model, **Q6_K** (lighter) | |
| | `Nesi2-Q6_K-mtp.gguf` | Q6_K with the MTP head already merged (single file) | |
| |
| --- |
| |
| ## Two configurations |
| |
| Both serve the same `nesi2` model over an OpenAI-compatible `/v1`. Pick by VRAM |
| and quality target. |
| |
| | Config | Files | VRAM (weights + MTP) | When | |
| |---|---|---|---| |
| | **A — Q8_0 + MTP** | `Nesi2-Q8_0.gguf` + `Nesi2-mtp-Q8_0.gguf` | ~29 + ~3 GB | max quality, 96 GB card | |
| | **B — Q6_K (+MTP)** | `Nesi2-Q6_K.gguf` (or `Nesi2-Q6_K-mtp.gguf`) | ~22 GB | lighter, more slots / smaller card | |
| |
| > **MTP (speculative decode)** needs a llama.cpp build that supports |
| > `--spec-type draft-mtp`. It helps **single-stream latency**; under heavy |
| > concurrency it did not improve aggregate throughput in our tests, so for |
| > multi-user serving you can run without it and keep it for low-latency use. |
| |
| --- |
| |
| ## Deploy on a GPU box from zero |
| |
| Everything below runs **on the GPU box as root**. The gateway bits run on your |
| LiteLLM host. Target image: `nvidia/cuda:12.8.1-devel-ubuntu22.04`, ≥ 60 GB disk |
| (≥ 90 GB if you want Q8 **and** Q6 on the same box). |
| |
| ### 0) Pick a GPU |
| | GPU | VRAM | Fits | single-stream tok/s (240k slot) | |
| |---|---|---|---| |
| | RTX A6000 | 48 GB | 1 user @ 240k (Q6_K) | ~22 | |
| | RTX PRO 6000 **Max-Q** (power-capped ~300W) | 96 GB | 1–3 users @ 240k | ~20–25 (gateway end-to-end) | |
| | RTX PRO 6000 **WS** (full ~600W) | 96 GB | up to 5 users @ 240k (Q8+MTP) | **~55** | |
|
|
| > Blackwell (PRO 6000, `sm_120`) needs CUDA toolkit **≥ 12.8**; build llama.cpp |
| > with `-DCMAKE_CUDA_ARCHITECTURES=120`. On Ampere (A6000) use `86`. |
| |
| ### 1) Deps + build llama.cpp |
| ```bash |
| export DEBIAN_FRONTEND=noninteractive |
| apt-get update -qq |
| apt-get install -y -qq build-essential cmake git python3 python3-pip libcurl4-openssl-dev ccache |
| pip install -q -U "huggingface_hub[cli]" hf_transfer |
| export HF_HUB_ENABLE_HF_TRANSFER=1 # fast download |
| # the HF CLI in hf_hub 2.x is `hf`, NOT `huggingface-cli` |
| |
| cd /root && git clone --depth 1 https://github.com/ggml-org/llama.cpp && cd llama.cpp |
| # Blackwell PRO 6000 -> 120 ; Ampere A6000 -> 86 |
| cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_CURL=ON -DGGML_CCACHE=ON |
| cmake --build build --config Release -j"$(nproc)" --target llama-server |
| ``` |
| |
| ### 2) Download the model |
| ```bash |
| # Config A — Q8 + MTP |
| hf download nesilabs/Nesi2 --include "Nesi2-Q8_0.gguf" "Nesi2-mtp-Q8_0.gguf" --local-dir /root/models/nesi2 |
| # Config B — Q6_K |
| hf download nesilabs/Nesi2 --include "Nesi2-Q6_K*.gguf" --local-dir /root/models/nesi2 |
| ``` |
| |
| ### 3) Launch `llama-server` in tmux |
| Never start it with `setsid`/`nohup` over SSH (they die when the pipe closes). |
| Use **tmux** so it survives the SSH session. |
| |
| #### Config A — Q8_0 + MTP, 5 users @ 240k (production, RTX PRO 6000 WS) |
| `--parallel N` splits `--ctx-size` across slots → `n_ctx_slot = ctx_size / parallel`. |
| For **5 slots of 240k each** you need `--ctx-size 1228800` (= 5 × 245760), a |
| **quantized KV cache** (`q8_0`), and the MTP draft: |
| ```bash |
| tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ |
| --model /root/models/nesi2/Nesi2-Q8_0.gguf --alias nesi2 \ |
| --n-gpu-layers 999 -fa on \ |
| --cache-type-k q8_0 --cache-type-v q8_0 \ |
| --spec-type draft-mtp --model-draft /root/models/nesi2/Nesi2-mtp-Q8_0.gguf --spec-draft-n-max 3 \ |
| --ctx-size 1228800 --parallel 5 \ |
| --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \ |
| > /root/llama.log 2>&1" |
| ``` |
| VRAM ≈ 84 / 96 GB. Drop `--spec-type/--model-draft` if your build lacks MTP. |
| |
| #### Config B — Q6_K, single power user @ 240k |
| ```bash |
| tmux new-session -d -s nesi2 "/root/llama.cpp/build/bin/llama-server \ |
| --model /root/models/nesi2/Nesi2-Q6_K.gguf --alias nesi2 \ |
| --ctx-size 245760 --parallel 1 --n-gpu-layers 999 -fa on \ |
| --host 0.0.0.0 --port 8080 --api-key <YOUR_API_KEY> \ |
| > /root/llama.log 2>&1" |
| ``` |
| For **3 users @ 240k each** on a 96 GB card, use `--ctx-size 737280 --parallel 3` |
| plus `--cache-type-k q8_0 --cache-type-v q8_0`. If it OOMs, drop to |
| `q4_0` KV, lower per-user context (e.g. `--ctx-size 393216 --parallel 3` = 128k |
| each), or move to a 2× GPU box. Under `--parallel 3` per-stream speed drops |
| (~55 → ~20–30 tok/s) but aggregate throughput goes up. |
| |
| Wait for load (VRAM up + `server is listening`): |
| ```bash |
| grep -m1 "server is listening" <(tail -f /root/llama.log) |
| ``` |
| |
| ### 4) Point the gateway (LiteLLM) at the box |
| The box's **public IP and 8080 port change on every (re)start**, so this is the |
| only step you repeat. On the LiteLLM host: |
| ```bash |
| # .env |
| NESI2_API_BASE=http://<BOX_PUBLIC_IP>:<BOX_8080_HOSTPORT>/v1 |
| NESI2_API_KEY=<YOUR_API_KEY> |
| # then |
| docker compose up -d litellm |
| ``` |
| |
| ### 5) Verify end-to-end |
| ```bash |
| curl -s -X POST https://<your-gateway>/v1/chat/completions \ |
| -H "Authorization: Bearer <client_key>" -H "Content-Type: application/json" \ |
| -d '{"model":"nesi2","messages":[{"role":"user","content":"di OK"}],"max_tokens":5}' |
| ``` |
| |
| --- |
| |
| ## Lessons learned & recommendations |
| |
| - **`--parallel 1` is mandatory to actually get the full 240k in one slot.** The |
| llama.cpp default (`--parallel 4`) silently gives each slot `ctx/4`. |
| - **`-fa on` (flash-attention)** — keep it on; large context is unusable without it. |
| - **tmux, not setsid/nohup.** Over SSH the detached process dies and you get an |
| empty log; tmux survives. |
| - **Agent discipline > raw capability.** As an agent Nesi2 shines because of |
| *tool-calling discipline*: it acts (one concrete tool call per turn) instead of |
| narrating. Give it a lean system prompt that says: act don't narrate; write your |
| own Python for anything past a simple HTTP request (WebSocket, sockets, rotating |
| CSRF); get code-exec first and confirm with `id`; and **keep durable state on |
| disk** (a `NOTES.md` + reusable scripts) so a context compaction doesn't make it |
| re-derive what it already solved. |
| - **Isolate the agent's filesystem for honest evals.** A harness whose shell can |
| read your machine will "solve" a CTF by reading the solution/writeups off disk |
| instead of attacking live. Deny reads outside the target (regex guard in the tool |
| layer, or run the agent under `sandbox-exec`/a container). Network to the target |
| stays open. |
| - **KV quantization scales users cheaply.** `--cache-type-k/v q8_0` roughly doubles |
| how many 240k slots fit, with no quality drift we could measure; it's what makes |
| the multi-user profiles fit a single 96 GB card. |
| - **Max-Q vs WS.** The power-capped Max-Q PRO 6000 is ~2.5× slower than the full WS |
| for ~0.25 $/h less. For interactive/agent use pay for the WS; for cheap batch, |
| Max-Q is fine. |
| - **Speculative decode (MTP).** The `*-mtp*.gguf` draft head is included; under |
| concurrency it did **not** help throughput in our tests, so we run without it for |
| multi-user serving and keep it available for single-stream latency. |
| |
| ## ⚠️ Legal Notice / Disclaimer |
| |
| **Nesi2 is intended EXCLUSIVELY for security professionals, for lawful and |
| authorized activities.** |
| |
| - **Authorization required.** Use it only against systems, networks, or |
| applications for which you have **explicit written permission** (a contracted |
| penetration test, an in-scope bug bounty program, or your own/lab environments). |
| Using it against systems without authorization is **illegal** and strictly |
| prohibited. |
| - **User responsibility.** You are the **sole responsible party** for how you use |
| this model and for complying with all applicable laws in your jurisdiction |
| (including computer-misuse, data-protection/GDPR, and intellectual-property laws). |
| - **No warranty.** The model is provided "as is", **without warranty** of accuracy, |
| fitness, or results. It may produce incorrect information — always verify before acting. |
| - **Limitation of liability.** Nesilabs is **not liable** for any damages, losses, |
| or legal consequences arising from the use or misuse of this model, including any |
| unauthorized or unlawful use. |
| - **Prohibited use.** Using Nesi2 for illegal activities, unauthorized access, |
| extortion, harm to third parties, or any purpose that violates the law or these |
| terms is prohibited. |
| |
| By using Nesi2 you accept this notice and the Nesilabs Terms of Service and |
| Acceptable Use Policy. |
| |
| ## License |
| Released under **Apache 2.0**. Packaged and optimized by Nesilabs. |
| |
