README.md

---
license: apache-2.0
library_name: gguf
base_model: ZySec-AI/SecurityLLM
pipeline_tag: text-generation
model_creator: Orionfold LLC
language:
  - en
tags:
  - gguf
  - spark-tested
  - orionfold
  - "base_model:ZySec-AI/SecurityLLM"
---
# SecurityLLM GGUF

`GGUF` quantizations of `ZySec-AI/SecurityLLM`, verified end-to-end on the NVIDIA DGX Spark (GB10, 128 GB unified memory).

## Notebooks

Two runnable notebooks ship with this model — open either on a free cloud GPU:

| Notebook | What it does | Open |
|---|---|---|
| **Builder** | Reproduce this model's build and DGX Spark benchmarks end-to-end with `fieldkit`. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/cyber/builder.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/cyber/builder.ipynb) |
| **User** | Load the published model and call it from your own app in a few lines. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/manavsehgal/ai-field-notes/blob/main/notebooks/cyber/user.ipynb) [![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/manavsehgal/ai-field-notes/blob/main/notebooks/cyber/user.ipynb) |

## Spark-tested

Every Orionfold quant ships with a measurement quad on the NVIDIA DGX Spark (GB10, 128 GB unified memory): perplexity, sustained `tok/s`, thermal envelope, and **CyberMetric (n=50, mcq_letter)** accuracy. The numbers below are the actual run, not a wishlist.

| Variant | Size | Perplexity (wikitext-2) | tok/s on Spark | CyberMetric (n=50, mcq_letter) |
|---|---|---|---|---|
| Q4_K_M | 4.1 GB | 7.400 | 47.7 | 40.0% |
| Q5_K_M | 4.8 GB | 7.314 | 40.0 | 38.0% |
| Q6_K | 5.5 GB | 7.313 | 35.0 | 36.0% |
| Q8_0 | 7.2 GB | 7.307 | 30.3 | 36.0% |
| F16 | 13.5 GB | 7.301 | 17.4 | 34.0% |

**Thermal envelope:** sustained-load minutes before thermal throttle on a single GB10 = **5 min**. Beyond this, expect tok/s degradation; the duty-cycle disclosure is per Orionfold's quant-card standard.

## Variants

| Variant | Recommended use |
|---|---|
| Q4_K_M | Best balance — fits comfortably in Spark unified memory at 70B; default pick. |
| Q5_K_M | Higher quality than Q4_K_M with modest size bump. |
| Q6_K | Near-lossless; recommended if memory headroom allows. |
| Q8_0 | Effectively lossless; reach for this when quality matters more than throughput. |
| F16 | Reference — no quantization. Use only for measurement / baseline. |

## How to run

Pull a variant:

```bash
huggingface-cli download Orionfold/SecurityLLM-GGUF model-Q5_K_M.gguf \
  --local-dir ./models/securityllm
```

Serve it via `llama-server` (OpenAI-compatible API):

```bash
llama-server -m ./models/securityllm/model-Q5_K_M.gguf \
  -c 4096 -ngl 99 -t 8 \
  --host 0.0.0.0 --port 8080
```

Or run in-process via `llama-cpp-python`:

```python
from llama_cpp import Llama
llm = Llama(
    model_path="./models/securityllm/model-Q5_K_M.gguf",
    n_ctx=4096, n_gpu_layers=99, chat_format="zephyr",
)
out = llm.create_chat_completion(
    messages=[
        {"role": "user",
         "content": "What is the primary purpose of a key-derivation function (KDF)?\n\n"
                    "A) Generate public keys\n"
                    "B) Authenticate digital signatures\n"
                    "C) Encrypt data using a password\n"
                    "D) Transform a secret into keys and Initialization Vectors\n\n"
                    "Reply with only the single letter A, B, C, or D."}
    ],
    temperature=0.0,
)
print(out["choices"][0]["message"]["content"])
```

LM Studio and Ollama (via a Modelfile) load the GGUF directly with no additional setup.

## Methods

Full methodology and Spark-side measurement protocol: [Vertical-curator quants on Spark — SecurityLLM-GGUF + CyberMetric mini-eval](https://ainative.business/field-notes/becoming-a-cyber-curator-on-spark/).

## Other Orionfold vertical curators

Same Spark-tested recipe across the curator-on-Spark series:

- **[finance-chat-GGUF](https://huggingface.co/Orionfold/finance-chat-GGUF)** — AdaptLLM finance-chat (Llama-2-7B lineage) for FinanceBench-shaped queries
- **[Saul-7B-Instruct-v1-GGUF](https://huggingface.co/Orionfold/Saul-7B-Instruct-v1-GGUF)** — Equall Saul-7B legal-instruct for LegalBench-shaped queries
- **[II-Medical-8B-GGUF](https://huggingface.co/Orionfold/II-Medical-8B-GGUF)** — Qwen3-8B + DAPO reasoning for MedMCQA-shaped queries

Each card lists its own measurement quad; the headline numbers are recorded as the actual sweep ran, never pre-corrected.

---

Published by **Orionfold LLC** · [orionfold.com](https://orionfold.com) · Methods documented at [ainative.business/field-notes](https://ainative.business/field-notes/).


> Want to know when the next Orionfold vertical curator drops? [Join the launch list at orionfold.com](https://orionfold.com).