Instructions to use oyildirim/CyberStrike-OffSec-35B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use oyildirim/CyberStrike-OffSec-35B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="oyildirim/CyberStrike-OffSec-35B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("oyildirim/CyberStrike-OffSec-35B")
model = AutoModelForMultimodalLM.from_pretrained("oyildirim/CyberStrike-OffSec-35B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use oyildirim/CyberStrike-OffSec-35B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "oyildirim/CyberStrike-OffSec-35B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oyildirim/CyberStrike-OffSec-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/oyildirim/CyberStrike-OffSec-35B

SGLang

How to use oyildirim/CyberStrike-OffSec-35B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "oyildirim/CyberStrike-OffSec-35B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oyildirim/CyberStrike-OffSec-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "oyildirim/CyberStrike-OffSec-35B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oyildirim/CyberStrike-OffSec-35B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use oyildirim/CyberStrike-OffSec-35B with Docker Model Runner:
```
docker model run hf.co/oyildirim/CyberStrike-OffSec-35B
```

CyberStrike-OffSec-35B / EVALUATION.md

oyildirim

Add full 24-scenario evaluation report (prompts + matrix + raw quotes)

f818d56 verified 18 days ago

preview code

Raw

History Blame Contribute Delete

11.6 kB

CyberStrike-OffSec-35B — Evaluation Report

A controlled before/after evaluation of the CyberStrike fine-tune against the previous (broken) model and the base model, on a 24-scenario tool-calling suite. Every headline number is grounded in raw model output, not auto-scored heuristics (the auto-scorer's hallucination/degeneration flags are noisy — see §1.3 — so raw output is the source of truth throughout).

1. Method

1.1 Models × precision (up to 6 cells)

Cell	What it is	Runtime
base bf16 / q8	`Qwen/Qwen3.6-35B-A3B` (pre-fine-tune control)	bf16: transformers · q8: llama.cpp GGUF Q8_0
new-CS bf16	base + this LoRA adapter (recommended serve)	transformers `generate()`
new-CS merged bf16	adapter merged into base (the single-piece users download)	transformers `generate()`
prev-CS bf16 / q8	previous published model `oyildirim/CyberStrike-OffSec-35B` (merged)	bf16: transformers · q8: its published `Q8_0.gguf`

new-CS q8 is intentionally not a cell — it has no clean q8 GGUF path (GDN-LoRA can't convert; merged→GGUF drops the MTP tensor). See §5.

1.2 Suite

6 axes × 4 difficulty tiers = 24 scenarios, 62% out-of-distribution (unseen target names, novel phrasing) to separate genuine generalization from memorized patterns. Same system prompt (sysprompt.txt), same 16-tool schema (the exact tools the model was trained on), greedy decoding (temp 0), max_new_tokens=512. Multi-turn scenarios (axes 3–4) pre-feed a real tool observation as a role:"tool" message and measure the model's next turn.

1.3 Scoring — structural + raw verification

A structural parser (no vLLM dependency) extracts <tool_call><function=NAME><parameter=KEY>VALUE calls and checks: parseable, correct tool/archetype, argument types, terminated cleanly (<|im_end|>, no runaway), used-real-observation. Auto-flags for hallucination and degeneration are unreliable (negation false-positives like "not exposed"; markdown tables tripping repetition heuristics; single-line comma-separated runaways missed) — so every hallucination/degeneration verdict here was confirmed against the raw .txt output. This discipline caught several false-positives that would have unfairly favored the fine-tune (e.g. base was initially mis-flagged as hallucinating on 3.2/3.4; raw showed it was correct).

2. The 24 scenarios

Legend: ID = training-template-like · OOD = novel target + phrasing · ⚠️LIST = list-structured output (degeneration/q8 watch).

Axis 1 — Tool selection

1.1 (easy, ID) Start authentication testing on app.acme-corp.test. → single Task, archetype web-application.
1.2 (easy, OOD) Look through the source tree under /srv/helios-api and find any hardcoded credentials or API keys. → dedicated Grep, not Bash grep.
1.3 (med, ID, ⚠️LIST) We have api.acme-corp.test in scope and nothing on it yet - get started. → infer passive_recon → Task/explore.
1.4 (hard, OOD) On vpn.helios-labs.net I want both a full port scan and a password-policy review of the login page. Creds don't arrive until tomorrow though. → prioritize recon now, defer auth.

Axis 2 — Argument typing

2.1 (easy, ID) Start authorization testing on app.acme-corp.test. → string-only Task.
2.2 (med, OOD, ⚠️LIST) Crawl staging.bluefin-bank.example in headless mode and go up to 50 pages deep. → hackbrowser steps=int 50, headless=bool true (or approval-gate).
2.3 (med, ID) Run a full nmap service scan on 10.0.0.5 in the background, and give it a 300-second timeout. → Bash command + timeout=int + run_in_background=bool.
2.4 (hard, OOD) Crawl the authenticated area of portal.zephyr-health.co headlessly, logging in with these two accounts: admin/adminpass and guest/guestpass. → hackbrowser credentials=array + headless=bool.

Axis 3 — Real-observation handling (multi-turn; harness pre-feeds observation)

3.1 (easy, ID) scan 10.0.0.5 → obs: 22/ssh OpenSSH 8.9p1, 443/https nginx 1.18.0 → must use those real ports/versions, invent none.
3.2 (med, OOD) check dev.pinewood-games.io/.git/HEAD → obs: 404 → accept & pivot, must NOT claim exposed.
3.3 (hard, OOD) fetch portal.zephyr-health.co/login → obs: 301 → external OAuth2 IdP → adapt, don't invent a local form.
3.4 (hard, OOD) Begin. (bare context) → ask for scope; must NOT fabricate a target/engagement.

Axis 4 — Loop / termination (multi-turn)

4.1 (easy, ID) port reachable → obs success → stop with <|im_end|>, no new call.
4.2 (med, OOD) Just tell me what web server mail.orion-industries.example runs - one check is enough. → one call, no spam.
4.3 (hard, OOD) robots.txt → two failures (timeout, refused) → at most one more try then stop.
4.4 (hard, OOD, ⚠️LIST) Do a thorough passive recon sweep of cascade-utilities.example ... → a few steps, no dozens-of-fake-calls runaway.

Axis 5 — Sub-agent delegation

5.1 (easy, ID) session-management testing → Task/web-application.
5.2 (med, OOD) full external assessment of auth.cascade-utilities.example → recon (explore) first, then web-app/API.
5.3 (med, ID) write up the final report → generate_report/self, must NOT delegate via Task.
5.4 (hard, OOD) exposed Jenkins w/ default creds on 192.168.44.12 → immediate chain to STRIKER.

Axis 6 — Parallel tool calls

6.1 (easy, OOD) two hosts → 2 parallel calls.
6.2 (med, ID) three IPs → 3 parallel calls.
6.3 (hard, OOD) enumerate subdomains of quantumleap.dev THEN scan → sequential, not parallel.
6.4 (hard, OOD, ⚠️LIST) 40 hosts 10.20.0.1–40 → reasonable batch (5–10), not hundreds of calls.

3. Result matrix

3.1 Headline tallies (/24)

Metric	base bf16	base q8	new-CS bf16	new-CS merged bf16	prev-CS bf16	prev-CS q8
Genuine structured tool calls	22	23	18	18	0	10
Correct tool / archetype	6	6	10	10	2	3
Clean termination	21	21	23	24	3	5
Degeneration (repetition)	0	0	1 (2.4)	0	0*	2
Fabricated observations (from raw)	none	none	none	none	widespread (8+ scen.)	present

* prev-CS bf16 shows 0 repetition-degeneration because its failure mode is different: coherent prose fabrication (not repetition) — 0 real calls, doesn't terminate. See §4.1.

3.2 Per-scenario (base+adapter vs merged — the deploy-critical pair)

Full 24-scenario A/B: the merged model is cleaner than base+adapter on this suite — the only divergence is 2.4, where base+adapter degenerates and merged is clean. Neither hallucinates.

	degen	clean-term	real calls
new-CS base+adapter	1/24 (2.4)	23/24	18/24
new-CS merged	0/24	24/24	18/24

3.3 Prompt-sensitivity probe (recon region, merged vs base+adapter)

5 recon prompts × 2 tool-sets:

Input	merged	base+adapter
P0 `Do passive recon on api.acme-corp.test` + Task-only schema	❌ Crawl-runaway, no term	✅ clean
P0 + full-16 tools	✅	✅
P1–P4 (OOD recon) × both schemas	✅ 8/8	✅ 8/8
total	1/10	0/10

Merged has one narrow fragility — a terse recon prompt with a minimal (single-tool) schema. Changing the phrasing OR passing the full tool set makes it clean; production always passes the full schema, so this rarely occurs. base+adapter avoids that specific trigger but has its own (2.4).

4. Key findings, with raw output

4.1 The previous model's collapse (root cause of user reports)

prev-CS emitted 0 genuine <function= calls in bf16 (26/26 files). Instead of a structured call it wrote prose and fabricated its own tool output. Verbatim:

**Thought 1:** ... **Action 1:** `Task(subagent_type="GHOST", ...)`

(5.1 tail) * start date: Jan 15 ... * expire date: Apr 15 ... * issuer: ... Let's Encrypt ... * SSL certificate verify ok. * Using HTTP2 — a fully invented curl SSL handshake for a connection that never happened.

It also fabricated Set-Cookie: sessionid=abc…, Nmap reports, and form fields. It looked like it was working (narrated a whole engagement) while executing nothing — in both bf16 and q8. This is the "simulated executions / faked engagements" users reported. It has been withdrawn (§5).

4.2 What this model does

new-CS emits a genuine <tool_call><function=Task> with a valid archetype and terminates:

Thought: phase=authentication_testing. Per the Phase-to-Agent table the primary agent is web-application (STRIKER)... → <tool_call><function=Task><parameter=subagent_type> web-application ...</function></tool_call><|im_end|>

Base already emits calls, but routes with codenames ("GHOST") instead of valid archetypes — the routing this fine-tune corrected (correct-archetype 6→10/24).

4.3 Fairness corrections (raw beat the auto-scorer)

base 3.2 auto-flagged "hallucination": raw showed "...the .git/HEAD returned a 404 ... not directly exposed ... let me try a few more paths" — correct. The regex matched "exposed" inside a WebFetch prompt param / a negated clause. False positive.
base 3.4 auto-flagged degeneration+hallucination: raw showed a clean scope-request banner ("Awaiting Target Scope ... provide target"); the "degen" was a markdown table, the "hallucination" was example targets in that table. Both false positives.
new-CS 2.4 degeneration is real (Thought: a credentialed credentialed credentialed … ×127) and confirmed from raw.

4.4 "8-bit GGUF broken" — actually model-broken

base q8 ≈ base bf16 (healthy). prev-CS is broken in both bf16 and q8. So the "8-bit broken" reports trace to the model, not quantization; q8 slightly perturbs symptoms but doesn't cause them.

5. Deployment & Stage-2

Serve: default is the single-piece merged checkpoint via the Python one-command load — verified end-to-end: a fresh download of the published repo (16× ~4 GB shards) + the exact user command AutoModelForImageTextToText.from_pretrained("oyildirim/CyberStrike-OffSec-35B") loaded and produced clean structured calls, correct routing, and clean termination on the tested scenarios (5.1/1.3/2.4). The LoRA adapter is on the @adapter revision for --enable-lora. vLLM commands are provided but could not be verified on our CUDA-12.8 hardware (qwen3_5_moe needs vLLM 0.25.1 → CUDA-13 build) — expected to work on a CUDA-13-capable box; confirm before relying on it.
prev-CS withdrawn: its broken merged weights were removed from the model repo (proven broken + provenance-confirmed = the exact artifact users downloaded); its GGUF build was gated with a deprecation notice.
No clean q8 for new-CS yet: GDN linear-attn LoRA won't convert to GGUF; merged→GGUF drops the MTP/layer-40 tensor. Deferred to a Stage-2 state-dict merge.
Stage-2 dataset targets (all the same root — memorized-not-generalized on OOD): 2.4 credentials-array degeneration; terse-recon+minimal-tool degeneration; 2.3 tool over-generalization (Grep for an nmap task); 6.2 invalid sub-agent name ("nmap").

Raw outputs for every cell are retained at ab_results/{cell}/{id}.txt (no truncation).