README.md

---
library_name: transformers
license: apache-2.0
language:
- en
tags:
- qwen
- qwen3.5
- kali
- terminal
- tool-calling
- function-calling
- cybersecurity
pipeline_tag: text-generation
datasets:
- iselabvn/Kali-terminal-executor-v2
model-index:
- name: Kali-Terminus-v3
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Kali-terminal-executor-v2
      type: iselabvn/Kali-terminal-executor-v2
    metrics:
    - name: Strict Pass Rate
      type: strict_pass_rate
      value: 96.15
      verified: false
    - name: Semantic Command Accuracy
      type: semantic_accuracy
      value: 96.15
      verified: false
base_model:
- Qwen/Qwen3.5-0.8B-Base
---

# Kali-Terminus-v3

**Kali-Terminus-v3** is a fine-tuned version of **Qwen/Qwen3.5-0.8B-Base** on the **iselabvn/Kali-terminal-executor-v2** dataset. It is designed to generate accurate Kali Linux terminal commands from natural language instructions, with built-in reasoning via Qwen3.5's native ` thinking`/` response` tokens.

## Model Details

| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3.5-0.8B-Base |
| Architecture | Hybrid: Gated DeltaNet (linear attention) + Gated Attention (full attention) |
| Parameters | 0.8B |
| Context Length | 262,144 |
| Fine-Tuning | LoRA (r=16, alpha=32) |
| Training Data | iselabvn/Kali-terminal-executor-v2 (2,418 samples) |
| Format | `<\|im_start\|>` / `<\|im_end\|>` with native `<\|tool_call\|>`/`<\|tool_response\|>` XML tool calls |
| Precision | bfloat16 (merged) |

## Performance

Evaluated across 5 scenarios (26 turns) covering network recon, system enumeration, web recon, DNS recon, and self-correction:

| Metric | Score |
|---|---|
| **Strict Pass Rate** | **96.15%** |
| Semantic Command Accuracy | 96.15% |
| Tool Call Parse Rate | 100.0% |
| Exactly One Tool Call | 100.0% |

### Comparison with Kali-Terminus-v2

| Model | Strict Pass Rate | Improvement |
|---|---|---|
| Kali-Terminus-v2 (LFM2.5-350M) | 92.31% | Baseline |
| **Kali-Terminus-v3** | **96.15%** | **+3.84 pp** |

## Tool Call Format

This model uses Qwen3.5's native tool call format with `<tool_call>` XML tags:

```xml
<tool_call>
<function=exec>
<parameter=command>
nmap -p- 127.0.0.1
</parameter>
</function>
</tool_call>
```

The model also outputs reasoning before tool calls using Qwen3.5's native ` thinking`/` response` tokens:

```
<|im_start|>assistant
 thinking
The user wants to scan all 65535 ports. I'll use nmap with -p-.
 response

<tool_call>
<function=exec>
<parameter=command>
nmap -p- 127.0.0.1
</parameter>
</function>
</tool_call><|im_end|>
```

## Usage (Transformers)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "iselabvn/Kali-Terminus-v3"
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

exec_tool = [{
    "type": "function",
    "function": {
        "name": "exec",
        "description": "Execute a shell command in the Kali Linux terminal.",
        "parameters": {
            "type": "object",
            "properties": {"command": {"type": "string"}},
            "required": ["command"],
        },
    },
}]

messages = [{"role": "user", "content": "Scan all open ports on 127.0.0.1"}]
inputs = tokenizer.apply_chat_template(
    messages, tools=exec_tool, add_generation_prompt=True,
    enable_thinking=True, tokenize=True, return_dict=True, return_tensors="pt",
)
inputs = {k: v.to(model.device) for k, v in inputs.items()}

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, pad_token_id=tokenizer.eos_token_id)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=False)
print(response)
# Expected: <|im_start|>assistant
#  thinking
# I need to scan all ports...
#  response
# 
# <tool_call>
# <function=exec>
# <parameter=command>
# nmap -p- 127.0.0.1
# </parameter>
# </function>
# </tool_call><|im_end|>
```

## Usage (llama.cpp GGUF)

The GGUF version is available at `Kali-Terminus-v3-GGUF/kali-terminus-v3-bf16.gguf`. Run with llama-server:

```bash
llama-server -m kali-terminus-v3-bf16.gguf --host 127.0.0.1 --port 8080 -ngl 99
```

## Training Details

### Dataset
- **Source**: iselabvn/Kali-terminal-executor-v2
- **Splits**: 2,084 train / 232 validation / 102 test
- **Features**: Each sample includes `reasoning_content` for natural language reasoning before tool calls

### Fine-Tuning Parameters
| Parameter | Value |
|---|---|
| LoRA r | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `in_proj`, `out_proj`, `gate_proj`, `up_proj`, `down_proj` |
| Batch size | 4 (effective 16) |
| Learning rate | 2e-4 (cosine) |
| Epochs | 3 |
| Max seq length | 1024 |
| Precision | bfloat16 |

## Limitations

- The model is specialized for Kali Linux terminal command generation and may not generalize well to other domains
- Performance may vary with complex multi-step scenarios
- The model uses Qwen3.5's native ` thinking`/` response` reasoning tokens; for optimal results, use `enable_thinking=True` in the generation config

## Files

| File | Size | Description |
|---|---|---|
| `model.safetensors` | 1.4 GB | Merged model weights (bfloat16) |
| `config.json` | — | Model configuration (text-only, MTP disabled) |
| `tokenizer_config.json` | — | Tokenizer configuration with full chat template |
| `chat_template.jinja` | — | Qwen3.5 native chat template |

## Citation

If you use this model, please cite:

```bibtex
@misc{kali-terminus-v3,
  title  = {Kali-Terminus-v3: Fine-tuned Qwen3.5 for Kali Linux Terminal Command Generation},
  author = {Kali-Terminus Team},
  year   = {2026},
}
```