---
language:
  - en
license: lgpl-3.0
tags:
  - text-generation
  - gpt-oss
  - cybersecurity
  - vllm
base_model: unsloth/gpt-oss-20b-BF16
libraries:
  - transformers
library_name: transformers
model_type: gpt_oss
pipeline_tag: text-generation
datasets:
  - AlicanKiraz0/All-CVE-Records-Training-Dataset
---

# CyberOSS-CVE

# CyberOSS-CVE

Fine-tuned `gpt-oss-20b` on the `AlicanKiraz0/All-CVE-Records-Training-Dataset` using Unsloth with LoRA (rank 32) and merged back to BF16 for compatibility with vLLM, Hugging Face Transformers, and GGUF conversions.

## Training Overview
- **Base model**: `unsloth/gpt-oss-20b-BF16`
- **Dataset**: `AlicanKiraz0/All-CVE-Records-Training-Dataset`
- **Hardware**: single NVIDIA H100 80GB
- **Sequence length**: 2048
- **Batch**: 2 (grad accum 4 → effective 8)
- **Learning rate**: 2e-4, linear warmup 5 steps
- **Steps**: 100 for quick verification run (expand for full epoch)
- **Loss masking**: full conversation (system, user, assistant)

## Files
- `model-0000X-of-00009.safetensors`: merged BF16 shards
- `config.json`: GPT-OSS architecture config
- `tokenizer.json` and template: Harmony/GPT-OSS chat format
- `chat_template.jinja`: OpenAI Harmony-compatible chat template

## Quick Usage (vLLM)
```bash
pip install vllm==0.11.2 transformers==4.57.2
python - <<'PY'
from vllm import LLM, SamplingParams
from transformers.processing_utils import ProcessorMixin
import transformers
transformers.ProcessorMixin = ProcessorMixin

llm = LLM(
    model="Kushalkhemka/CyberOSS-CVE",
    tokenizer="unsloth/gpt-oss-20b-BF16",
    dtype="bfloat16",
)
prompt = "You are a cybersecurity assistant. Summarize CVE-2010-3763."
out = llm.generate([prompt], SamplingParams(max_tokens=128))[0]
print(out.outputs[0].text)
PY
```

## HF Transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Kushalkhemka/CyberOSS-CVE", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("unsloth/gpt-oss-20b-BF16")
```

## License
Matches upstream `unsloth/gpt-oss-20b` (LGPL-3.0). Respect dataset terms when redistributing.