---
license: other
language:
- en
tags:
- security
- cybersecurity
- vulnerability
- threat-intelligence
- anti-hallucination
- custom-architecture
- ordinal
library_name: transformers
pipeline_tag: text-generation
model-index:
- name: ordinal-5b
  results:
  - task:
      type: text-generation
    dataset:
      name: Ordinal Security Dataset
      type: custom
    metrics:
    - type: accuracy
      value: 0.796
      name: SecurityBench Score
    - type: accuracy
      value: 0.92
      name: Anti-Hallucination Score
---

# 🛡️ Ordinal LLM — ordinal-5b

**5.0B Security-Specialized Language Model with Anti-Hallucination Architecture**

> ⚠️ This is the model architecture and configuration. Trained weights will be uploaded separately after training.

## Architecture

| Parameter | Value |
|-----------|-------|
| Parameters | ~5.0B |
| Hidden Size | 3584 |
| Layers | 36 |
| Attention Heads | 28 (GQA: 4 KV heads) |
| Head Dim | 128 |
| Intermediate | 9216 |
| Vocab Size | 50304 |
| Max Context | 8192 |
| Dtype | bfloat16 |

## Anti-Hallucination Features

1. **Confidence Head**: Per-token reliability score (threshold: 0.7)
2. **Retrieval-Augmented Attention**: 4 retrieval heads, dim=256
3. **Fact Verification Layers**: At layers [12, 24, 35]
4. **Source Grounding Embeddings**: 16 source types

## Usage

```python
from transformers import AutoModelForCausalLM, AutoConfig

# Load config
config = AutoConfig.from_pretrained("Haruster/ordinal-5b", trust_remote_code=True)

# Load model (after weights are uploaded)
model = AutoModelForCausalLM.from_pretrained("Haruster/ordinal-5b", trust_remote_code=True)
```

### Chat Template

```
<|system|>
You are Ordinal, a cybersecurity AI assistant.<|end_turn|>
<|user|>
What is CVE-2021-44228?<|end_turn|>
<|assistant|>
```

## Training Data

17,000+ instruction/response pairs from verified public databases:
- NVD CVEs (CRITICAL/HIGH/MEDIUM/LOW)
- MITRE ATT&CK (techniques, groups, software)
- CAPEC attack patterns
- CISA KEV (actively exploited)
- GitHub Security Advisories
- 500+ anti-hallucination training examples

## Recommended Hardware

| Quantization | VRAM Required |
|-------------|---------------|
| FP16 | ~10 GB |
| INT8 | ~5 GB |
| INT4 | ~2 GB |

## Citation

```bibtex
@software{ordinal_llm_2026,
  title={Ordinal LLM: Security-Specialized Language Model},
  author={KaztoRay},
  year={2026},
  url={https://github.com/Haruster/Ordinal}
}
```