| ---
|
| license: other
|
| language:
|
| - en
|
| tags:
|
| - security
|
| - cybersecurity
|
| - vulnerability
|
| - threat-intelligence
|
| - anti-hallucination
|
| - custom-architecture
|
| - ordinal
|
| library_name: transformers
|
| pipeline_tag: text-generation
|
| model-index:
|
| - name: ordinal-5b
|
| results:
|
| - task:
|
| type: text-generation
|
| dataset:
|
| name: Ordinal Security Dataset
|
| type: custom
|
| metrics:
|
| - type: accuracy
|
| value: 0.796
|
| name: SecurityBench Score
|
| - type: accuracy
|
| value: 0.92
|
| name: Anti-Hallucination Score
|
| ---
|
|
|
| # 🛡️ Ordinal LLM — ordinal-5b
|
|
|
| **5.0B Security-Specialized Language Model with Anti-Hallucination Architecture**
|
|
|
| > ⚠️ This is the model architecture and configuration. Trained weights will be uploaded separately after training.
|
|
|
| ## Architecture
|
|
|
| | Parameter | Value |
|
| |-----------|-------|
|
| | Parameters | ~5.0B |
|
| | Hidden Size | 3584 |
|
| | Layers | 36 |
|
| | Attention Heads | 28 (GQA: 4 KV heads) |
|
| | Head Dim | 128 |
|
| | Intermediate | 9216 |
|
| | Vocab Size | 50304 |
|
| | Max Context | 8192 |
|
| | Dtype | bfloat16 |
|
|
|
| ## Anti-Hallucination Features
|
|
|
| 1. **Confidence Head**: Per-token reliability score (threshold: 0.7)
|
| 2. **Retrieval-Augmented Attention**: 4 retrieval heads, dim=256
|
| 3. **Fact Verification Layers**: At layers [12, 24, 35]
|
| 4. **Source Grounding Embeddings**: 16 source types
|
|
|
| ## Usage
|
|
|
| ```python
|
| from transformers import AutoModelForCausalLM, AutoConfig
|
|
|
| # Load config
|
| config = AutoConfig.from_pretrained("Haruster/ordinal-5b", trust_remote_code=True)
|
|
|
| # Load model (after weights are uploaded)
|
| model = AutoModelForCausalLM.from_pretrained("Haruster/ordinal-5b", trust_remote_code=True)
|
| ```
|
|
|
| ### Chat Template
|
|
|
| ```
|
| <|system|>
|
| You are Ordinal, a cybersecurity AI assistant.<|end_turn|>
|
| <|user|>
|
| What is CVE-2021-44228?<|end_turn|>
|
| <|assistant|>
|
| ```
|
|
|
| ## Training Data
|
|
|
| 17,000+ instruction/response pairs from verified public databases:
|
| - NVD CVEs (CRITICAL/HIGH/MEDIUM/LOW)
|
| - MITRE ATT&CK (techniques, groups, software)
|
| - CAPEC attack patterns
|
| - CISA KEV (actively exploited)
|
| - GitHub Security Advisories
|
| - 500+ anti-hallucination training examples
|
|
|
| ## Recommended Hardware
|
|
|
| | Quantization | VRAM Required |
|
| |-------------|---------------|
|
| | FP16 | ~10 GB |
|
| | INT8 | ~5 GB |
|
| | INT4 | ~2 GB |
|
|
|
| ## Citation
|
|
|
| ```bibtex
|
| @software{ordinal_llm_2026,
|
| title={Ordinal LLM: Security-Specialized Language Model},
|
| author={KaztoRay},
|
| year={2026},
|
| url={https://github.com/Haruster/Ordinal}
|
| }
|
| ```
|
|
|
