| --- |
| language: en |
| license: mit |
| tags: |
| - hackernet |
| - cybersecurity |
| - penetration-testing |
| - pytorch |
| - moe |
| - bitnet |
| pipeline_tag: text-generation |
| --- |
| |
| # HackerNet-Beta v1 |
|
|
| > ⚠️ **Research only.** This model is trained for cybersecurity research and authorized penetration testing. |
|
|
| ## Architecture |
|
|
| | Parameter | Value | |
| |-----------|-------| |
| | Hidden Dim | 768 | |
| | Layers | 16 | |
| | Attention Heads | 12 | |
| | KV Heads (GQA) | 4 | |
| | MoE Experts | 4 (top-2) | |
| | Vocab Size | 32,768 | |
| | Max Seq Len | 4,096 | |
| | BitNet | ✅ Enabled | |
|
|
| ## Training |
|
|
| | Setting | Value | |
| |---------|-------| |
| | Final Avg Loss | `0.0000` | |
| | Learning Rate | `0.0001` | |
| | Gradient Accumulation | 4 steps | |
| | Max Length | 1024 tokens | |
| | Precision | float16 + AMP | |
| | Epochs | 4 | |
|
|
| ## Datasets Trained On |
|
|
| - `OpenAssistant/oasst_top1_2023-08-25` |
| - `ai4bharat/indic-instruct-data-v0.1` |
| - `HydraLM/hindi_multiturn_conversations` |
| - `uonlp/CulturaX` |
| - `HydraLM/hinglish_multiturn_conversations` |
| - `festvox/cmu_indic` |
| - `HuggingFaceH4/ultrachat_200k` |
| - `teknium/OpenHermes-2.5` |
|
|
| ## Versioning |
|
|
| | File | Description | |
| |------|-------------| |
| | `hackernet_v1.pt` | PyTorch state dict | |
| | `hackernet_v1.safetensors` | Safetensors format | |
| | `versions/v1/config.json` | Architecture config | |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from core.model import HackerNetModel |
| from core.config import get_optimal_config |
| |
| config = get_optimal_config() |
| model = HackerNetModel(config) |
| model.load_state_dict(torch.load("hackernet_v1.pt", map_location="cpu")) |
| model.eval() |
| ``` |
|
|
| --- |
| *Auto-generated by HackerNet training pipeline on Lightning AI* |
|
|
