---
language: en
license: mit
tags:
  - hackernet
  - cybersecurity
  - penetration-testing
  - pytorch
  - moe
  - bitnet
pipeline_tag: text-generation
---

# HackerNet-Beta v1

> ⚠️ **Research only.** This model is trained for cybersecurity research and authorized penetration testing. (Under Cunstruction not for use this time)

## Architecture

| Parameter | Value |
|-----------|-------|
| Hidden Dim | 768 |
| Layers | 16 |
| Attention Heads | 12 |
| KV Heads (GQA) | 4 |
| MoE Experts | 4 (top-2) |
| Vocab Size | 32,768 |
| Max Seq Len | 4,096 |
| BitNet | ✅ Enabled |

## Training

| Setting | Value |
|---------|-------|
| Final Avg Loss | `0.0000` |
| Learning Rate | `0.0001` |
| Gradient Accumulation | 4 steps |
| Max Length | 1024 tokens |
| Precision | float16 + AMP |
| Epochs | 4 |

## Datasets Trained On

- `OpenAssistant/oasst_top1_2023-08-25`
- `ai4bharat/indic-instruct-data-v0.1`
- `HydraLM/hindi_multiturn_conversations`
- `uonlp/CulturaX`
- `HydraLM/hinglish_multiturn_conversations`
- `festvox/cmu_indic`
- `HuggingFaceH4/ultrachat_200k`
- `teknium/OpenHermes-2.5`

## Versioning

| File | Description |
|------|-------------|
| `hackernet_v1.pt` | PyTorch state dict |
| `hackernet_v1.safetensors` | Safetensors format |
| `versions/v1/config.json` | Architecture config |

## Usage

```python
import torch
from core.model import HackerNetModel
from core.config import get_optimal_config

config = get_optimal_config()
model = HackerNetModel(config)
model.load_state_dict(torch.load("hackernet_v1.pt", map_location="cpu"))
model.eval()
```

---
*Auto-generated by HackerNet training pipeline on Lightning AI*