---
license: mit
language:
- en
- fr
- code
tags:
- non-transformer
- cognitive-routing
- hierarchical-memory
- character-level
- aicl
- text-generation
- custom-architecture
pipeline_tag: text-generation
library_name: pytorch
---

# CogNet-40M

A 39.7M parameter non-transformer language model with O(n) cognitive routing and hierarchical memory.

## Architecture

| Component | Detail |
|-----------|--------|
| Architecture | Non-transformer (Cognitive Routing) |
| Parameters | 39,718,536 (~40M) |
| Hidden Dim | 512 |
| Blocks | 6 cognitive blocks |
| Channels | 6 routing channels x 128 dim |
| FF Dim | 1024 |
| Max Seq Len | 256 |
| Tokenizer | Character-level (136 vocab) |

## Hierarchical Memory

- Working Memory (32 slots): Active processing
- Episodic Memory (64 slots): Short-term recall
- Semantic Memory (128 slots): Long-term knowledge

## Training

| Metric | Value |
|--------|-------|
| Steps | 50,000 |
| Batch Size | 64 |
| LR | 3e-4 (cosine) |
| Precision | FP16 AMP |
| GPU | RTX 5060 Ti 16GB |
| Final Loss | ~0.005 |
| Final PPL | ~1.01 |

## Quick Start

```python
from inference import CogNetInference
ai = CogNetInference("cognet_best.pt", "tokenizer_v3.json")
print(ai.generate("Once upon a time"))
```

## AICL Integration

CogNet powers AICL (Architecture Compilation Language) as its native AI engine for code generation, diagnosis, and repair.

## Files

| File | Size | Description |
|------|------|-------------|
| cognet_best.pt | 152MB | FP32 checkpoint |
| cognet_fp16.pt | 77MB | FP16 checkpoint |
| tokenizer_v3.json | - | Char tokenizer (136 vocab) |
| config.json | - | Model config |
| cognet_model.py | - | Architecture source |
| inference.py | - | Inference script |

## Roadmap

- [x] CogNet-40M (39.7M)
- [x] HuggingFace integration
- [x] AICL native engine
- [ ] CogNet-1B (1B params)
- [ ] ONNX export

MIT License. Built with PyTorch on RTX 5060 Ti via QuickPod.