File size: 1,740 Bytes
227da3e 0e15fbd 227da3e fb85920 227da3e a27bcf6 227da3e fb85920 227da3e fb85920 227da3e fb85920 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | ---
license: mit
library_name: transformers
tags:
- causal-lm
- decoder-only
- reasoning
- deep-reasoning
- recurrent-transformer
- adaptive-computation
- chain-of-thought
- adaptive-reasoning
---
# Nano.Deep.Reasoner.11m-HyperMini
An approximately 11,094,003-parameter decoder-only
adaptive recurrent reasoning language model.
## Architecture
- Parameters: 11,094,003
- Context: 1096
- Vocabulary: 16,000
- Hidden size: 240
- Base Transformer blocks: 6
- Attention heads: 8
- Head dimension: 30
- Intermediate size: 1072
- Shared recurrent reasoning block
- Learned latent reasoning memory: 8 tokens
- Adaptive reasoning depth: 2-16
- Gated recurrent memory updates
- Adaptive halting controller
- Verification head
- Revision head
- RoPE
- Tied input/output embeddings
- Padding-aware causal attention
- Explicit padded-state suppression
## Reasoning tokens
<think>
</think>
<thought>
</thought>
<reasoning>
</reasoning>
<answer>
</answer>
## Training
Dataset:
Plans11/Organized_PreTrain_1k_Context
Each session contains up to 200,000 NEW examples.
Examples are protected by SHA-256 hashes.
Session reservations are committed before training so a hard
Kaggle interruption cannot cause the same reserved examples
to be selected again.
## Resume safety
The checkpoint contains:
- model.safetensors
- optimizer.pt
- rng_state.pt
- training_state.json
- example_ledger.json
- tokenizer.json
- tokenizer_config.json
- token_id_manifest.json
- config.json
Dataset fingerprint and tokenizer artifact hashes are verified
before resume.
## Current state
Completed sessions:
5
Unique examples reserved/trained:
200,000
Unique completed examples:
460,000
Global optimizer steps:
7,189
Last session loss:
0.17293200694084168
|