File size: 2,710 Bytes
a77741c cd59bf4 a77741c cd59bf4 a77741c cd59bf4 9156d62 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | ---
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- testgeniy
- text-generation
- causal-lm
- reasoning
- mathematics
- logic
- long-context
- 4k-context
- small-language-model
---
# TestGeniy 4K Context Reasoning Model
TestGeniy is a compact causal language model focused on mathematical reasoning, formal logic, and helpful text interaction.
This `main` release is the validated 4K-context anchor. It is the safe production checkpoint after context-extension and regression testing.
## Release summary
- Context window: 4096 tokens.
- RoPE: extended from 2048 to 4096 positions using the original theta value 500000.
- Attention: sliding attention with block size 1024 and global attention in layers 3, 7, 11, 15, 19, and 23.
- Weights: validated `logic_small_scope_step080` anchor, with context buffers extended to 4096.
- Evaluation questions were kept out of training.
- This main release does not include the rejected synthetic-CoT candidates.
## Validation
The 4K model remained finite on full 4096-token forward passes and answered a 3157-token long-context probe correctly.
Fixed paired reasoning gate, 12 examples per dataset:
| Benchmark | Anchor | 4K main |
|---|---:|---:|
| GSM8K | 2/12 | 2/12 |
| MATH-500 | 2/12 | 2/12 |
| ARC-Challenge | 5/12 | 5/12 |
| FOLIO | 4/12 | 4/12 |
The release is a verified context-capability improvement with no measured regression on this gate. It is not presented as a benchmark-accuracy improvement.
## Intended use
Use this checkpoint for compact English reasoning experiments, long-context prompting up to 4096 tokens, and further controlled fine-tuning.
## Limitations
This is a small research model. It can produce incorrect reasoning or answers, especially on difficult mathematics and formal logic. The benchmark gate above is a regression gate, not a broad capability estimate.
## Provenance
Base checkpoint: `logic_small_scope_step080` from this project. The published weights contain no benchmark questions and no synthetic-CoT training data.
## Budgie Alignment v2 research handoff
A later, gate-driven Budgie-500M post-training research track is stored under [`candidates/budgie-alignment-v2/`](./candidates/budgie-alignment-v2/).
Start with the comprehensive [`Budgie Alignment v2 README`](./candidates/budgie-alignment-v2/README.md). It documents the current research leader, exact checkpoint lineage, evaluation protocols, confidence intervals, training-source policy, Qwen3.8+DFlash2 teacher setup, retained and rejected experiments, known limitations, and recommended next steps for a human or another AI agent.
These research candidates do **not** replace this root checkpoint automatically.
|