| --- |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - testgeniy |
| - text-generation |
| - causal-lm |
| - reasoning |
| - mathematics |
| - logic |
| - long-context |
| - 4k-context |
| - small-language-model |
| --- |
| |
| # TestGeniy 4K Context Reasoning Model |
|
|
| TestGeniy is a compact causal language model focused on mathematical reasoning, formal logic, and helpful text interaction. |
|
|
| This `main` release is the validated 4K-context anchor. It is the safe production checkpoint after context-extension and regression testing. |
|
|
| ## Release summary |
|
|
| - Context window: 4096 tokens. |
| - RoPE: extended from 2048 to 4096 positions using the original theta value 500000. |
| - Attention: sliding attention with block size 1024 and global attention in layers 3, 7, 11, 15, 19, and 23. |
| - Weights: validated `logic_small_scope_step080` anchor, with context buffers extended to 4096. |
| - Evaluation questions were kept out of training. |
| - This main release does not include the rejected synthetic-CoT candidates. |
|
|
| ## Validation |
|
|
| The 4K model remained finite on full 4096-token forward passes and answered a 3157-token long-context probe correctly. |
|
|
| Fixed paired reasoning gate, 12 examples per dataset: |
|
|
| | Benchmark | Anchor | 4K main | |
| |---|---:|---:| |
| | GSM8K | 2/12 | 2/12 | |
| | MATH-500 | 2/12 | 2/12 | |
| | ARC-Challenge | 5/12 | 5/12 | |
| | FOLIO | 4/12 | 4/12 | |
|
|
| The release is a verified context-capability improvement with no measured regression on this gate. It is not presented as a benchmark-accuracy improvement. |
|
|
| ## Intended use |
|
|
| Use this checkpoint for compact English reasoning experiments, long-context prompting up to 4096 tokens, and further controlled fine-tuning. |
|
|
| ## Limitations |
|
|
| This is a small research model. It can produce incorrect reasoning or answers, especially on difficult mathematics and formal logic. The benchmark gate above is a regression gate, not a broad capability estimate. |
|
|
| ## Provenance |
|
|
| Base checkpoint: `logic_small_scope_step080` from this project. The published weights contain no benchmark questions and no synthetic-CoT training data. |
|
|
| ## Budgie Alignment v2 research handoff |
|
|
| A later, gate-driven Budgie-500M post-training research track is stored under [`candidates/budgie-alignment-v2/`](./candidates/budgie-alignment-v2/). |
|
|
| Start with the comprehensive [`Budgie Alignment v2 README`](./candidates/budgie-alignment-v2/README.md). It documents the current research leader, exact checkpoint lineage, evaluation protocols, confidence intervals, training-source policy, Qwen3.8+DFlash2 teacher setup, retained and rejected experiments, known limitations, and recommended next steps for a human or another AI agent. |
|
|
| These research candidates do **not** replace this root checkpoint automatically. |
|
|
