Senryu-2.2-1B

Hybrid Gated DeltaNet-2 + Parallax chat model (1.09B) with 2.17B parameters of n-gram memory

By KerzgrAI: Senryu-2.1-1B trained on 405M more tokens, half of them new conversations with tool use, full reasoning and system prompts

---

What's new in 2.2

Senryu-2.2-1B is the third model on the Senryu-2 architecture. It continues Senryu-2.1-1B with the same architecture, tokenizer, chat template and tool-call format, so it is a drop-in replacement.

• Better judgment about when to call tools. It searches the web when the answer needs facts it does not have, such as how something stands today or specifics it does not know, and answers from its own knowledge otherwise, so it makes fewer unnecessary searches. It fetches a page when given a link, and runs calculations through the calculator.
• Multi-step tool use. It takes several tool rounds when a question needs them: another search when the results are thin or the question has several parts, a page fetch when a result is cut short, then a calculation on the figures it found.
• Stronger reasoning. Half of its training tokens are new conversations with a full reasoning trace in nearly every turn's <think> block.
• Stronger system prompt following. About half of the new conversations were answered under their own system prompts: personas, output formats, rules and constraints from Multifaceted-Collection, SystemChat 2.0 and SystemChat 1.1.
• A new default system prompt. Its tool rules and its "When to use tools" section weigh each call against the wait it costs the user. inference.py and the demo start every chat with it and the three default tools.

---

Default system prompt

Conversations without a system prompt of their own were trained under this one, dated the day they were written (most with an earlier wording of its closing "When to use tools" section). inference.py and the demo fill in today's date in the same format, such as Tuesday, October 27, 2026:

You are Senryu-2.2, a large language model from KerzgrAI. Today's date is {Weekday, Month D, YYYY}.

Tool Rules:
- **Search the web whenever you truly need to. Do not use the tool for basic facts or opinions, only use it when you need it.**
- Write focused queries, and search again when a question has several parts or the results are thin, conflicting or off-topic.
- Fetch a page when the user links one, or when a promising result is cut short or lacks the details you need.
- **Use the calculator for every calculation beyond simple one-step mental math:** multi-digit arithmetic, percentages, unit conversions, rates and averages, interest and growth, powers, roots, logarithms, trigonometry and counting problems. Run each number through it even when you could work it out in your head, and never state a computed result you haven't calculated with it.
- Combine the tools when a question needs both: search for the figures, then calculate with them.
- Base your answer on what the tools return, and say so when they leave something unanswered.

Formatting Rules:
- Use Markdown for lists, tables, and styling.
- Use ```code fences``` for all code blocks.
- Format file names, paths, and function names with `inline code` backticks.
- **For all mathematical expressions, you must use dollar-sign delimiters. Use $...$ for inline math and $$...$$ for block math. Do not use \(...\) or \[...\] delimiters.**
- For responses with many sections where some are more important than others, use collapsible sections (HTML details/summary tags) to highlight key information while allowing users to expand less critical details.

When to use tools:
Each tool call makes the user wait, so call a tool only when it gives your answer something you cannot give on your own.

Web search: search when the answer depends on facts you do not have:
- the current state of something that changes or is new since your training, when the answer needs it as it is now;
- specifics the user asks for that you do not know, or know only well enough to guess, when the answer needs them exact;
- anything the user asks you to look up, find, check or source.
Search finds facts; it does not do your work. When the user asks you to work something out or create something, do it with your own reasoning, even when it is hard or looks like a problem with a known source, and search only for outside facts the work needs and you do not have; when the request seems ambiguous or flawed, say so and answer its most sensible reading. Do not search on your own initiative to confirm or cite what you already know, to add background the user did not ask for, or to pin down a detail the answer does not depend on: give such details as you know them, with the uncertainty they carry, or leave them out. The web cannot tell you who you are or what you can do, so answer questions about yourself from this system prompt and the conversation.
Before you search, name the fact you are missing and ask yourself: without it, would my answer be wrong, out of date or made up? Search only when the honest answer is yes. Then write one focused query for each missing fact rather than several phrasings of the same one, and search again only when the results leave that fact missing or unsettled.

Fetch page: call it when the user gives you a link, or when a search result is clearly the right source but its snippet is cut off or lacks the details you need. Do not fetch a page when the snippets already answer the question.

Calculator: call it for every calculation beyond simple one-step mental math, and never state a computed number you have not run through it. Do not call it when the answer involves no calculation.

In Python, tiny_gdn.senryu_chat.senryu2_2_1b_system_prompt() returns it dated today. Your own system prompt works too, since about half of the new conversations were answered under their own; pass --system "" or clear the demo's box to chat without any.

---

Default tools

Every new conversation offered these three tools, in this order. About 31% of the default-prompt conversations call one (27% the calculator, 9% web search, 1.5% fetch-page), and 41% of those take two or more tool rounds.

Tool	Arguments	Result, as trained
web-search	query (string, required)	Tavily search JSON: query, answer, and results with each hit's url, title and content
fetch-page	url (string, required), offset (integer)	URL: and Title: lines, then the page or PDF as Markdown, 12,000 characters at a time, ending with the offset to read on
calculator	expression (string, required)	The value, for example 452093301

The schemas, one per line as the system turn carries them:

{"type": "function", "function": {"name": "web-search", "description": "Search the web for a query.", "parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "Search query."}}, "required": ["query"]}}}
{"type": "function", "function": {"name": "fetch-page", "description": "Fetch a web page or PDF by URL and return its readable text as Markdown.", "parameters": {"type": "object", "properties": {"url": {"type": "string", "description": "The http or https URL to fetch."}, "offset": {"type": "integer", "description": "Character offset to start reading from, for pages longer than one result."}}, "required": ["url"]}}}
{"type": "function", "function": {"name": "calculator", "description": "Evaluate a math expression and return the result.", "parameters": {"type": "object", "properties": {"expression": {"type": "string", "description": "A math expression in standard calculator notation, for example (3 + 4)^2 * sqrt(2) / ln(10). Use + - * / ^ (power), % (remainder), ! (factorial) and parentheses; the constants pi and e; and functions such as sqrt, cbrt, exp, ln, log (natural, or log(x, base)), log10, log2, sin, cos, tan, asin, acos, atan (radians), abs, round, floor, ceil, factorial, comb, perm, gcd, lcm, min, max, sum, mean and median."}}, "required": ["expression"]}}}

They are tiny_gdn.senryu_chat.SENRYU_TOOLS. Other Hermes-style tools work the same way: list them in the <tools> block (--tools-json in the CLI), and run each call yourself.

---

Prompt format

Tool calls use the Hermes JSON format Senryu-2.2 trained on, which inference.py and the demo use by default. Two tool rounds exactly as training laid them out, and as tiny_gdn.chatml.encode_chatml_messages builds them (the system prompt is shortened here):

<|begin_of_text|><|im_start|>system
You are Senryu-2.2, a large language model from KerzgrAI. Today's date is Tuesday, October 27, 2026.

Tool Rules:
…
Calculator: call it for every calculation beyond simple one-step mental math, and never state a computed number you have not run through it. Do not call it when the answer involves no calculation.

You may call one or more functions to assist with the user query.
You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "web-search", …}}
{"type": "function", "function": {"name": "fetch-page", …}}
{"type": "function", "function": {"name": "calculator", …}}
</tools>

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call><|im_end|>
<|im_start|>user
What is 48213 * 9377, and what is 15% of that?<|im_end|>
<|im_start|>assistant
<|think|>
<think>
A five-digit product, then 15% of it: the calculator, one step at a time.
</think>

<tool_call>
{"name": "calculator", "arguments": {"expression": "48213 * 9377"}}
</tool_call><|im_end|>
<|im_start|>user
<|tool_response|>
452093301<|im_end|>
<|im_start|>assistant
<|think|>
<think>
Now 15% of 452093301.
</think>

<tool_call>
{"name": "calculator", "arguments": {"expression": "452093301 * 0.15"}}
</tool_call><|im_end|>
<|im_start|>user
<|tool_response|>
67813995.15<|im_end|>
<|im_start|>assistant
<|think|>

• Thinking. Each assistant turn starts with a control token: <|think|>\n, then a <think>…</think> block and the answer, or <|no_think|>\n and the answer only.
• Tools. The <tools> block follows the system prompt after a blank line. With no tools, the system turn is the prompt alone.
• Calls. The model writes any text first, then one <tool_call> block per call, each a JSON object with name and arguments. Independent calls share one turn; a call that needs an earlier result waits for the next round.
• Results. All results of a turn go back in one user turn, in call order, each starting with <|tool_response|>\n and separated by a newline. The model then answers in a new assistant turn, or calls again. With tiny_gdn, send each result as a {"role": "tool", "content": …} message and the encoder lays them out this way.

---

Install & run

pip install torch safetensors tokenizers huggingface_hub
hf download KerzgrAI/Senryu-2.2-1B inference.py --local-dir .
python inference.py

python inference.py --prompt "Explain recursion in two sentences."
python inference.py --prompt "What is 48213 * 9377?"
python inference.py --prompt "Summarize https://en.wikipedia.org/wiki/Senry%C5%AB in three bullets."
python inference.py --tools none --system ""
python inference.py --no-think --prompt "Reply in one sentence."
python inference.py --ngram-device cpu

inference.py downloads the weights, n-gram tables, tokenizer and tiny_gdn/, installs flash-linear-attention from git at the pinned commit (so Git must be on PATH) and installs the page readers fetch-page needs. A CUDA GPU is required: Gated DeltaNet-2 and Parallax run on Triton kernels.

The three default tools run automatically, round after round, up to --max-tool-rounds. web-search uses Tavily when TAVILY_API_KEY is set, and DuckDuckGo with Wikipedia otherwise, in the same JSON shape. fetch-page reads public http/https pages and PDFs only.

Flag	Default	Description
--prompt	—	One-shot user message
--system	the Senryu-2.2 prompt, dated today	System prompt, used verbatim; "" for none
--think / --no-think	think	Assistant control prefix
--tools	web-search,fetch-page,calculator	Built-ins, comma-separated: web-search, fetch-page, calculator, python; none for no tools
--tools-json	—	Extra tool schemas (JSON list) for the <tools> block
--max-tool-rounds	8	Tool rounds per reply
--temperature	0.6	Sampling temperature (0.3 suits tool-calling turns)
--top-p	0.95	Nucleus sampling
--top-k	50	Top-k sampling
--repetition-penalty	1.08	On content tokens seen twice in the last --repetition-window (256) tokens
--dry-multiplier	0.8	DRY penalty on verbatim loops and re-pasted replies; 0 disables
--dry-base	1.75	How fast the DRY penalty grows per repeated token
--dry-allowed-length	2	Tokens a phrase may repeat before DRY starts
--max-new-tokens	4096	Max generation length
--context-length	32768	Conversation tokens kept (oldest dropped first)
--device	cuda	CUDA device, e.g. cuda:1
--ngram-device	same as --device	Where the n-gram tables live; cpu saves 4.3 GB of VRAM

Sampling matches the KerzgrAI API: the repetition penalty leaves Markdown and punctuation alone, and DRY breaks verbatim loops but stays off inside code fences and tool calls.

Interactive commands: /think /no_think /system … /reset /exit. Ctrl+C stops the current reply.

---

Model architecture

Core parameters	1,094,961,660 (embeddings included)
N-gram memory	2,166,398,464 table parameters: 2- and 3-gram hashes, four hash heads per order
Layers	28: 7 × (3 Gated DeltaNet-2 + 1 Parallax attention)
Hidden	1,536
MLP	SiTU-GLU 3,584
Attention	Parallax: 12 query and 4 KV heads of 128, RoPE on half of each head (θ = 10⁶)
Linear	Gated DeltaNet-2, 12 heads × 128, with a short convolution
Residuals	Four-branch gated residual stream (rank-128 gate)
Memory read	Before block 1, with a context-aware gate per residual branch, as in Engram
Vocab	65,536 BPE, tied embeddings
Context	32,768

---

Training

Continued from Senryu-2.1-1B's final checkpoint (EMA weights) for 773 steps of 16 sequences × 32,768 tokens: one pass over a 404.8M-token mix, 229.2M of them assistant targets, in about 10 hours on one RTX PRO 6000 Blackwell GPU. Muon v2 + AdamW at a constant 5×10⁻⁴ after a 50-step warmup, and sparse Adam at 2.5×10⁻³ for the n-gram tables. Half the mix is new conversations, half replay:

Part	Conversations	Tokens	What it is
System prompts	13,343	88.1M	Multifaceted-Collection, SystemChat 2.0 and SystemChat 1.1 openings, answered with full reasoning under their own system prompts, with the three default tools
Default prompt and tools	14,695	114.3M	smol-magpie-ultra openings, answered with full reasoning as Senryu-2.2 under the default system prompt and the three default tools
Replay	65,646	202.4M	A fresh draw from Senryu-2.1's SFT mixture, without the conversations its final pass trained on, filtered of other models' self-descriptions

About 30% of the new conversations go on to a follow-up user turn. Only assistant turns are trained. The released EMA weights reach a loss of 1.829 on the mix's held-out split, against 1.896 for the raw weights. sft_config.json holds the run's settings.

---

Files

File	Contents
model.safetensors	Core weights (EMA), bfloat16
ngram_tables.safetensors	N-gram tables (bfloat16), canonical ids and hash multipliers
config.json	Architecture
tokenizer.json, special_token_ids.json	Tokenizer and control-token ids
inference.py, requirements.txt, tiny_gdn/	Standalone chat CLI and the model code it imports
sft_config.json	Training settings
windows_fla_patches/	Import patches inference.py applies to fla on Windows

---

License

Weights are Apache-2.0. Each training dataset keeps its own license; some carry attribution or non-commercial terms, so check them before commercial use.

---

Model family

Model	Notes	Hub
Senryu-2.2-1B	Senryu-2 architecture: better judgment about tools, multi-step tool use, stronger reasoning and system prompt following	this repo
Senryu-2.1-1B	Better reasoning and system prompt following, default prompt and tools	KerzgrAI/Senryu-2.1-1B
Senryu-2-1B	First Senryu-2 model	KerzgrAI/Senryu-2-1B
Senryu	Previous generation (803M)	kerzgrr/Senryu
Haiku	Earlier generation (~655M)	kerzgrr/Haiku

---

References

• Gated DeltaNet-2: https://arxiv.org/abs/2605.22791
• Parallax attention: https://arxiv.org/abs/2605.29157
• Engram conditional memory: https://arxiv.org/abs/2601.07372
• flash-linear-attention, pinned at cbb0a72

@misc{senryu2_2_1b_2026,
  title={Senryu-2.2-1B: A 1.1B Hybrid Gated DeltaNet-2 + Parallax Chat Model with Hashed N-gram Memory},
  author={KerzgrAI},
  year={2026},
  url={https://huggingface.co/KerzgrAI/Senryu-2.2-1B}
}