---
license: apache-2.0
language:
  - en
tags:
  - text-generation
  - causal-lm
  - pytorch
  - sft
  - instruction-tuned
  - chat
  - reasoning
  - tool-use
  - hybrid
  - gated-deltanet
  - gqa
  - tercet
pipeline_tag: text-generation
library_name: tiny_gdn
datasets:
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceTB/smoltalk2
  - HuggingFaceTB/smoltalk
  - nvidia/Llama-Nemotron-Post-Training-Dataset
  - nvidia/Nemotron-Post-Training-Dataset-v1
  - nvidia/Nemotron-Agentic-v1
  - nvidia/Nemotron-SFT-Agentic-v2
  - nvidia/Nemotron-SFT-Instruction-Following-Chat-v2
  - nvidia/Nemotron-Instruction-Following-Chat-v1
  - nvidia/Nemotron-Cascade-2-SFT-Data
  - Agent-Ark/Toucan-1.5M
  - interstellarninja/hermes_reasoning_tool_use
  - NousResearch/Hermes-3-Dataset
  - allenai/WildChat-4.8M
  - open-thoughts/OpenThoughts3-1.2M
  - teknium/OpenHermes-2.5
  - open-r1/Mixture-of-Thoughts
  - CohereLabs/aya_dataset
  - NousResearch/hermes-function-calling-v1
  - simplescaling/s1K-1.1
  - allenai/tulu-3-sft-personas-instruction-following
  - Salesforce/xlam-function-calling-60k
  - THUDM/LongAlign-10k
base_model: kerzgrr/Tercet-R-1.0
---

<div align="center">

# Tercet-R-1.1

### Reasoning + tool-call chat model (~502M) — latest Tercet-R snapshot

[![Model](https://img.shields.io/badge/Model-~502M_params-blue)](.)
[![Stage](https://img.shields.io/badge/Stage-SFT_stage--3-green.svg)](.)
[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](LICENSE)
[![Demo](https://img.shields.io/badge/Space-tercet--r--1.1--demo-indigo.svg)](https://huggingface.co/spaces/kerzgrr/tercet-r-1.1-demo)

*A ~502M hybrid GDN-2 + GQA model, continued on multi-turn tools and instruction following*

</div>

---

## What this is

**Tercet-R-1.1** is the latest public reasoning / tool-use checkpoint in the Tercet-R line.

- Previous public snapshot: [`kerzgrr/Tercet-R-1.0`](https://huggingface.co/kerzgrr/Tercet-R-1.0)
- Base (pretrain): [`kerzgrr/Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base)
- Live demo: [`kerzgrr/tercet-r-1.1-demo`](https://huggingface.co/spaces/kerzgrr/tercet-r-1.1-demo)
- Hub weights are the **EMA** snapshot in **bfloat16**

This upload is the latest **complete** stage-3 SFT checkpoint (optimizer step **5,300**). The stage-3 run is still in progress toward a much longer schedule; 1.1 is the snapshot that existed when this repo was published.

Initialization was the instruct-run EMA at step 5,000 (`tercet-r-instruct`), not the public 1.0 file (that release packaged instruct step 4,500). The chat contract is unchanged.

---

## Chat contract

### Thinking

Each assistant turn is prefixed with a **zero-loss control token**:

| Mode | Prefix | Typical body |
|------|--------|----------------|
| think | `<\|think\|>\n` | `<think>…</think>` then the answer |
| no-think | `<\|no_think\|>\n` | answer only |

`inference.py` streams the `<think>` region live (dim yellow) and hides the control tokens.

### Tool calls (SmolTalk JSON)

```
<tool_call>
{"name": "web-search", "arguments": {"query": "…"}}
</tool_call>
```

Stage-3 also saw `stateful_python_code_exec` leftovers from earlier Cascade-2 SFT. `inference.py` auto-runs:

| Built-in | Tool name | Observation |
|----------|-----------|-------------|
| `--tools web-search` | `web-search` | Tavily-shaped JSON (Tavily if `TAVILY_API_KEY` is set, else DuckDuckGo + Wikipedia) |
| `--tools python` | `stateful_python_code_exec` | Jupyter-style stdout / last value from a restricted interpreter |
| `--tools calculator` | `calculator` | Numeric result of a math expression |

### Tool results

Each observation is a `tool` (or `user`) turn prefixed with:

```
<|tool_response|>
{observation}
```

---

## Install & run

```bash
pip install torch safetensors tokenizers huggingface_hub
hf download kerzgrr/Tercet-R-1.1 inference.py --local-dir .
python inference.py
```

```bash
python inference.py --prompt "What is the capital of France?"
python inference.py --tools web-search,python,calculator
python inference.py --no-think --prompt "Reply in one sentence."
```

`inference.py` auto-downloads weights / tokenizer / `tiny_gdn/` and auto-installs pinned `flash-linear-attention`. Git is required on `PATH`.

| Flag | Default | Description |
|------|---------|-------------|
| `--prompt` | — | One-shot user message |
| `--system` | — | System prompt, used verbatim |
| `--think` / `--no-think` | think | Assistant control prefix |
| `--tools` | — | Built-ins: `web-search`, `python`, `calculator` (comma-separated) |
| `--temperature` | `0.7` | Sampling temperature |
| `--max-new-tokens` | `4096` | Max generation length |
| `--device` | `cuda` if available | `cuda` / `cpu` |

Interactive commands: `/think` `/no_think` `/system …` `/reset` `/exit`.

---

## Model architecture

Same TinyGDN hybrid as [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) (501,635,264 parameters):

| | |
|--|--|
| **Layers** | 32 (GDN-2 ×3 + GQA every 4th) |
| **Hidden** | 1,024 |
| **MLP** | SwiGLU 2,624 |
| **Attention** | 8 Q / 2 KV, head dim 128, partial RoPE |
| **Linear** | Gated DeltaNet-2, 8 heads × 128 |
| **Vocab** | 49,152 BPE |
| **Context** | 16,384 |

---

## Training

| Stage | Details |
|-------|---------|
| **Base** | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) → [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) |
| **Mid + instruct SFT** | [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid / SFT → instruct EMA @ step 5,000 |
| **Stage-3 SFT** | Multi-turn mix: Nemotron PTD v1, Agentic v1/v2 (including `web-search`), IF-Chat v1/v2, Cascade-2 instruction following, Toucan 1.5M, Hermes-3, Hermes reasoning tool-use, WildChat-4.8M. Seq 16,384, AdamW 3×10⁻⁵, 27.5 hours so far, 602M training tokens |
| **Checkpoint** | optimizer step 5,300 (latest complete stage-3 snapshot) |
| **Weights** | EMA (this repo's `model.safetensors`) |
| **Val loss (EMA)** | 2.027 (ppl 7.59) on the stage-3 holdout — a harder mix than the instruct-only 1.0 val set |

Stage-3 does **not** re-include Cascade-2 `math/math_tool.jsonl`. The restricted Python tool in `inference.py` is there so leftover `stateful_python_code_exec` calls still compute instead of failing open.

---

## Limitations

- **In-progress run**: this is a mid-stage-3 snapshot, not a finished 1.1 train
- **Scale**: ~502M is a research / edge model, not a frontier system
- Requires `flash-linear-attention`; not GGUF / llama.cpp compatible today
- The Python tool is a restricted interpreter (math-oriented imports only)

---

## Model family

| Model | Stage | Hub |
|-------|-------|-----|
| Tercet-base | Pretrain | [`kerzgrr/Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) |
| Tercet | SFT chat | [`kerzgrr/Tercet`](https://huggingface.co/kerzgrr/Tercet) |
| Tercet-R-1.0 | SFT reasoning + tools | [`kerzgrr/Tercet-R-1.0`](https://huggingface.co/kerzgrr/Tercet-R-1.0) |
| **Tercet-R-1.1** | **Stage-3 SFT (latest)** | **this repo** |
| Demo | ZeroGPU Space | [`kerzgrr/tercet-r-1.1-demo`](https://huggingface.co/spaces/kerzgrr/tercet-r-1.1-demo) |

---

## Citation

```bibtex
@misc{tercetr11,
  title={Tercet-R-1.1: A 502M Hybrid GDN-2 + GQA Reasoning Model},
  author={kerzgrr},
  year={2026},
  url={https://huggingface.co/kerzgrr/Tercet-R-1.1}
}
```

---

<div align="center">

*R is for reasoning.*

</div>