Tercet-R-1.1

Reasoning + tool-call chat model (~502M) — latest Tercet-R snapshot

A ~502M hybrid GDN-2 + GQA model, continued on multi-turn tools and instruction following

What this is

Tercet-R-1.1 is the latest public reasoning / tool-use checkpoint in the Tercet-R line.

Previous public snapshot: kerzgrr/Tercet-R-1.0
Base (pretrain): kerzgrr/Tercet-base
Live demo: kerzgrr/tercet-r-1.1-demo
Hub weights are the EMA snapshot in bfloat16

This upload is the latest complete stage-3 SFT checkpoint (optimizer step 5,300). The stage-3 run is still in progress toward a much longer schedule; 1.1 is the snapshot that existed when this repo was published.

Initialization was the instruct-run EMA at step 5,000 (tercet-r-instruct), not the public 1.0 file (that release packaged instruct step 4,500). The chat contract is unchanged.

Chat contract

Thinking

Each assistant turn is prefixed with a zero-loss control token:

Mode	Prefix	Typical body
think	`<\|think\|>\n`	`<think>…</think>` then the answer
no-think	`<\|no_think\|>\n`	answer only

inference.py streams the <think> region live (dim yellow) and hides the control tokens.

Tool calls (SmolTalk JSON)

<tool_call>
{"name": "web-search", "arguments": {"query": "…"}}
</tool_call>

Stage-3 also saw stateful_python_code_exec leftovers from earlier Cascade-2 SFT. inference.py auto-runs:

Built-in	Tool name	Observation
`--tools web-search`	`web-search`	Tavily-shaped JSON (Tavily if `TAVILY_API_KEY` is set, else DuckDuckGo + Wikipedia)
`--tools python`	`stateful_python_code_exec`	Jupyter-style stdout / last value from a restricted interpreter
`--tools calculator`	`calculator`	Numeric result of a math expression

Tool results

Each observation is a tool (or user) turn prefixed with:

<|tool_response|>
{observation}

Install & run

pip install torch safetensors tokenizers huggingface_hub
hf download kerzgrr/Tercet-R-1.1 inference.py --local-dir .
python inference.py

python inference.py --prompt "What is the capital of France?"
python inference.py --tools web-search,python,calculator
python inference.py --no-think --prompt "Reply in one sentence."

inference.py auto-downloads weights / tokenizer / tiny_gdn/ and auto-installs pinned flash-linear-attention. Git is required on PATH.

Flag	Default	Description
`--prompt`	—	One-shot user message
`--system`	—	System prompt, used verbatim
`--think` / `--no-think`	think	Assistant control prefix
`--tools`	—	Built-ins: `web-search`, `python`, `calculator` (comma-separated)
`--temperature`	`0.7`	Sampling temperature
`--max-new-tokens`	`4096`	Max generation length
`--device`	`cuda` if available	`cuda` / `cpu`

Interactive commands: /think /no_think /system … /reset /exit.

Model architecture

Same TinyGDN hybrid as Tercet-base (501,635,264 parameters):


Layers	32 (GDN-2 ×3 + GQA every 4th)
Hidden	1,024
MLP	SwiGLU 2,624
Attention	8 Q / 2 KV, head dim 128, partial RoPE
Linear	Gated DeltaNet-2, 8 heads × 128
Vocab	49,152 BPE
Context	16,384

Training

Stage	Details
Base	HuggingFaceFW/fineweb-edu → `Tercet-base`
Mid + instruct SFT	HuggingFaceTB/smoltalk2 Mid / SFT → instruct EMA @ step 5,000
Stage-3 SFT	Multi-turn mix: Nemotron PTD v1, Agentic v1/v2 (including `web-search`), IF-Chat v1/v2, Cascade-2 instruction following, Toucan 1.5M, Hermes-3, Hermes reasoning tool-use, WildChat-4.8M. Seq 16,384, AdamW 3×10⁻⁵, 27.5 hours so far, 602M training tokens
Checkpoint	optimizer step 5,300 (latest complete stage-3 snapshot)
Weights	EMA (this repo's `model.safetensors`)
Val loss (EMA)	2.027 (ppl 7.59) on the stage-3 holdout — a harder mix than the instruct-only 1.0 val set

Stage-3 does not re-include Cascade-2 math/math_tool.jsonl. The restricted Python tool in inference.py is there so leftover stateful_python_code_exec calls still compute instead of failing open.

Limitations

In-progress run: this is a mid-stage-3 snapshot, not a finished 1.1 train
Scale: ~502M is a research / edge model, not a frontier system
Requires flash-linear-attention; not GGUF / llama.cpp compatible today
The Python tool is a restricted interpreter (math-oriented imports only)

Model family

Model	Stage	Hub
Tercet-base	Pretrain	`kerzgrr/Tercet-base`
Tercet	SFT chat	`kerzgrr/Tercet`
Tercet-R-1.0	SFT reasoning + tools	`kerzgrr/Tercet-R-1.0`
Tercet-R-1.1	Stage-3 SFT (latest)	this repo
Demo	ZeroGPU Space	`kerzgrr/tercet-r-1.1-demo`

Citation

@misc{tercetr11,
  title={Tercet-R-1.1: A 502M Hybrid GDN-2 + GQA Reasoning Model},
  author={kerzgrr},
  year={2026},
  url={https://huggingface.co/kerzgrr/Tercet-R-1.1}
}

R is for reasoning.