LFM2.5-1.2B-Thinking — LiteRT-LM

LiquidAI/LFM2.5-1.2B-Thinking converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime (requires litert-lm ≥ 0.14 / a recent AI Edge Gallery). Sibling of litert-community/LFM2.5-1.2B-Instruct.

Update (2026-08-04): the .litertlm files were updated in place to add the ExecutorMetadata section that litert-lm ≥ 0.15 requires to bind the hybrid conv/attention state buffers (without it, 0.15 fails at inference with missing some output TensorBuffers). Weights and graph are byte-identical to the original release, and the files continue to run on litert-lm 0.14.

LFM2.5-1.2B-Thinking is the reasoning variant of Liquid AI's hybrid conv-attention flagship: it works problems inside <think>…</think> before answering. The bundle declares the thought channel in its metadata, so LiteRT-LM ≥ 0.14 streams the reasoning on a separate thought channel — your app can show or hide it natively, and past thinking is stripped from multi-turn context automatically.

File	Recipe	Size	GSM8K (n=100)
LFM2.5-1.2B-Thinking_int8.litertlm	int8 dynamic (linears + embedding; convs float)	1.24 GB	77% (bf16 reference: 81%)
LFM2.5-1.2B-Thinking_int4.litertlm	int4 blockwise-32 + OCTAV linears, int8 embedding, convs float	736 MB	72%

Context (KV cache)	4096 max
Backend	CPU (the hybrid conv graph is not supported by current GPU delegates)
Template	bundled — full chat template + thought channel (<think>/</think>)
Base model	LiquidAI/LFM2.5-1.2B-Thinking (LFM Open License v1.0)

Accuracy

GSM8K (greedy, 0-shot CoT, max-tokens 2048 — a thinking model needs the budget, n=100, same harness for all rows): PyTorch bf16 81% · LiteRT int8 77% (−4pt) · int4 72% (−9pt). Both files pass an 8-question sanity gate (7/8, zero degenerate); the reasoning stream arrives on the thought channel and the final answer follows cleanly after </think>.

Usage

litert-lm run ./LFM2.5-1.2B-Thinking_int8.litertlm --prompt "A train travels 60 km in 45 minutes. What is its average speed in km/h?"

Give it a generous token budget (≥2048) — a reasoning model truncated mid-thought produces no final answer. The reasoning arrives on the thought channel; the final answer arrives on the main text channel after </think>.

Run on Android

Install a recent Google AI Edge Gallery, import this repo (or adb push a file and use local import: menu → Models → “+” → From local model file), select the CPU backend, set max tokens high (2048–4096), and chat.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, CPU backend, -p 256 -d 256 --runs 3 (the tool averages three iterations), warm-up run discarded, otherwise idle machine. Decode on this family depends strongly on the KV budget, so both settings are listed:

Variant	--max-num-tokens	Prefill (256)	Decode	TTFT
int8	1024	1536 tok/s	98.8 tok/s	0.18 s
int4	1024	386 tok/s	118.9 tok/s	0.67 s
int8	4096	1121 tok/s	83.2 tok/s	0.24 s
int4	4096	342 tok/s	77.4 tok/s	0.76 s

Set --max-num-tokens to the smallest value your use case needs — 1024 is a good chat default, and the file allows up to 4096. At 1024 the int4 file decodes fastest; at 4096 the two variants converge.

Use the CPU backend — this bundle cannot create a GPU engine. It comes from the pre-0.9.2 ShortConv export generation, whose prefill graph still carries INT64 ADD/CAST inside Lfm2ShortConv, plus GATHER_ND and a GREATER_EQUAL with const inputs. The GPU delegate takes 536 of the 579 operations and leaves 43 on the CPU, and the runtime then refuses the partial split: Hint fully delegated to single delegate is set, but the graph is not fully delegated. Re-exporting from the post-0.9.2 lineage removes those INT64 ops and does run fully delegated on the macOS GPU; the remaining iOS Metal failure is tracked upstream in LiteRT-LM#3129.

On a Pixel 8a (Tensor G3, CPU backend, measured in AI Edge Gallery) int8 decodes at ~19 tok/s and int4 at ~31 tok/s — on phone-class memory bandwidth the int4 file is about 1.7× faster as well as 41% smaller, so prefer int4 on mid-range devices. Those Android figures are single ship-gate runs, not medians. First device load compiles the graph and can take about a minute; later loads are instant.

Known limitation — multi-turn conversations on litert-lm >= 0.15

On the current runtime a conversation's context retains previous turns' <think> reasoning (history is appended, not re-rendered), while the model was trained with past reasoning stripped from its context. Single-turn quality is unaffected, but across several turns in one conversation the model can drift — a later answer may repeat an earlier turn's reply instead of addressing the new question (observed at turn 3 of a 3-turn probe, litert-lm 0.15.0 CPU, both variants). Practical guidance: start a fresh conversation per task (conversation objects are cheap; the engine can stay loaded), and size --max-num-tokens so thinking turns complete — a reply truncated mid-<think> derails the turns after it.

Conversion notes

Converted with released litert-torch 0.9.1 with the same exporter fix as the Instruct sibling: the stock LFM2 short-conv block saves its conv state from padded prefill columns, corrupting the first generated token of nearly every reply; the fix derives the chunk's valid length from the attention mask in-graph and gathers the state from the last valid columns (verified token-identical to an exact per-token reference loop). Multi-length prefill signatures (1–1024). Quantization: export-time int8 including convs, or post-hoc int4 on linears only — post-hoc conv quantization breaks generation.

License and changes

Distributed under the LFM Open License v1.0 (see LICENSE, inherited from the base model). Note the license's commercial-use limitation for organizations above US$10M annual revenue. Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged unmodified; exporter conv-state fix as described in Conversion notes. This repository is a community conversion and is not affiliated with Liquid AI.