VibeVoice-ASR-BitNet-slim

A repack of microsoft/VibeVoice-ASR-BitNet with a redundant tensor removed. No retraining; the ternary transformer body is byte-for-byte the original.

	LM	VAE	Total	bits/weight (LM)
microsoft/VibeVoice-ASR-BitNet	992.9 MB	703.1 MB	1.70 GB	4.44
this repo	526.1 MB	703.1 MB	1.23 GB	2.69

Runtime, kernels and speed engineering live in the VibeASR.cpp fork and are not repeated here. This card covers the weights: what changed, how accurate they are, and where they break.

What changed in the weights

The released LM GGUF stores output.weight as F16 (466.7 MB — 47% of the file) next to token_embd.weight as Q6_K. In the source checkpoint tie_word_embeddings is true and the two matrices are bit-identical, so the F16 tensor is the same matrix twice at higher precision. llama.cpp loads LLM_TENSOR_OUTPUT as optional and falls back to token_embd, so the duplicate is simply dropped: the output projection runs through the Q6_K copy. That is the only numerical change, and it also removes 466.7 MB from every decoded token's memory traffic.

Verified twice on independent suites: +0.30 and +0.32 corpus WER versus the original weights, with per-language deltas scattering in both directions — i.e. no measurable accuracy cost for a 27% size reduction.

Accuracy

Greedy decoding, corpus-level WER with digits and years spelled out on both sides. Two regimes, because they rank engines differently.

Short clips (5–25 s), 7 language/register sets

Engine	WER
whisper.cpp large-v3-turbo q5_0	2.39
whisper.cpp small q5_1	5.79
microsoft original weights	8.41
this repo	8.73

On short clips whisper is more accurate than this model, and whisper-small is also faster. If short-clip transcription accuracy is what you need, use whisper.

Per language (100-clip suite, this repo vs the original weights — the comparison this card is actually about):

	de	en	es	fr	fr-MLS	it	pt	all
microsoft original	16.7	6.6	5.8	32.7	21.6	8.3	10.8	15.75
this repo	15.6	6.6	7.1	33.6	21.9	8.6	11.1	16.07

(The absolute level differs from the table above because that one uses a 14-item two-regime suite; both compare engines measured in the same session.)

Long-form (real TED talks, whole recordings)

Engine	mode	7.7 min	13.9 min
whisper.cpp small q5_1	native	5.10	4.78
this repo	60 s chunks	5.67	4.70
microsoft original weights	60 s chunks	5.61	5.38
whisper.cpp large-v3-turbo q5_0	native	29.00	39.69

On genuine long-form this model is level with whisper-small — each wins one talk — and roughly 5 WER against a 2.4 on short clips, so the gap to whisper closes substantially as recordings get longer. whisper-turbo collapses into a repetition loop on both talks under whisper.cpp's default flags (its reference implementation has temperature-fallback logic that suppresses this).

Measured on real continuous speech (distil-whisper/tedlium-long-form), not on concatenated short clips — splicing independent utterances produces speaker jumps that break every engine and measure nothing.

Limits you must design around

• ~80 seconds per pass. Past that the decoder emits its end token early and silently drops the tail (measured: 101 s of audio → 119 of 280 reference words, with the token budget 98% unused). This is a property of the weights — the original checkpoint behaves identically. Chunk longer audio; 60 s chunks give the long-form numbers above.
• No speaker labels. The runtime can request a {Start, End, Speaker, Content} format, but that prompt targets the 7B checkpoint and this 1.5B model emits no speaker turns and transcribes ~1.4 WER worse when asked for it. The CLI rejects it. There is no diarization here.
• Language coverage. VibeVoice-ASR was trained on en, zh, fr, it, ko, pt, vi. Spanish and German are out of distribution but generalise usably; French is in-distribution yet weak on FLEURS' proper-noun-heavy register (33 WER) and much better on read speech (22 on MLS). Other languages degrade sharply. This repack does not change coverage.

Usage

Drop-in for the released model — same runtime, same flags:

./build/bin/asr_infer \
    --vae-model vibeasr-vae-encoder-i8_s.gguf \
    --lm-model  vibeasr-lm-i2_s-tied.gguf \
    --audio input.wav -t 4 --greedy

Domain terms can be biased at decode time with --hotwords a,b,c --hotword-boost 5 (token-trie logit boosting): measured on FLEURS-French with oracle terms it recovers 36.0 → 31.9 WER, and feeding the wrong clip's terms at the same strength does not degrade the baseline. λ=8 is past the stability knee.

Provenance

Produced with tools/requant_lm_head.cpp --drop. The VAE encoder and tokenizer files are copied unmodified from upstream. Licensed MIT, as upstream.