Irodori-TTS v4 Small — Karume

What is this

A Japanese text-to-speech distribution: the Irodori-TTS v4 (Small) rectified-flow DiT converted into the WebGPU inference runtime Karume's container format (a single safetensors file = weights + a graph JSON embedded in __metadata__). Runs as-is in the browser and in Deno.

Eight graphs make up the chain: a shared Japanese ModernBERT backbone, the text_proj and caption_proj condition projectors, the reference-latent speaker encoder, the duration predictor, the dit itself (one forward per Euler step, plus one per classifier-free-guidance branch), and the DACVAE codec (Aratako/Semantic-DACVAE-Japanese-32dim) as codec_decoder / codec_encoder.
Text in, waveform out. generate() returns f32 mono samples at the codec's sample rate, ready for encodeWav. The decoder is run in tiles so that it also fits GPUs with the default 128MiB storage-buffer limit; the tiling is bit-exact against a single-shot decode.
Voice cloning is wired up both ways: a reference speaker is passed either as audio (mono f32 — what decodeWav returns), which codec_encoder turns into a DACVAE latent, or as such a latent directly. Reference audio must already be at this distribution's own 48000 Hz: there is no resampler, and a mismatch is refused rather than silently converted. Unlike the decoder, codec_encoder is not tiled, so a long reference can exceed the default 128MiB storage-buffer limit.
Not readable by the upstream implementation (it's a different container with an embedded graph); the reader is a pipeline that implements irodori/1.
Exporter used for the conversion: karume/0.5.0. The distribution manifest is karume.json (karume/4).

Base weights and attribution

Converted into the container format — the original checkpoint is not distributed here.

Weights: Aratako/Irodori-TTS-v4-Small, licensed MIT (as of retrieval). The training / inference implementation it comes with (Aratako/Irodori-TTS) is MIT as well; Karume's runtime contains none of that code — it is an independent implementation that reads these weights from its own container format.
Text backbone: fine-tuned from sbintuitions/modernbert-ja-310m, licensed MIT (as of retrieval). It is redistributed here in the container format as the backbone component, so that license travels with this repository too.
Codec: Aratako/Semantic-DACVAE-Japanese-32dim, licensed MIT (as of retrieval). Upstream ships it as a separate repository; it is redistributed here in the container format as the codec_decoder / codec_encoder components so that text-to-audio runs from this repository alone.
Changes made here: conversion into the Karume container format and quantization of the weights — every component is stored as f32 / f16 / i8 series, and dit adds an i4 series rounded with GPTQ calibration (the quant table below says which storage each quant selects). No retraining, no fine-tuning — the f32 series is the source checkpoint's own values, re-laid out per graph.

Models

Model	Pipeline	Quants	Default quant
`v4-small` (default)	`irodori/1`	`f32` / `f16` / `i8` / `i8-a8` / `i8+dit4`	`i8-a8`

model selects one of these; omitted, it is v4-small. quant defaults to that model's own default quant.

Usage

import { decodeWav, encodeWav, IrodoriPipeline } from "jsr:@karume/models";

using pipeline = await IrodoriPipeline.fromPretrained({
  repo: "hdae/karume-irodori-v4-small",
  // Pin a commit for reproducible builds — without it you track `main`, and a future
  // repo update (renamed files, new manifest format) may break your app.
  // Copy the full hash from this repo's "Files and versions" tab:
  // revision: "<full commit sha>",
}, {
  // model: "v4-small", // default — available: v4-small
  // quant: "i8-a8", // default — available: f16 / f32 / i8 / i8+dit4 / i8-a8
});

const audio = await pipeline.generate({
  text: "こんにちは、これはテストです。",

  // Voice Design — describe the voice in Japanese prose:
  // caption: "落ち着いた女性の声で、ゆっくりと丁寧に話している。",

  // Voice cloning — condition on a reference speaker. The WAV must already be
  // 48000 Hz mono or stereo (there is no resampler; a mismatch is refused):
  // speaker: { audio: decodeWav(await Deno.readFile("reference.wav")) },
  // ...or pass a DACVAE latent you saved earlier instead of the audio:
  // speaker: { latent: savedLatent },

  seed: 42, // same seed + same inputs → same audio
  // durationSeconds: 5, // override the predicted utterance length (seconds)
});

await Deno.writeFile(
  "out.wav",
  encodeWav(audio.data, audio.sampleRate),
);

generate() returns { data, sampleRate, frames, seed, forwards }, where data is f32 mono already trimmed to the predicted length. generateLatent() is the same run stopped one stage earlier: it returns { data, frames, latentDim, seed, forwards } with the patched DACVAE latent, for callers that want the embedding rather than audio. caption and speaker are both optional: without them the voice is picked by the model alone, and the guidance branches for the missing conditions are skipped. Weights are fetched once and cached (verified against karume.json's size / sha256). You can also build from bytes you fetched yourself (IrodoriPipeline.fromAssets).

Model: v4-small

Files

Key	Dtype	Path	Size	sha256
`backbone`	f32	`v4-small/backbone/model.f32.safetensors`	1.17 GiB (1,260,268,120 B)	`68adfa6bbc0055e1…`
`backbone`	f16	`v4-small/backbone/model.f16.safetensors`	601.89 MiB (631,122,240 B)	`10212e4bd86290c4…`
`backbone`	i8	`v4-small/backbone/model.i8.safetensors`	303.24 MiB (317,974,280 B)	`c1c65725c7fbcf16…`
`text_proj`	f32	`v4-small/text_proj/model.f32.safetensors`	6.51 MiB (6,830,520 B)	`228c213b511cdb87…`
`text_proj`	f16	`v4-small/text_proj/model.f16.safetensors`	3.26 MiB (3,422,632 B)	`3d5b32ae1d7762ff…`
`text_proj`	i8	`v4-small/text_proj/model.i8.safetensors`	1.65 MiB (1,727,368 B)	`f0d675e4cb4188f7…`
`caption_proj`	f32	`v4-small/caption_proj/model.f32.safetensors`	6.52 MiB (6,833,408 B)	`0f57d1199c5ec853…`
`caption_proj`	f16	`v4-small/caption_proj/model.f16.safetensors`	3.27 MiB (3,425,520 B)	`97fab35c37731917…`
`caption_proj`	i8	`v4-small/caption_proj/model.i8.safetensors`	1.65 MiB (1,730,320 B)	`28d3626bcfccbad7…`
`speaker`	f32	`v4-small/speaker/model.f32.safetensors`	231.29 MiB (242,524,180 B)	`6ee04bca91a6d3de…`
`speaker`	f16	`v4-small/speaker/model.f16.safetensors`	115.93 MiB (121,560,884 B)	`a11619e0969d188c…`
`speaker`	i8	`v4-small/speaker/model.i8.safetensors`	58.53 MiB (61,368,828 B)	`770d3c510dd081d6…`
`duration`	f32	`v4-small/duration/model.f32.safetensors`	83.13 MiB (87,169,416 B)	`e8663958481ebe99…`
`duration`	f16	`v4-small/duration/model.f16.safetensors`	41.63 MiB (43,651,392 B)	`ab0daf938f098bbb…`
`duration`	i8	`v4-small/duration/model.i8.safetensors`	20.99 MiB (22,010,268 B)	`7e17651ca36917a9…`
`dit`	f32	`v4-small/dit/model.f32.safetensors`	1.36 GiB (1,463,747,360 B)	`4126659457418bf6…`
`dit`	f16	`v4-small/dit/model.f16.safetensors`	698.59 MiB (732,528,384 B)	`0877e45172160570…`
`dit`	i8	`v4-small/dit/model.i8.safetensors`	351.45 MiB (368,519,976 B)	`2b0cf06ef8643c80…`
`dit`	i4	`v4-small/dit/model.i4.safetensors`	234.92 MiB (246,334,968 B)	`419af3abecba5474…`
`codec_decoder`	f32	`v4-small/codec_decoder/model.f32.safetensors`	249.34 MiB (261,450,332 B)	`714305c76d8ae27f…`
`codec_decoder`	f16	`v4-small/codec_decoder/model.f16.safetensors`	124.77 MiB (130,835,348 B)	`8a20bec7402a8014…`
`codec_decoder`	i8	`v4-small/codec_decoder/model.i8.safetensors`	62.55 MiB (65,584,104 B)	`a049fc62ab844109…`
`codec_encoder`	f32	`v4-small/codec_encoder/model.f32.safetensors`	104.29 MiB (109,358,808 B)	`a0f0c8ca4c051658…`
`codec_encoder`	f16	`v4-small/codec_encoder/model.f16.safetensors`	52.23 MiB (54,766,168 B)	`5c226477dee9253c…`
`codec_encoder`	i8	`v4-small/codec_encoder/model.i8.safetensors`	26.24 MiB (27,510,632 B)	`e172fac7467744e9…`
`tokenizer`	—	`v4-small/tokenizer/tokenizer.json`	3.08 MiB (3,232,154 B)	`a4a7095b103696ce…`

Only the first 16 hex digits of the sha256 are shown (the full value and size live in karume.json — verify against that at the fetch layer). Dtype labels use the runtime's storage dtype vocabulary (f16 / i8 / i4), not the fp16 spelling common elsewhere in the ecosystem. A path under shared/ is one this model shares byte for byte with another model in this repository (it is fetched and cached once).

Quants

Quant	What it is	Weights	Compute
`f32`	Full precision (f32) — Every graph in f32 storage — the largest download, and the source checkpoint's own values re-laid out per graph.	`backbone` = `f32` / `text_proj` = `f32` / `caption_proj` = `f32` / `speaker` = `f32` / `duration` = `f32` / `dit` = `f32` / `codec_decoder` = `f32` / `codec_encoder` = `f32`	—
`f16`	Half size (f16) — Every graph stored as f16 and computed in f32 — half the f32 download, with no audible difference in the listening check.	`backbone` = `f16` / `text_proj` = `f16` / `caption_proj` = `f16` / `speaker` = `f16` / `duration` = `f16` / `dit` = `f16` / `codec_decoder` = `f16` / `codec_encoder` = `f16`	—
`i8`	Quarter size (int8) — Every graph stored as int8 and computed in f32 — about a quarter of the f32 download, with the execution path left unchanged.	`backbone` = `i8` / `text_proj` = `i8` / `caption_proj` = `i8` / `speaker` = `i8` / `duration` = `i8` / `dit` = `i8` / `codec_decoder` = `i8` / `codec_encoder` = `i8`	—
`i8-a8` (default)	Balanced (int8, int8 linear) — The int8 weights with per-token int8 activations in the DiT's linear layers — the same download as plain int8, and faster per Euler step.	`backbone` = `i8` / `text_proj` = `i8` / `caption_proj` = `i8` / `speaker` = `i8` / `duration` = `i8` / `dit` = `i8` / `codec_decoder` = `i8` / `codec_encoder` = `i8`	`linearCompute` = `a8`
`i8+dit4`	Lowest memory (int4 DiT) — The DiT in GPTQ-calibrated int4 (group-32) while the other seven graphs stay int8 — the smallest download and the least resident memory.	`backbone` = `i8` / `text_proj` = `i8` / `caption_proj` = `i8` / `speaker` = `i8` / `duration` = `i8` / `dit` = `i4` / `codec_decoder` = `i8` / `codec_encoder` = `i8`	—

If no quant is given, it runs as i8-a8 (this model's recommended default).

Shape

Derived from the checkpoint's own config, and checked against the exported graphs when this repository was assembled.

text: up to 256 tokens (BOS included), width 512
caption: up to 512 tokens (BOS included), width 512
reference speaker: up to 750 patched rows (4 latent frames each = 120s of audio), width 768
latent: up to 750 frames at 25 Hz (30s), width 32
audio: 48000 Hz mono, 1920 samples per latent frame

Defaults

The sampler knobs are fixed by the manifest — generate() takes none of them.

steps: 40 Euler steps (initScale 0.999)
guidance: text 3.0 / speaker 5.0 / caption 3.0, applied for t in [0.5, 1.0]
duration: clamped to [0.5, 30.0] seconds (the duration predictor decides within that, unless durationSeconds is passed)

seed is the one knob the manifest does not carry — it defaults to 0, and the same seed with the same request gives the same audio.

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for hdae/karume-irodori-v4-small

Base model

Aratako/Irodori-TTS-500M-v2

Finetuned

Aratako/Irodori-TTS-600M-v3-VoiceDesign

Finetuned

Aratako/Irodori-TTS-v4-Small

Quantized

(1)

this model