schism-audio/beat-this-coreml

Beat This! — Core ML

CPJKU/beat_this (final0 checkpoint; "Beat This! Accurate Beat Tracking Without DBN Postprocessing", Foscarin, Schlüter & Widmer, ISMIR 2024, MIT licensed) for Core ML on Apple devices. Frame-wise beat and downbeat tracking at 50 fps with minimal peak-picking postprocessing — no DBN.

Converted from the same verified reference used by the schism-mlx MLX ports. Two variants per model:

File	Precision	Compute units	max logit diff
BeatThis_fp16.mlpackage	FLOAT16	ALL (ANE)	6.7e-2, peak-picked beat/downbeat times identical on tested real drum audio
BeatThis_fp32.mlpackage	FLOAT32	CPU+GPU	5.2e-5, peak-picked beat/downbeat times identical on all tested clips

Verified on-device-equivalently via coremltools on an M5 Max, against the reference implementation on real audio. fp16 is ANE-eligible and recommended for iPhone / iPad; fp32 is the tight-parity fallback.

Download

.mlpackage bundles must be materialized as real files — the Core ML compiler rejects the symlinks that a default snapshot_download creates in the Hugging Face cache:

from huggingface_hub import snapshot_download
path = snapshot_download("schism-audio/beat-this-coreml", local_dir="./beat-this-coreml")

(or hf download schism-audio/beat-this-coreml --local-dir ./beat-this-coreml). Swift hosts downloading files directly are unaffected.

I/O contract

• input spect: (1, 1500, 128) float32 — one 30 s chunk of the log-mel frontend (22.05 kHz mono, n_fft 1024, hop 441, 128 slaney mels 30–11000 Hz without area norm, magnitude STFT normalized 1/sqrt(n_fft), log1p(1000 x)), 50 fps
• outputs beat_logits / downbeat_logits: (1, 1500) float32 — frame-wise logits at 50 fps; sigmoid for probabilities, peak-pick for times (local maxima within ±3 frames with logit > 0, adjacent peaks merged by averaging, downbeats snapped to the nearest beat — the reference "minimal" postprocessor)
• longer audio: 1500-frame chunks starting at -6 + k*1488, last start shifted left to n - 1494 so the last chunk ends at the piece end; chunk borders that fall outside the piece are zero-padded on the spectrogram; discard the 6 border frames of every chunk, earlier chunks winning on overlap ("keep_first"); frames no chunk covers keep the filler -1000 (see test_vectors_beatmel.npz params)
• pieces shorter than 1488 frames (~29.8 s): the reference runs one shorter window that this fixed-shape graph cannot reproduce — attention is global, so zero-padding to 1500 frames changes all logits slightly (beat times on tested real audio were still identical; pad with zeros and drop the padded frames, or prefer the MLX port for exactness)

DSP frontend (host-side)

The Core ML graph contains the network only. The host implements the log-mel frontend and must match schism_mlx.analyze.beat_this.model.logmel_beat_this numerically — test_vectors_beatmel.npz in this repo holds deterministic input/output pairs plus the exact slaney filterbank matrix (float32; match within ~1e-4 relative). Its params json documents the full 30 s chunk/stitching contract and the peak-picking recipe. A validated Swift implementation (Accelerate) is available at schism-audio/schism-dsp, tested against these exact vectors.

License

MIT, inherited from the source repository (code and released weights). Model by the Institute of Computational Perception, JKU Linz; Core ML conversion by schism-audio.