laion/voiceclap-large-v2-genuineness

VoiceCLAP-large-v2 - Speech Genuineness Predictor

Predicts a 0-6 genuineness score for a speech clip: how much it sounds like a real, lived-in spoken moment (6) versus a rehearsed or synthetic read (0). Genuineness is about believability and lived-in delivery - natural timing, breath, micro-imperfections, and emotional grounding - not about audio fidelity.

This repository is fully standalone. It bundles the original frozen laion/voiceclap-large-v2 audio embedder under voiceclap_large_v2/ plus a small trained MLP head, so inference needs no network access.

What it predicts

A single continuous score in [0, 6]:

• 0-2 - reads as rehearsed, flat, or synthetic
• 3-4 - plausibly natural, some life to it
• 5-6 - sounds like a genuine, unscripted, lived-in moment

How it was trained

• ~10k speech clips drawn from a mix of various TTS systems and the Emolia expressive-speech collection, spanning a wide range of naturalness.
• Each clip labelled 0-6 for genuineness by Gemini-3.1-Pro.
• Every clip is encoded once with the frozen VoiceCLAP-large-v2 embedder (3584-d). A small MLP head - Linear(3584,16) -> GELU -> Dropout(0.2) -> Linear(16,1) - is trained on top with Huber loss (delta 1.5, Adam, standardized inputs). The VoiceCLAP backbone is never fine-tuned.
• Two heads ship: full (default, trained on the full label distribution) and balanced (retrained on a class-balanced subset - flatter per-bucket error, slightly higher overall MAE).

Validation metrics (this model)

Held-out val set = 140 clips (20 per genuineness value 0-6, stratified, seed 1234). Full head:

Metric	Value
MAE	0.81
Pearson r	0.83
RMSE	1.17

See val_predictions.html for every val clip with an audio player, ground-truth vs predicted score (sorted by prediction), a pred-vs-GT scatter, and per-bucket errors.

Usage

from genuineness_scorer import GenuinenessScorer

scorer = GenuinenessScorer(pkg_dir=".", device="cuda")   # loads bundled VoiceCLAP + head locally
print(scorer.score("clip.wav"))                          # -> float in [0, 6]
print(scorer.score_batch(["a.wav", "b.wav"]))            # -> [float, float]

# class-balanced head instead of the default full-data head:
scorer_bal = GenuinenessScorer(pkg_dir=".", model="balanced", device="cuda")

Or from the shell:

pip install -r requirements.txt
python example.py clip.wav          # prints genuineness (0-6)

The embedder loads from the local voiceclap_large_v2/ folder with trust_remote_code=True; set HF_HUB_OFFLINE=1 to guarantee no network access at inference time.

Files

• voiceclap_large_v2/ - bundled frozen VoiceCLAP-large-v2 embedder (~17 GB)
• genuineness_head.pt - full-data MLP head (default)
• genuineness_head_balanced.pt - class-balanced MLP head
• genuineness_scorer.py - GenuinenessScorer inference class
• example.py, requirements.txt, val_predictions.html

Limitations

• Label skew: training labels lean low (most clips fall in 0-2), so the model is most reliable there and noisier on rare high-genuineness clips (per-bucket MAE grows toward g6).
• Genuineness is a subjective, model-labelled construct; treat scores as a soft ranking signal, not ground truth.
• Trained on the languages/domains present in the source mix; out-of-domain audio (heavy noise, music, non-speech) is out of scope.

License

CC-BY-4.0. The bundled VoiceCLAP-large-v2 weights retain their original license from laion/voiceclap-large-v2.