CallEnhancer — Call-Centre / Telephony Speech Restoration

Restore Multi-lingual narrowband, codec'd, noisy call-centre / telephony speech (e.g. 8 kHz G.711/GSM phone audio) to clean 48 kHz.

Benchmark — CER

Character Error Rate (Whisper large-v3, lower is better) on a held-out private call-centre test set. CallEnhancer restores telephony-degraded speech so an ASR reads it far more accurately than the raw input or general-purpose speech enhancers.

model	CER % ↓
degraded (original)	71.48
voicefixer	58.27
resemble-enhance	54.96
sidon-v0.1	41.06
CallEnhancer-small (this repo, open source)	38.91
CallEnhancer-base	19.01

This repository hosts CallEnhancer-small (open source); CallEnhancer-base is the larger model.

Quick start — infer from the HF checkpoint

pip install torch torchaudio "transformers>=4.56" "descript-audio-codec>=1.0.0" soundfile "huggingface_hub[cli]"

# pull the CLI + the two slim checkpoints from the Hub
hf auth login   # private repo: log in first (or export HF_TOKEN=hf_...)
hf download Scicom-intl/CallEnhancer \
    infer_callcentre.py fe_callcentre/fe_adapter_full.pt \
    decoder_callcentre/decoder_only.pt --local-dir CallEnhancer

cd CallEnhancer && python infer_callcentre.py \
    --input your_call.wav --out-dir out \
    --fe-adapter fe_callcentre/fe_adapter_full.pt \
    --decoder decoder_callcentre/decoder_only.pt \
    --chunk 0 --device cuda   # --chunk 0 = NO chunking (default single pass); --device cpu if no GPU
# -> out/your_call_restored48k.wav (clean 48 kHz)  +  out/your_call_orig48k.wav (A/B)

Prefer Python (load weights from the Hub with hf_hub_download)? See Python below.

Status: CallEnhancer-small is fully trained and restores real 8 kHz call-centre audio well.

Files

Use the current-run checkpoints under fe_callcentre/ and decoder_callcentre/:

path	role	size
fe_callcentre/fe_adapter_full.pt	FE adapter (inference) — 144 tensors: 96 LoRA + 48 trained output_dense biases	~63 MB
decoder_callcentre/decoder_only.pt	decoder (inference) — 188M DAC decoder	~0.75 GB
fe_callcentre/last.pt, decoder_callcentre/last.pt	raw checkpoints (resume training)	~2.5 / 2.8 GB
infer_callcentre.py	inference CLI (below)	—

For inference you only need the two slim files + infer_callcentre.py. (Root-level fe_adapter_full.pt / decoder_only.pt are from an earlier run and are superseded.)

End-to-end example (straight from HuggingFace)

pip install torch torchaudio "transformers>=4.56" "descript-audio-codec>=1.0.0" soundfile "huggingface_hub[cli]"

# pull the CLI + the two slim checkpoints, straight from this repo
hf auth login   # private repo: log in first (or export HF_TOKEN=hf_...)
hf download Scicom-intl/CallEnhancer \
    infer_callcentre.py \
    fe_callcentre/fe_adapter_full.pt \
    decoder_callcentre/decoder_only.pt \
    --local-dir CallEnhancer
cd CallEnhancer

# restore your audio end-to-end
python infer_callcentre.py \
    --input your_call.wav \
    --out-dir out \
    --fe-adapter fe_callcentre/fe_adapter_full.pt \
    --decoder   decoder_callcentre/decoder_only.pt \
    --chunk 0 --device cuda   # --chunk 0 = NO chunking (single straight pass, default); --device cpu if no GPU

Outputs:

• out/your_call_restored48k.wav — the restored clean 48 kHz speech.
• out/your_call_orig48k.wav — the input, naively upsampled to 48 kHz (no model), for an A/B listen.

--input accepts a file or a directory (.wav/.flac/.mp3/.ogg/.opus/.m4a). Stereo (e.g. agent/customer on separate channels) is restored per channel and recombined.

Python (pull weights from the Hub)

import numpy as np, soundfile as sf, torch, torchaudio
from huggingface_hub import hf_hub_download
from transformers import AutoFeatureExtractor, Wav2Vec2BertModel
import dac

REPO, SSL, FE_SR, SR_OUT = "Scicom-intl/CallEnhancer", "facebook/w2v-bert-2.0", 16000, 48000
dev = torch.device("cuda" if torch.cuda.is_available() else "cpu")

ck = torch.load(hf_hub_download(REPO, "fe_callcentre/fe_adapter_full.pt"), map_location="cpu")
ad, scale = ck["adapter"], ck["lora_alpha"] / ck["r"]
fe = Wav2Vec2BertModel.from_pretrained(SSL, num_hidden_layers=ck.get("layers", 24), layerdrop=0.0)
sd = fe.state_dict()                                     # merge LoRA -> base (no peft needed)
for p in sorted({k[:-len(".lora_A.default.weight")] for k in ad if k.endswith(".lora_A.default.weight")}):
    sd[p+".weight"] = sd[p+".weight"].float() + scale * (ad[p+".lora_B.default.weight"].float() @ ad[p+".lora_A.default.weight"].float())
    if p+".base_layer.bias" in ad: sd[p+".bias"] = ad[p+".base_layer.bias"].to(sd[p+".bias"].dtype)
fe.load_state_dict(sd); fe.to(dev).eval()

dck = torch.load(hf_hub_download(REPO, "decoder_callcentre/decoder_only.pt"), map_location="cpu")
dec = dac.model.dac.Decoder(input_channel=1024, channels=dck.get("dec_channels", 3072), rates=[8,5,4,3,2])
dec.load_state_dict(dck["decoder"]); dec.to(dev).eval()
proc = AutoFeatureExtractor.from_pretrained(SSL)

@torch.no_grad()
def restore(path, out="restored48k.wav"):                # single straight pass
    x, sr = sf.read(path, dtype="float32"); x = x.mean(1) if x.ndim > 1 else x
    if sr != FE_SR: x = torchaudio.functional.resample(torch.from_numpy(x)[None], sr, FE_SR)[0].numpy()
    x = x / (np.abs(x).max() + 1e-9) * 0.95
    feats = {k: v.to(dev) for k, v in proc(x, sampling_rate=FE_SR, return_tensors="pt").items()}
    y = dec(fe(**feats).last_hidden_state.transpose(1, 2)).squeeze().float().cpu().numpy()
    sf.write(out, y / (np.abs(y).max() + 1e-9) * 0.97, SR_OUT); print("wrote", out)

restore("your_call.wav")   # <-- your own telephony/call-centre audio