VoxCPM2-Ghana: Multilingual Ghanaian IPA Text-to-Speech

A multilingual text-to-speech model for 42 Ghanaian languages trained on IPA phoneme inputs. Built by finetuning VoxCPM2 on Ghanaian speech with IPA transcriptions.

Try it live: HuggingFace Space

Author: Mich-Seth Owusu (@michsethowusu)

Demo Samples

The samples below cover a subset of the supported languages. The model supports IPA-based generation for all 42 Ghanaian languages plus English — use any IPA text from the supported languages with the inference script.

Listen to generated samples:

Language	Text	Audio
Asante Twi	Nnipakuo bi a wɔɔyɛ adwuma ɛwɔ bɔɔhol bi ɛho	ref_00
Akuapem Twi	Dɛɛbi dɛɛbi me pɛsɛ me tɔ	ref_00
Fante	MMA MMA KWAN MA BIRIBIARA NNSƐƐ W'ENYIGYE	ref_00
Ewe	Mevɔ̃na ɣesiaɣi be ame aɖe akpɔ wo le nugoe me	ref_00
Dagbani	Bi zɔri mi ni be chɛŋ shukuru be konko	ref_00
Dagaare	Bibiiri bayɔpoi la zeŋ, ba mine leɛ la ba nime kyaare taa	ref_00
Dangme	Suɔmi Yeɔ Bua Wɔ Nɛ Wa Daa Ninyɛ Nya	ref_00
Ninkare/Frafra	To wan eŋɛ la wani le tara pupeelum?	ref_00
Nzema	Bɛde ɔ bo kɛ nlanwonvoninli ɛnleanlɛ kola ka	ref_00
Sehwi	Sɛ Ɛkyɛ Nikye A, Ɔkɔmaa Wɔnye Kɔgye Paa	ref_00
Code-switch	Sɛ wopɛ a, yɛbɛtumi anante, it's not that far	ref_00
English	Government sitting on US$200m Akufo-Addo left	ref_00

More samples (without reference audio) are in the samples/ directory.

Model Details

Architecture: VoxCPM2 (LLaMA-like LM + AudioVAE decoder)
Parameters: ~5B (LM) + AudioVAE
Base model: VoxCPM2 (Microsoft)
Finetuning: Full model finetuning (not LoRA) on 40k steps
Sample rate: 48 kHz output (16 kHz latent encoding)
Language support: 42 Ghanaian languages + English + code-switch
Input format: IPA phoneme sequences (space-separated)
Tokenizer: SentencePiece (73,440 vocab, 55,305 IPA/multichar units)

Training Details

Hyperparameters

Parameter	Value
Optimizer	AdamW
Learning rate	1e-5
Weight decay	0.01
Warmup steps	1,000
Batch size	4
Gradient accumulation	8 (effective batch = 32)
Max gradient norm	1.0
Max steps	40,000
Training epochs	~3.04
Max batch tokens	8,192

Training Data

Metric	Value
Total audio	~747 hours
Total clips	~369,347
Languages	42 Ghanaian + English
Max hours per language	20h (balanced)
English hours	200h
Dev clips per language	40
Audio encoding	AudioVAE latents (fp16, [T_f, 64])
Text format	IPA phonemes (space-separated)

Source corpora:

Bible speech recordings (JW / Waxal / UNICEF) — majority of data
Ghana News corpus (English)
Finance domain (Akuapem Twi)
Code-switch datasets (Twi-English)

Training Curves

Step	Train Diff Loss	Train Stop Loss	Val Total Loss	Val Diff Loss
0	1.033	0.062	0.941	0.865
20,000	~0.90	~0.005	0.838	0.821
39,999	0.665	0.001	0.813	0.804

Training converged by step 40,000 with decreasing loss across all metrics.

G2P Tools & Phonemisation

All text was phonemised to IPA before training. The following G2P tools were used:

Ghanaian Languages (42 languages)

Tool: africa-g2p (bundled in training scripts)
Convention: Space-separated IPA with punctuation kept as standalone tokens
Coverage: 42 Ghanaian languages with language-specific grapheme-to-phoneme rules

English

Tool: GhanaNLP/ghana-g2p — GhanaEnglishG2P
Note: English uses Ghana-accented English phoneme conventions, not RP/GA. This is intentional: the model was trained on Ghanaian-accented English speech, so the IPA must match that accent.

ASR-Assisted Transcription

Tool: Custom Ghanaian phoneme ASR (CTC, 172 phoneme vocab) finetuned to English IPA
Purpose: Transcribe audio that lacked IPA labels, then relabel training latents
Repo: ghanaopendata/ghana-speech-ASR-IPA

Datasets

Training Latents

Repo: ghananlpcommunity/voxcpm2-ghana-speech-ipa-latents
Configs: 42 language configs (balanced to 20h each)
Columns: feat (AudioVAE latent fp16), text (IPA), text_ids (tokenized), dataset_id, split

Audio Companion

Repo: ghananlpcommunity/ghana-speech-ipa-audio
Purpose: Audio clips + IPA text for listening/verification

ASR Transcriptions

Repo: ghanaopendata/ghana-speech-ASR-IPA
Coverage: 434 shards, all 42 languages transcribed with IPA ASR

Inference

from voxcpm import VoxCPM
import numpy as np, soundfile as sf

# Load model
model = VoxCPM(
    voxcpm_model_path="ghananlpcommunity/VoxCPM2-Ghana",
    optimize=False,
    device="cuda",
)

# Generate from IPA text
ipa_text = "n n ɪ pʰ a kʰ uo b ɪ a w ɔ ɔ j ɛ a d͡ʒ u m a"
wav = model.generate(
    text=ipa_text,
    cfg_value=2.0,
    inference_timesteps=10,
    normalize=False,  # CRITICAL: must be False for IPA input
)

sf.write("output.wav", np.array(wav).squeeze(), 48000)

With Reference Audio (Voice Cloning)

wav = model.generate(
    text=ipa_text,
    reference_wav_path="path/to/reference.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    normalize=False,
)

Important Notes

normalize=False is required. The text normaliser is designed for orthographic text and will corrupt IPA strings.
The model outputs at 48 kHz. Writing at the wrong sample rate will pitch-shift the output.
Voice cloning via reference_wav_path is supported (VoxCPM2 only). Use a clean 3-8 second reference clip.

Scripts

The training and data preparation scripts are included in the training/ directory:

Script	Purpose
`voxcpm2_ghana_latents.yaml`	Training config
`train_voxcpm_finetune.py`	Main training script (from VoxCPM)
`precompute_voxcpm2_ipa_latents.py`	Precompute AudioVAE latents from audio + IPA
`precompute_voxcpm_latents.py`	Precompute latents (V1 format)
`g2p_english_ipa.py`	Phonemise English with ghana-english-g2p
`build_ipa_from_sidecar.py`	Build IPA labels from sidecar files
`build_ipa_audio_datasets.py`	Build audio companion datasets
`generate_samples.py`	Generate TTS samples for evaluation
`build_eval_prompts.py`	Build evaluation prompt set
`relabel_latents_asr_ipa.py`	Relabel latent dataset with ASR IPA
`transcribe_ghana_gpu_asr.py`	GPU ASR transcription

Limitations

IPA input only. This model takes IPA phoneme sequences, not raw text. You need a G2P tool for each supported language to convert text to IPA before synthesis.
English phoneme convention. English uses a Ghana-accented IPA convention (ghana-g2p), not standard RP or GA. The English phoneme ASR was finetuned on g2p targets so output matches this convention.
Bible speech domain bias. The training data is predominantly read Bible speech (JW/Waxal). The model performs best on formal read speech and may struggle with casual/conversational prosody.
Background noise in English. Some English training clips had residual background noise from the source recordings. The English subset was not fully cleaned.
Limited training epochs. The model was trained for ~3 epochs (40k steps). Further finetuning may improve quality, especially for underrepresented languages.
Uneven language coverage. While data was balanced to 20h per language, some languages have more diverse speakers/domains than others. Languages like Dagbani and Ewe have richer UNICEF data; smaller languages rely primarily on Bible speech.
No explicit speaker modelling. The model does not condition on speaker identity. Each generation may produce a slightly different voice.
48 kHz output. The model outputs 48 kHz audio. For downstream use at 16 kHz, resample after generation.
Single GPU inference only. The model requires ~10 GB VRAM for inference. Tested on NVIDIA H200.
Text normaliser disabled. normalize=False is mandatory for IPA input. Using the normaliser will corrupt phoneme sequences.

Citation

If you use this model, please cite:

@misc{ghananlp2025voxcpm2ghana,
  title={VoxCPM2-Ghana: Multilingual Ghanaian IPA Text-to-Speech},
  author={GhanaNLP},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/ghananlpcommunity/VoxCPM2-Ghana}
}

License

CC-BY-NC 4.0 (non-commercial). See LICENSE for details.

This license applies due to the non-commercial nature of the training data sources (Bible/JW speech recordings, UNICEF educational recordings). The upstream VoxCPM2 model uses Apache 2.0.

Downloads last month: 73

Safetensors

Model size

2B params

Tensor type

F32

ghananlpcommunity
/

VoxCPM2-Ghana