Kokoro TTS Can Now Clone Any Voice With Minimal Extra VRAM
A developer successfully combined the Kokoro text-to-speech model with OpenVoice's ToneColorConverter to enable arbitrary voice cloning. This integration was added as an 'any-tts' recipe in audio.cpp, requiring only about 600 MB of additional peak VRAM. This method offers a lightweight way to equip existing text-to-speech models with voice cloning capabilities without requiring expensive retraining. It lowers the hardware barrier for running advanced, customizable audio synthesis locally. The integration incurs only a ~10% slowdown on CUDA and operates at about 0.5x the original speed on a CPU, which still achieves 2x real-time performance. The base TTS model determines pronunciation and prosody, while the ToneColorConverter applies the target speaker's timbre.
## BACKGROUND
Kokoro is a lightweight, open-weight text-to-speech model with 82 million parameters known for generating natural-sounding speech efficiently. OpenVoice is a voice cloning framework featuring a standalone ToneColorConverter component responsible for transforming the timbre of source audio to match a target speaker.