Best AI Voice Cloning in 2026: How to Clone Your Voice With AI

# Best AI Voice Cloning in 2026: I cloned my own voice seven ways and ranked them blind Source: [https://nexustrade.io/blog/best-ai-voice-cloning-2026-how-to-clone-your-voice-with-ai-20260824](https://nexustrade.io/blog/best-ai-voice-cloning-2026-how-to-clone-your-voice-with-ai-20260824) I write about markets and AI for a living\. Most of what I publish never becomes a video, because making one means setting up a camera and reading my own writing out loud, and I would rather spend those hours on the research\. I wanted to feed in the text and get narration in my voice\. A paid clone was the obvious first try and it was not good enough, so I trained my own\. The short answer ## Fine\-tune CosyVoice 3 or VibeVoice\. Those are the only two that captured both my voice and the way I speak\. **Press play on any row\.**Ranked on one question only: does it sound like me\. Several of these are excellent synthesis that happens to be someone else\. Every clip reads the identical script, and each zero\-shot model cloned from the same 20 seconds cut out of the recording the fine\-tunes trained on\. **This table merges three blind rounds run over two days:**an early screening round, the reader test below, and the ranking session near the end\. Each round covered part of the field, and the ranks here are my ordering across all three\. Read adjacent rows as ties\. Four answers, because there are four questions Closest clone of my voiceCosyVoice 3, fine\-tuned Just as good, brighterVibeVoice 1\.5B, fine\-tuned Best with no training at allChatterbox, zero\-shot Best paid option, if you would rather buy itElevenLabs Professional ★**Me, actually reading it** humanthe control Recorded on a phone, reading the same script\. 1**CosyVoice 3 0\.5B, fine\-tuned** open, freeyes, and closest of the two Fine\-tuned on 45 min\. The lower of my two winners and the one I judge closest to the recording\. The same model untrained is further down\. 1**VibeVoice 1\.5B, fine\-tuned** open, freeyes, brighter Fine\-tuned on the same 45 min\. Reads higher and more forward than I actually sound, which suits short\-form\. 3ElevenLabs Professional Voice Clone $22/moclose, not close enough The better of the two paid clones I made, from cleaner source audio\. This is the one ranked here; the weaker one opens the article\. 4IndexTTS\-2, zero\-shot open, freesounds like me, rhythm is off Newest model tested\. It got the voice from 20 seconds and the rhythm is off\. Cloned from the same clean reference the fine\-tunes trained on\. 4Qwen3\-TTS, zero\-shot open, freesounds like me, rhythm is off Same model as the fine\-tuned Qwen row below, without the training\. 6Qwen3\-TTS 1\.7B, fine\-tuned open, freegreat audio, not my voice Genuinely good synthesis: clean, natural, well paced\. Fine\-tuned on the same 45 minutes as the two winners, and the speaker it produces is not me\. 6Chatterbox, zero\-shot open, freeright rhythm, wrong voice The best\-sounding zero\-shot output of the four and no closer to my voice than the rest, which is why it sits here rather than higher\. 6CosyVoice 3, zero\-shot open, freeright rhythm, wrong voice The winning architecture with no training\. Compare it to row 1: the only difference is 11 minutes of fine\-tuning\. 8XTTS\-v2, fine\-tuned open, freelast, by a distance Fine\-tuned on the same 45 minutes, 6 epochs at batch 3 with untuned settings\. Two separate takes both landed bottom of a blind set\. A model being fine\-tunable does not make it a contender\. The finding that decides everything **The best zero\-shot voice cloning model is Chatterbox\.**It produced the most natural, usable audio of the four I ran with no training at all\. It ranks sixth in the table above because that table asks one question only, does it sound like me, and on that question IndexTTS\-2 and Qwen3\-TTS got closer while getting the rhythm wrong\. **Twenty seconds of reference audio buys you half a voice\.**IndexTTS\-2 and Qwen3\-TTS caught what I sound like but not how I talk: the words are right, the voice is close, the rhythm is not mine\. Chatterbox did the opposite, landing the pacing and missing the voice\. Only fine\-tuning gave me both at once, and not automatically: the fine\-tuned Qwen3\-TTS lower down is clean, natural audio that is somebody else\. In my first screening round, five files ranked blind, the two I said*sound like me*were the two fine\-tuned adapters in that set and the three I said did not were all zero\-shot\. That round predates the Qwen fine\-tune and did not contain it\. **CosyVoice 3 and Qwen3\-TTS each appear twice, trained and untrained**, on the same reference and script\. Whatever you hear between those pairs is what the training bought\. Two limits worth stating: the fine\-tunes had**45 minutes**and the zero\-shot models had**20 seconds**, so I measured 20s zero\-shot against 45min fine\-tuning, not zero\-shot as a technique\. The ElevenLabs row also used cleaner source audio than my corpus, so it is not matched on input either\. Take the test ## Try it yourself: one of these five is a real human **This is a reader exercise, not one of my ranking sessions\.**I assembled it from the same clips under the same conditions so you can try the thing I did: identical script, loudness matched, labels hidden\. The session that actually produced the rankings is near the end of the article, and it used a different five files\. The setup ## How to clone your voice, in one pass 1. **Record 30 to 45 minutes in one session\.**One room, one mic, one sitting\. Mine was an iPhone with no interface and no treated room\. Do not pool old clips from different days: that was the single biggest quality jump in my testing\. 2. **Convert to 24 kHz mono and skip enhancement\.**`ffmpeg \-i src\.mov \-ar 24000 \-ac 1 \-c:a pcm\_s16le audio\.wav`\. I tested an enhanced version of the same recording and it scored worse\. 3. **Cut into 4 to 12 second clips**on sentence boundaries, then check the clip manifest against your recording length\. My first chunker dropped over\-long segments instead of splitting them and silently lost 27 of 52 minutes\. 4. **Transcribe with`openai\-whisper`, model`large\-v3`,**running on the same GPU\. Each training row is a \(text, audio\) pair\. 5. **Rent one 24 GB GPU and prove it is real**before installing anything:`python \-c "import torch; assert torch\.cuda\.is\_available\(\)"`\. A silent fall back to CPU will hide a dead machine for a whole run\. 6. **Read the project's Dockerfile\.**It is the only authoritative record of the environment\. For CosyVoice the load\-bearing details are Python 3\.10 and`pynini`from conda\-forge\. 7. **Train, saving a checkpoint per epoch\.**About 22 seconds per epoch for 45 minutes of audio\. Do not assume the last checkpoint is the best one: mine peaked at epoch 4 of 31\. 8. **Render three takes per checkpoint, then transcribe each one and diff it against your script\.**Generation is stochastic and most of these models take no seed, so takes differ in how much of the script they actually speak\. I measured 25%, 54% and 90% word coverage across three renders of one input\. Keep the take with the highest coverage and re\-render anything below about 90%\. 9. **Rank blind\.**Normalise loudness, pad to one duration, hide the labels until after you commit\. Copy\-paste plan for an AI coding agentPaste this into Claude Code, Cursor, or any agent with shell access\. It encodes every failure that cost me time, including the ones that fail silently\. VOICE CLONE PLAN ``` Clone my voice with an open-source TTS model. Work through these steps in order and verify each one before moving on. Tell me when something fails; do not silently retry. === 0. SAFETY: THIS PROMPT CONTAINS NO SECRETS. DO NOT ADD ANY. === Do not paste an API key, token, or password into this prompt or into any chat window. Everything below authenticates interactively or through a file the agent never prints. Agent: never echo, log, or repeat back the contents of a credential file. If you need to confirm a token loaded, print its LENGTH, not its value. === 0b. TOOLING YOU NEED FIRST === RunPod (rents the GPU). Connect the MCP server so the agent can create and destroy pods: claude mcp add --transport http runpod -s user https://mcp.getrunpod.io/ This uses a browser OAuth flow, so no API key is stored on disk and nothing sensitive passes through this prompt. Then add an ACCOUNT-LEVEL SSH key at runpod.io/console/user/settings. The SSH proxy authenticates against that key, not the pod's PUBLIC_KEY, and without it every connection fails with "Permission denied (publickey)". Hugging Face (hosts the base models, your corpus, and your trained weights). Create a token at huggingface.co/settings/tokens. Prefer a FINE-GRAINED token scoped to only the repos you will create, with write access. A global write token can delete every repo you own, and an agent will be handling it. Log in INTERACTIVELY so the token never lands in shell history or in a process argument list: huggingface-cli login To place it on a pod, read it from stdin and write it to a file. Never pass a token as a command-line flag: read -rs HF_TOKEN && printf '%s' "$HF_TOKEN" > /root/.hf_token && unset HF_TOKEN chmod 600 /root/.hf_token === 1. RECORD === 30-45 minutes in ONE session. One room, one mic, one sitting. A phone is fine. Do not pool clips from different days: mixed sessions teach the model the room differences along with the voice. Scraped audio does not scale: 14.5 min of clips pulled from old videos tied 2.5 min of the same. One clean session of 30-45 min is what actually worked. No noise reduction, no enhancement. Enhanced audio scored worse. === 2. BUILD THE CORPUS === ffmpeg -i SOURCE.mov -ar 24000 -ac 1 -c:a pcm_s16le audio_24k.wav Chunk into 4-12s clips: split on . ! ?, merge forward until each clip is >= 4s, force-split anything over 12s at a comma or a gap > 0.35s. NEVER discard a segment for being too long. Print a manifest and check it against the source length: clips=360 total_min=45.36 min=4.00 mean=7.56 max=12.00 STOP if total_min is not within ~10% of what I recorded. An early version of my chunker dropped over-long segments instead of splitting them and silently lost 27 of 52 minutes with no error. Cut restarts, coughs, off-mic lines and dead air. A bad clip hurts more than a missing one. === 3. TRANSCRIBE === openai-whisper, model large-v3, on the GPU. Write train.jsonl with one row per clip: {"text": ..., "audio": ...} Correcting the transcripts afterwards is OPTIONAL. I tested it and it did not improve output quality. Skip unless I ask. === 4. PIN THE CORPUS TO HUGGING FACE, BEFORE ANY TRAINING === huggingface-cli repo create my-voice-corpus --type dataset --private # upload the clips + train.jsonl This is not bookkeeping. The corpus must be ONE artifact that every training run fetches, or you will end up comparing models trained on different data without realising it. Every training script should fetch the pin and assert the clip count before starting. === 5. START A POD === Use the RunPod MCP tools. Ask for a 24GB GPU (RTX 4090 is enough for a 0.5B model). Community 4090 is about $0.34/hr, secure about $0.74/hr. CUDA MUST BE >= 12.8. create-pod returns cudaVersion: if it is 12.2 or 12.4, DELETE the pod and create another. Those hosts crash-loop the container while the API still reports RUNNING. Each rejected pod costs under a cent. Then prove the GPU is real before installing anything: python -c "import torch; \ print('GPUCHECK', torch.cuda.is_available(), \ (torch.randn(999,999,device='cuda') @ torch.randn(999,999,device='cuda')).sum().item())" If that fails, destroy the pod and get another. Never let training fall back to CPU: a silent CPU fallback hid a dead GPU from me for an entire run. === 6. BUILD THE ENVIRONMENT === FIND THE PROJECT'S docker/Dockerfile AND FOLLOW IT. It is the only authoritative record of the environment. Guessing at it cost me many hours. For CosyVoice 3 specifically: # the image may not ship conda at all; install miniconda if `conda` is missing conda create -y -n cosyvoice python=3.10 --override-channels -c conda-forge conda install -y -n cosyvoice --override-channels -c conda-forge pynini==2.1.5 pip install "setuptools<81" wheel pip install torch==2.3.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121 pip install -r requirements.txt # hold out openai-whisper pip install openai-whisper --no-build-isolation pip install huggingface_hub hf_transfer Four traps, all of which fail SILENTLY or misleadingly: - `conda activate` does nothing in a non-interactive shell. Use absolute paths: /opt/conda/envs/NAME/bin/python. Otherwise your verification passes against the WRONG interpreter. - Anaconda default channels now need a ToS accept that fails unattended. Use --override-channels -c conda-forge. - openai-whisper fails to build inside isolation and aborts the ENTIRE requirements install, so nothing else lands. - Some images export HF_HUB_ENABLE_HF_TRANSFER=1 without shipping hf_transfer, so every Hugging Face download raises until you install it. Assert every import before training: /opt/conda/envs/cosyvoice/bin/python -c \ "import torch,torchaudio,hyperpyyaml,whisper,pkg_resources; print('DEPS_OK')" === 7. TRAIN === Save a checkpoint EVERY epoch. Expect ~22 seconds per epoch for a 45-minute corpus on a 4090, so roughly 11 minutes total. Checkpoints are ~2GB each, so prune to a sparse ladder as you go or a 150GB disk fills at about epoch 36. Do NOT assume the last checkpoint is the best. Audition early ones. Mine peaked at epoch 4 of 31. Epoch count scales INVERSELY with corpus size: one epoch on 45 min is ~3x the gradient steps of one epoch on 15 min, so a bigger corpus peaks at a LOWER epoch number. === 8. PUSH EVERY CHECKPOINT TO HUGGING FACE === huggingface-cli repo create my-voice-model --private # upload each kept checkpoint, winner first NON-NEGOTIABLE, and do it BEFORE destroying the pod. Verify by reading the file sizes back off Hugging Face. A stopped RunPod pod is NOT a saved pod. /workspace is container overlay storage, not a network volume. I have seen both failure modes in one hour: a stopped pod that could not restart ("not enough free GPUs on the host"), and one that restarted on a different host with /workspace completely empty. === 9. RENDER, 3+ TAKES PER CHECKPOINT === Generation is stochastic and most of these models take NO seed, so one take is a lottery draw rather than a measurement. Render at least 3. Do NOT pick by duration: in both cases I measured, the shortest takes were the ones missing words, because truncation shortens a take. Transcribe each render, diff it against the input text, and keep the highest word coverage. Treat a wide spread in coverage as checkpoint instability rather than as bad quality. If the inference call returns a GENERATOR, consume ALL of it and concatenate. CosyVoice yields one segment per sentence group; taking only the first silently truncated a 29.5s clip to 19.7s for me, with no error. Verify completeness: transcribe each render and diff it against the input text. Expect >90% word coverage. === 10. BLIND TEST === Normalise every candidate to -16 LUFS, pad to one duration, and force an identical byte size. Loudness, length and file size all leak which file is which. Shuffle the labels, seal the mapping, and do not reveal it until after I have ranked. Keep sets to 5-6 files. Balance the arms: do not put 4 checkpoints of one model against 1 of another. One bad render sinks a lone entry. === 11. TEAR DOWN === Once checkpoints are pushed AND verified by read-back, TERMINATE the pod. Do not stop it. Stopping still bills for storage and may lose the disk anyway. === REPORT BACK === Manifest numbers, per-epoch loss, the checkpoint list, take durations and word coverage per render. Name what failed. ``` What mattered ## Two levers moved the result\. The rest were a wash or worse\. **Fine\-tuning instead of zero\-shot**decisive **Recording one clean session**large effect Scraping more clips from old videosa wash More training stepsa wash Correcting the transcriptsa wash Enhancing the audio before trainingscored worse ### One clean recording beat everything I scraped I started by scraping audio out of videos I had already published, because it was free and it was there\. My first clone came from**2 minutes 33 seconds**of it and was good enough to be my keeper for weeks\. Here it is: 2\.55 minutes of training data Built from 13 short takes, 2m33s total Not bad\. Recognisably in the right territory, and it was my keeper for weeks before anything beat it\. VibeVoice\-1\.5B · 22 clips · peaked at 200 steps Scraping more of it stopped helping quickly\. I built a second corpus of**14 minutes 28 seconds**from the same videos and it tied with the 2\.5 minute version in a blind ranking\. Then I sat down and recorded**45 minutes in one session**, one room, one microphone, one sitting\. That corpus produced both of the winners at the top of this page\. **Record properly instead of scraping\.**Clips pulled from videos shot across different rooms and days carry all those rooms into the model\. Forty five minutes recorded on purpose is a couple of hours of your time and it is the single biggest quality jump I found\. ### The peak arrives early With one model I extended a run from 3200 to 6400 steps because the curve looked like it was still climbing\. The first 3200 were the whole gain\. My CosyVoice keeper peaked even earlier, at epoch 4 of 31, so plan on auditioning early checkpoints rather than training longer\. Blind test 2 ## The ranking that picked the winners A different five files from the test at the top of this page: both ElevenLabs voices, both of my fine\-tunes, and the real recording\. This is the set that decided the ranking, and it includes both paid clones so the comparison is against the service at its best and its worst\. FileWhat it wasWhat I said, before the revealAElevenLabs, scavenged source"bad"BElevenLabs, cleaner source"good"C**VibeVoice, fine\-tuned**"good, maybe slightly better\. honestly best"D**Me, the real recording**"so good that I think it's probably legit me"ECosyVoice, fine\-tuned"good, maybe slightly better than B" Read C and D together\.**I called the fine\-tuned model "honestly best" and, separately, flagged the real recording as "probably legit me\."**I did identify the human\. I also rated a synthetic clone at the top of the set while doing it\. Both paid clones landed below both fine\-tunes\. That is the practical bar for narration, and 45 minutes of phone audio cleared it\. Cost ## What each option actually costs OptionCostLimitYou own it?ElevenLabs Creator$22/mo forever100k credits, ~100 minnoElevenLabs Pro$99/mo forever500k creditsno**Fine\-tune your own**under $1 per run ~$5 to a first clone plus GPU time to rendernoneyes, the weights The whole project, including every failed experiment and dozens of training runs, came to about**$17 of GPU time**\. That is less than one month of the cheapest tier that supports professional voice cloning\. I did not provision or run any of this infrastructure by hand\. I described what I wanted to an AI coding agent, it stood up the environment, rented the GPU, ran the training, and shut the machine down when it was done\. The plan above is the instruction set, and everything it needs to get right is already in it\. **The cost is a few dollars of rented GPU and the time it takes you to record\.** One honest asterisk on that comparison: the $22 a month buys inference too, and self\-hosting does not\. Training is a one\-off, but every minute of narration afterwards needs a GPU\. At my volume that is pennies of pod time per batch\. If you are generating a hundred minutes a month you should price it against the tier that covers it, not against the training run\. Generalisation ## Does it hold up on text it has never seen? A clone that only works on the paragraph you tested is not useful\. This is 124 words the model never saw in training, including finance jargon it has no special reason to pronounce correctly\. Unseen text · 124 words Generalisation test Written after the training session, so none of this text appears in the corpus\. Listen for "Sortino", "RSI", and "SPY"\. CosyVoice 3 epoch 4 · 46s · word coverage 91\-96% verified by transcription Word coverage across three renders came back at 96%, 91% and 93% when transcribed and diffed against the source, with the residual differences being transcription artifacts rather than missing speech\. Before you ship it ## Licence, consent, and the things that decide this in practice ### Check the licence before you pick a winner "You own the weights" in the cost table is doing more work than it looks\. You own the*adapter*you trained, on top of a base model whose terms you did not write\. Checked on Hugging Face, 24 August 2026: Base modelLicenceFunAudioLLM/Fun\-CosyVoice3\-0\.5B\-2512Apache 2\.0vibevoice/VibeVoice\-1\.5BMITIndexTeam/IndexTTS\-2custom: "bilibili Model Use License Agreement"Qwen/Qwen3\-TTS\-12Hz\-1\.7B\-BaseApache 2\.0ResembleAI/chatterboxMITcoqui/XTTS\-v2Coqui Public Model License 1\.0\.0, non\-commercial Two of these want reading rather than glancing\.**IndexTTS\-2**ships a`LICENSE\.txt`headed "bilibili Model Use License Agreement", with its own definitions of derivative works and acceptance\-on\-use terms\.**XTTS\-v2**is Coqui Public Model License 1\.0\.0, which permits non\-commercial use only, and that restriction covers the model's*outputs*as well as its weights\. The model that came last is also the one you are least free to publish from\. One provenance note: I trained on`vibevoice/VibeVoice\-1\.5B`, a community copy, and Microsoft's first\-party`microsoft/VibeVoice\-1\.5B`is also live and also MIT\.**Prefer the first\-party repo, and read the licence on the specific checkpoint you pull**, since a mirror can carry different terms than its source and terms change between releases\. ### Clone your own voice, and say that it is synthetic Everything here is my voice, my recording, my consent\. Cloning someone else's without permission is a legal problem in a growing number of places: Tennessee's ELVIS Act covers voice explicitly, right\-of\-publicity law applies in many US states, and the EU AI Act carries transparency obligations for synthetic media\. None of that is legal advice, and all of it is worth ten minutes before you point this at a voice that is not yours\.**Label synthetic narration as synthetic\.** ### Where this test stops Seven systems is what I ran, and the field is bigger\.**F5\-TTS, Fish Speech, Higgs Audio, and Sesame CSM went untried\.**F5\-TTS is the one I would run next, because it fine\-tunes and therefore competes in the same category as the winners rather than the zero\-shot group\. Length is the other open question: my generalisation test runs 124 words, the use case is ten\-minute narration, and long\-form drift is where these models tend to struggle\. ### Versions Text to speech moves monthly, so these results are pinned to what I ran in**August 2026**:`FunAudioLLM/Fun\-CosyVoice3\-0\.5B\-2512`,`vibevoice/VibeVoice\-1\.5B`,`IndexTeam/IndexTTS\-2`,`Qwen/Qwen3\-TTS\-12Hz\-1\.7B\-Base`,`coqui/XTTS\-v2`, and`chatterbox\-tts`from PyPI\. Assume a model released after this article behaves differently, and re\-run the test rather than trusting the table\. ## What I would tell someone starting today Fine\-tune\. Every zero\-shot model I tried got half of it and missed the other half, and it split evenly: two landed my voice and missed the rhythm, two landed the rhythm and missed the voice\. None got both\. Record one clean session before you touch a model\. Forty five minutes from a phone in a quiet room beat everything I scavenged\. Then rank blind\. Every time I evaluated with the labels visible, I confirmed what I already believed\. Why I built this ### So I can narrate the research instead of filming it I run**NexusTrade**, an AI\-powered trading research platform, and I publish what it finds\. The writing is the work; recording it was the tax\. Now the articles narrate themselves and I spend the time on the analysis\. The platform is built the same way this test was: run the experiment, rank it blind, publish the number even when it says you were wrong\. [Explore NexusTrade](https://nexustrade.io/)