Ming-3.0
Collection
5 items • Updated • 8
Ming-3.0-tts-0.5B
Dense 0.5B multilingual speech synthesis with paralinguistic and free-form control.
GitHub |
Blog & Demos |
ModelScope
This repository contains Ming-3.0-tts-0.5B, the dense Ming-3.0-tts checkpoint built on Qwen2.5-0.5B. The checkpoint includes the backbone, DiT head, audio tokenizer, speaker encoder and text tokenizer.
Inference requires the Ming-3.0-tts GitHub repository. This Hugging Face repository is for checkpoint distribution and does not provide a standalone inference setup. Clone the GitHub repository and follow its README for the current dependencies, inference code and usage instructions. The CLI scripts and Python APIs referenced below are provided by the GitHub repository.
Model Details
| Item | Value |
|---|---|
| Architecture | Dense, Qwen2.5-0.5B backbone |
| Parameters | 0.5B |
| Languages | Chinese, English, German, Spanish, French, Korean, Japanese |
| Dialects | Sichuanese, Cantonese |
| Python API | ming_tts.MingTTS |
| CLI | infer.py |
| Sample rate | 44.1 kHz |
Capabilities
- Zero-shot voice cloning from a short reference clip
- Voice Design from a natural-language speaker description
- Free-form control of delivery and speaking style
- 100+ built-in Chinese voices and five built-in English voices
- 14 inline tags for breath, cough, sigh, laughter, hesitation, surprise and question/confirmation sounds in Chinese or English
- Explicit emotion, speed, pitch and volume control
- Multilingual synthesis and Sichuanese/Cantonese dialect control
Quick Start
Installation
git clone https://github.com/inclusionAI/Ming-3.0-tts.git
cd Ming-3.0-tts
pip install -r requirements.txt
Tested on NVIDIA H800-80GB and H20-96G with CUDA 12.4. flash-attn is optional; inference falls back to PyTorch SDPA.
Numbers, dates, units and symbols are normalized with FST grammars. Install pynini, or pass --no_text_normalization to skip text normalization:
conda install -c conda-forge pynini==2.1.6
Download
pip install "huggingface_hub[cli]"
hf download inclusionAI/Ming-3.0-tts-0.5B --local-dir Ming-3.0-tts-0.5b
CLI Inference
python infer.py \
--model_path inclusionAI/Ming-3.0-tts-0.5B \
--task TTS \
--language_id Chinese \
--text "傍晚的风从山口吹过来，远处的云慢慢染成了金色。" \
--prompt_wav_path data/wavs/10002287-00000094.wav \
--prompt_text "在此奉劝大家别乱打美白针。" \
--use_spk_emb \
--output output/tts.wav
Python API
from ming_tts import MingTTS
model = MingTTS("inclusionAI/Ming-3.0-tts-0.5B")
model.synthesize(
task="TTS",
language_id="Chinese", # English for English text
text="傍晚的风从山口吹过来，远处的云慢慢染成了金色。",
prompt_wav_path="data/wavs/10002287-00000094.wav",
prompt_text="在此奉劝大家别乱打美白针。",
use_spk_emb=True,
output_path="output/tts.wav",
)
Supported tasks are TTS, FREEFORM, VOICE_DESIGN, MULTILINGUAL, BASIC, EMOTION, DIALECT and IP. See the task cookbook for controls and examples.
MULTILINGUAL requires language_id for German, Spanish, French, Korean or Japanese; DIALECT takes no language ID. Freeform uses its own task and cannot be combined with voice design, a built-in voice, emotion, dialect or attribute controls. With a reference clip it follows that voice; without a reference, the instruction can choose the voice.
Verify the Checkpoint
python tools/preflight.py --model_path /path/to/Ming-3.0-tts-0.5b
Performance
Benchmark comparisons from the project README.
Bold and underline mark the best and second-best values among the listed open-source models; ties share a mark. Seed-TTS and NVbench-GT are unranked reference rows.
Zero-shot TTS (Seed-TTS testset)
| Model | seed-tts-eval-zh | seed-tts-eval-en | ||
|---|---|---|---|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | |
| Seed-TTS | 1.110% | 0.796 | 2.240% | 0.762 |
| GLM-TTS | 1.030% | 0.761 | 2.230% | 0.672 |
| Qwen3-TTS | 1.070% | — | 1.390% | — |
| Qwen3-TTS-12Hz-0.6B-Base | 0.920% | — | 1.320% | — |
| Qwen3-TTS-12Hz-1.7B-Base | 0.770% | — | 1.240% | — |
| CosyVoice3-0.5B | 1.160% | 0.780 | 2.020% | 0.718 |
| CosyVoice3-1.5B | 0.710% | 0.775 | 1.450% | 0.695 |
| F5-TTS | 1.560% | 0.741 | 1.830% | 0.647 |
| Ming-3.0-tts-0.5b | 0.800% | 0.758 | 1.584% | 0.691 |
| Ming-3.0-tts-tiny | 0.820% | 0.781 | 1.666% | 0.728 |
Multilingual (MiniMax test set, selected languages)
Evaluated on the German, Spanish, French, Korean, and Japanese subsets of MiniMaxAI/TTS-Multilingual-Test-Set.
| Model | German | Spanish | French | Korean | Japanese | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | WER ↓ | SIM ↑ | CER ↓ | SIM ↑ | CER ↓ | SIM ↑ | |
| dots.tts | 0.452% | 0.798 | 2.593% | 0.839 | 19.878% | 0.789 | 4.837% | 0.853 | 5.134% | 0.838 |
| VoxCPM2 | 0.963% | 0.790 | 1.324% | 0.836 | 3.380% | 0.761 | 3.547% | 0.839 | 4.358% | 0.825 |
| Fish Audio S2 | 1.045% | 0.714 | 1.216% | 0.748 | 3.636% | 0.698 | 1.675% | 0.761 | 3.427% | 0.755 |
| FireRedTTS3 | 0.348% | 0.825 | 1.086% | 0.868 | 4.366% | 0.807 | 1.644% | 0.864 | 3.418% | 0.825 |
| Ming-3.0-tts-0.5b | 0.365% | 0.753 | 1.099% | 0.763 | 4.264% | 0.675 | 1.728% | 0.793 | 3.765% | 0.765 |
| Ming-3.0-tts-tiny | 0.325% | 0.791 | 0.971% | 0.797 | 3.470% | 0.733 | 6.000% | 0.797 | 3.427% | 0.780 |
Each open-source model was evaluated using its recommended inference settings under a unified ASR-based WER/CER evaluation protocol.
Paralinguistic Control (NVBench)
| Model | Single | Multi | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Chinese | English | Chinese | English | |||||||||
| CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER | CER | PCER | OCER | |
| NVbench-GT | 3.86 | 9.38 | 4.07 | 6.73 | 8.31 | 6.90 | 3.79 | 23.71 | 5.07 | 7.26 | 21.41 | 8.62 |
| Orpheus-TTS | 11.36 | 88.77 | 13.91 | 9.03 | 71.92 | 10.63 | 19.83 | 84.85 | 24.38 | 8.68 | 71.46 | 11.89 |
| SMIIP-NV-CV2 | 8.80 | 75.64 | 11.34 | 17.92 | 56.80 | 19.47 | 10.66 | 77.20 | 14.79 | 20.93 | 54.49 | 23.87 |
| Emilia-NV-CV2 | 5.05 | 40.00 | 6.64 | 12.50 | 55.30 | 13.21 | 5.54 | 48.74 | 8.09 | 11.71 | 60.28 | 14.63 |
| CosyVoice3 | 3.85 | 57.69 | 5.86 | 7.87 | 62.75 | 9.06 | 4.75 | 61.94 | 8.26 | 6.39 | 57.84 | 10.69 |
| NV-FlexiVoice | 6.98 | 31.08 | 8.15 | 11.88 | 50.43 | 13.21 | 8.20 | 39.37 | 10.39 | 9.60 | 51.32 | 13.76 |
| NV-CV3 | 3.80 | 27.69 | 4.90 | 8.33 | 46.13 | 9.44 | 3.44 | 30.04 | 4.84 | 6.70 | 47.13 | 10.10 |
| Ming-3.0-tts-0.5b | 2.70 | 22.80 | 3.20 | 7.60 | 37.00 | 8.90 | 3.50 | 36.60 | 8.10 | 7.90 | 47.30 | 10.40 |
| Ming-3.0-tts-tiny | 2.80 | 17.23 | 3.20 | 5.86 | 32.95 | 7.05 | 4.49 | 32.88 | 5.81 | 5.12 | 46.13 | 7.68 |
Dialect Control
| Model | WSC-Eval-TTS-easy | WSC-Eval-TTS-hard | WSYue-TTS-eval-Base | WSYue-TTS-eval-Coverage | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | CER ↓ | SIM ↑ | ACC ↑ | |
| Qwen3-TTS | 4.13% | — | — | 7.35% | — | — | — | — | — | — | — | — |
| CosyVoice3-0.5B | 3.17% | 0.696 | 76.05% | 4.07% | 0.723 | 81.41% | 8.36% | 0.611 | 91.70% | 8.95% | 0.658 | 95.80% |
| CosyVoice3-1.5B + DiffRO-EMO | 7.14% | 0.703 | — | 9.06% | 0.601 | — | 14.38% | 0.812 | — | 13.74% | 0.826 | — |
| CosyVoice2-WSC | 4.28% | 0.728 | — | 8.87% | 0.626 | — | — | — | — | — | — | — |
| CosyVoice2-WSC-SFT | 4.08% | 0.788 | — | 7.22% | 0.680 | — | — | — | — | — | — | — |
| CosyVoice2-Yue | — | — | — | — | — | — | 10.33% | 0.821 | — | 9.49% | 0.834 | — |
| Sparks-TTS | 10.83% | 0.677 | — | 12.52% | 0.545 | — | 27.79% | 0.762 | — | 24.25% | 0.781 | — |
| Ming-3.0-tts-0.5b | 2.27% | 0.689 | 80.68% | 3.35% | 0.691 | 87.64% | 6.15% | 0.586 | 98.00% | 8.15% | 0.622 | 95.27% |
| Ming-3.0-tts-tiny | 2.53% | 0.721 | 86.09% | 3.11% | 0.738 | 88.94% | 7.40% | 0.605 | 92.90% | 8.67% | 0.653 | 95.16% |
Emotion Control (CV3)
| Model | Average | Text-Related | Text-Unrelated | ||||
|---|---|---|---|---|---|---|---|
| Happy | Sad | Angry | Happy | Sad | Angry | ||
| CosyVoice3-0.5B | 66.33% | 92% | 70% | 72% | 64% | 42% | 58% |
| CosyVoice3-1.5B | 63.00% | 86% | 64% | 72% | 64% | 44% | 48% |
| CosyVoice3-1.5B + DiffRO-EMO | 77.67% | 98% | 68% | 84% | 98% | 50% | 68% |
| F5-TTS | 64.67% | 92% | 52% | 72% | 80% | 28% | 64% |
| Sparks-TTS | 55.33% | 80% | 56% | 50% | 50% | 60% | 36% |
| Ming-3.0-tts-0.5b | 72.33% | 96% | 76% | 94% | 52% | 36% | 80% |
| Ming-3.0-tts-tiny | 74.00% | 98% | 82% | 90% | 60% | 36% | 78% |
Emotion Control (CV3N, hard emotion control)
| Model | Average | Text-Related | Text-Unrelated | ||||
|---|---|---|---|---|---|---|---|
| Happy | Sad | Angry | Happy | Sad | Angry | ||
| CosyVoice3-0.5B | 40.00% | 68% | 30% | 78% | 14% | 4% | 46% |
| Ming-3.0-tts-0.5b | 37.33% | 70% | 36% | 70% | 14% | 4% | 30% |
| Ming-3.0-tts-tiny | 30.67% | 60% | 40% | 44% | 18% | 10% | 12% |
Voice Design (InstructTTSEval-ZH)
| Model | Average | APS | DSD | RP |
|---|---|---|---|---|
| Qwen3-TTS | 77.13% | 85.20% | 81.10% | 65.10% |
| Mimo-Audio-7B-Instruct | 70.50% | 75.70% | 74.30% | 61.50% |
| VoiceSculptor | 67.30% | 75.70% | 64.70% | 61.50% |
| VoxInstruct | 47.47% | 47.50% | 52.30% | 42.60% |
| Ming-3.0-tts-0.5b | 72.29% | 78.54% | 77.28% | 61.06% |
| Ming-3.0-tts-tiny | 75.37% | 85.10% | 79.90% | 61.10% |
Basic Attribute Control
| Model | Speed | Volume | Pitch | Average | WER ↓ | SIM ↑ |
|---|---|---|---|---|---|---|
| CosyVoice3-0.5B | 100.00% | 97.67% | 65.33% | 87.67% | 1.210% | 0.580 |
| Ming-3.0-tts-0.5b | 99.00% | 95.00% | 86.00% | 93.30% | 0.317% | 0.712 |
| Ming-3.0-tts-tiny | 93.00% | 93.00% | 78.70% | 88.23% | 0.274% | 0.742 |
Limitations
- Speaker similarity is weaker than content accuracy; French SIM is the clearest multilingual gap.
- NVBench Multi trails Single; stability with several paralinguistic events in one sentence is still improving.
- Emotion control is harder when the words carry no emotional signal.
- Pitch is less accurate than speed and volume, and open-ended Voice Design does not yet match the strongest evaluated system.
Usage Disclaimer
- Voice cloning and built-in character voices are provided for research and legitimate creative use. Do not use them to impersonate a real person without consent, to deceive, or for any illegal purpose.
- Label synthetic speech as synthetic where it could be mistaken for a real recording.
- The authors accept no liability for misuse. If you find abuse of this project, please report it to us.
Acknowledgements
- Qwen2.5 for the backbone
- CAM++ for speaker embedding extraction
- Ming-omni-tts, the previous generation, whose audio tokenizer this release builds on
License
Released under the MIT license.
- Downloads last month
- 318