Routyo: spoken-language router for Haitian Creole, English, French and Spanish

Routyo listens to a sentence and says which language it is: Haitian Creole (ht), English (en), French (fr) or Spanish (es), with a rate for each. It is made to sit in front of small speech and language models: it sends each sentence to the right one, so Kreyòl and the other three languages can be mixed in one conversation without any button to press.

It works with any voice and any accent. A Haitian speaking English is English; a foreigner speaking Kreyòl is Kreyòl. It runs on CPU (about 35 ms per sentence on a Mac) and weighs about 50 MB in all.

Built by a Haitian team, on one Mac, for the 12+ million people who speak Kreyòl and for anyone who moves between these four languages.

Results

Whole sentences, four-way accuracy. Nothing below was used in training: other speakers, other voices, other sentences. "Pair" asks Kreyòl against one other language only, as an interpreter app does. Whisper large-v3-turbo is scored on the same clips with its language guess limited to the same four languages.

test set	clips	Routyo (+ word check)	Routyo, sound only	Whisper large-v3-turbo
real speech, other people	1,751	99.7 % (pair 99.8)	99.7 %	91.7 %
synthetic voices never heard in training (Haitian, foreign, native)	2,130	96.6 % (pair 98.5)	95.2 %	51.5 %
earlier synthetic Haitian voices, held out	628	96.7 % (pair 97.9)	97.1 %	64.6 %
one Haitian speaker, live on a phone	47	93.6 % (pair 97.9)	89.4 %	74.5 %

On Kreyòl itself the gap is widest. Kreyòl spoken in foreign voices: Routyo 97.3 %, Whisper 27.9 %. Haitian voices speaking English, French or Spanish: Routyo 95.1 %, Whisper 31.9 %.

From only the first 1.5 s of speech (sound only): real speech 92.7 % (pair 96.2), unseen voices 87.4 % (pair 93.4).

Speed on an Apple-silicon Mac, 200 real sentences of 8.7 s on average:

	per sentence	size
Routyo, sound only (CPU)	35 ms	~50 MB
Routyo + word check (CPU)	~0.3 s	same
Whisper large-v3-turbo language detection (Mac GPU)	0.8 s	1.5 GB

Use it

pip install cactus-compute numpy soundfile soxr huggingface_hub
hf download jsbeaudry/Routyo --local-dir routyo
python routyo/routyo.py clip.wav                  # all four languages
python routyo/routyo.py clip.wav --words          # with the word check (more accurate, ~0.3 s)
python routyo/routyo.py clip.wav --pair ht,en     # Kreyòl against English only
python routyo/routyo.py clip.wav --first 1.5      # decide from the first 1.5 s of speech

import sys; sys.path.insert(0, "routyo")
from routyo import Routyo, load_audio

r = Routyo()                              # downloads the stock Whistle encoder on first use
r.rates(load_audio("clip.wav"))           # [{"language": "en", "name": "English", "rate": 0.987}, ...]
r.rates(samples_16k, pair=("ht", "fr"))   # 16 kHz mono float32 samples work too

Runs on the Cactus engine (cactus-compute, tested with 4.1.0), which runs Whistle on CPU. The results above were measured with its earlier version (cactus-needle, engine 3.3.3); on 400 test clips the two engines pick the same language 397 times, with the same accuracy.

How it works

piece	what
encoders (frozen)	the stock Whistle encoder (whistle.cact, downloaded from its own repo) and a Kreyòl-tuned Whistle (whistle-kreyol.cact, in this repo). Each gives 512 numbers per 80 ms through the engine's embed call; Routyo joins them (1,024 per frame).
head (routyo.npz)	per-frame layers (1,024 → 256 → 256, ReLU), then attentive mean and spread over time beside the plain mean and spread, then 4 languages. Three such heads, trained from different seeds, are averaged (log-probabilities). Each is temperature-calibrated.
word check (optional, word_check.json)	Whistle transcribes the sentence forced into each language (the Kreyòl encoder for ht, the stock one for en/fr/es); the mean word probability wp[k] joins the sound score: score_k = (log p[k] + β·wp[k] + c_k) / T. β, c and T were fitted on validation data only.

routyo.json holds the same head as plain arrays with the combining rule, for porting to any runtime (it is what a phone app loads).

The Kreyòl-tuned encoder is Whistle fine-tuned on about 145 hours of transcribed speech, mostly Haitian Creole plus some French, English and Spanish (FLEURS) so it keeps those languages. As a transcriber it reaches 14.6 % word error rate on our Kreyòl test set; here only its encoder frames and its forced Kreyòl transcription are used.

Training data

Real speech: Kreyòl 12.4 h (radio, phone recordings, read speech, Bible readings, studio recordings, online videos), English 6.3 h, Spanish 3.9 h, French 3.1 h (FLEURS, MInDS-14, VoxPopuli, including accented English).

Synthetic speech, made for this model so that no voice and no accent gives the language away:

kind	engine	voices	gives
Haitian voices	Fatiman V3 (our Kreyòl text-to-speech)	13 consented Haitian voices (10 for training, 3 kept for test)	Haitian-accented English, French, Spanish, and Kreyòl
foreign voices speaking Kreyòl	Fatiman V3 cloning catalogue voices	102 voices with US, UK, French, Spanish, Mexican, German and Brazilian accents (87 train, 15 test)	Kreyòl in foreign voices
native voices	gradium/phonon-40m-multilingual	the same catalogue voices	native English, French and Spanish, and Kreyòl read with a foreign accent

The texts are everyday sentences written in all four languages with the same meaning, so the topic never tells the language. Every synthetic clip had to be read back correctly by Whistle in its own language (character error rate ≤ 0.35): 16,935 of 20,300 clips passed (15.9 h). No voice of a data contributor was cloned.

Limits

• Four languages only. Speech in any other language is still put in one of the four.
• Short clips are the hardest: under 2 s, and the first-1.5 s numbers, are lower than whole sentences.
• The live test is one speaker; the unseen-voice test uses synthetic voices.
• The hardest cases: Haitian voices speaking English (91.7 %) and French read by a synthetic voice (91.1 %). French and Kreyòl share many words and sounds.

Files

file	size	what
routyo.py	8 KB	inference: audio → language rates (command line and Python)
routyo.npz	4.3 MB	the three heads (numpy arrays)
routyo.json	22 MB	the same heads as JSON, for other runtimes
whistle-kreyol.cact	29 MB	the Kreyòl-tuned Whistle (Cactus .cact format)
word_check.json	1 KB	word-check weights

The stock whistle.cact (17 MB) is fetched from Cactus-Compute/whistle.

License and credits

Apache 2.0, like the Whistle base model and the Cactus engine by Cactus Compute.

Thanks to Cactus Compute for Whistle and the Cactus engine, and to Gradium for Phonon-40m-multilingual (CC BY-NC 4.0), which voiced part of the synthetic training audio.