Skip to content
Contents
ADR-0009: Voices and lip sync as provider chains
- Status: Accepted (2026-09-29)
- Date: 2026-09-29
Context
- Rhubarb Lip Sync. v1.14.0 (April 2025) is the latest release and the project is mostly
dormant. Its macOS release is Intel-only, and it crashes under Rosetta on this Mac.
There is no Linux arm64 build. It builds and runs natively from source on Apple
Silicon: CMake, Boost headers only, target
rhubarb only. It accepts WAV (8–32-bit,
16–96 kHz) and Ogg, and takes the transcript with -d, which improves accuracy.
- macOS
say. System voices are licensed for personal, non-commercial use only, so
they cannot voice shareable demo content.
- Local TTS. Kokoro-82M (code, weights and G2P all Apache-2.0) is the best permissive
local option and returns word timings. Caveat: its training data includes synthetic audio
from closed models.
- Cloud TTS. Several cloud voices return alignment data: phoneme timings from Cartesia
and Hume, visemes from Azure, per-character timings from ElevenLabs.
Decision
- TTS chain, per show:
recorded (approved takes, pinned by hash) → configured cloud
providers → kokoro (local, via a uv subprocess). Takes are cached by
hash(text, voice binding, delivery, provider+version) and stored as FLAC.
- Lip-sync chain:
rhubarb (built from source by tools/rhubarb/install.sh, pinned to
v1.14.0, run with -d <transcript> --extendedShapes GHX -f json) → later an
alignment provider that maps TTS or forced-alignment phoneme timings onto the same
A–H/X shapes. The engine sees only the resolved viseme track, never the provider.
- Demo dialogue for M1 is generated once with Kokoro, committed as FLAC under CC BY 4.0,
with provenance recording the model, voice and settings.
Consequences
- No step in the M1 demo depends on a network service or a paid key.
- Rhubarb is replaceable without touching the Timeline format.
- The toolchain grows by one optional Python environment. It is only needed to regenerate
demo audio or voice new lines locally.