Stable Audio 3.0 and text-to-speech models both accept words and produce waveforms. That superficial similarity causes a common design error: sending every audio request to one model.
Stable Audio is built to compose music, synthesize sound effects, transform recordings, and edit regions. TTS is built to speak a script intelligibly through a chosen voice. Dubbing adds a third workflow: translate an existing performance while preserving its timing and emotional qualities.
Choosing the correct category before generation is more important than choosing a provider.
Different contracts from the first token
| Requirement | Stable Audio 3.0 | TTS | Dubbing |
|---|---|---|---|
| Primary input | Creative description and optional source audio | Exact text plus voice and delivery controls | Source audio or video plus target language |
| Primary output | Music, ambience, effects, transformed audio | Spoken words | Translated performance |
| Must preserve wording | No | Yes | Meaning, with natural rephrasing |
| Must preserve a speaker | Not the central contract | Selected or cloned voice | Source speaker characteristics |
| Timing | Requested duration and musical structure | Derived from speech and controls | Aligned to source segments |
| Real-time fit | Generally offline generation | Batch or low-latency streaming | Generally offline |
| Editing | Audio-to-audio, inpainting, continuation | Regenerate text spans or adjust delivery | Transcript, speaker, segment, and sync workflows |
The categories can be combined in a finished production. They should not be collapsed into one model call.
Stable Audio creates a sonic event
A Stable Audio prompt describes what should be heard: instrumentation, genre properties, tempo, mood, acoustic space, sequence, duration, and production character. The model may invent notes, textures, timing, and arrangement.
The Stable Audio 3 family includes Small Music, Small SFX, Medium, and Large. The downloadable checkpoints can generate at variable lengths, transform source audio, inpaint one or more regions, continue a track, and accept LoRA adaptation. Output is 44.1 kHz stereo through the SAME autoencoder.
This is appropriate for:
- background music and stingers;
- game or interface sounds;
- ambience and environmental beds;
- impacts, transitions, and designed effects;
- musical sketches and variations;
- replacing or extending part of an authorized track.
It is not appropriate for a legally approved disclaimer that must be spoken word for word. Even if a music model produces a vocal line, lyric intelligibility and exact wording are not its primary contract.
TTS realizes a linguistic script
A TTS request starts with language. The words are known; the model decides how to pronounce and perform them. Inputs usually include a voice, model, text, output format, and controls for style, speed, emotion, pauses, or pronunciation.
ElevenLabs’ current model catalog illustrates the range. Eleven v3 emphasizes expressive delivery and supports more than 70 languages; Multilingual v2 targets stable long-form speech; Flash v2.5 targets low latency around the provider’s reported 75 milliseconds and supports 32 languages. Those are provider-specific capabilities, not universal TTS standards.
Use TTS for:
- narration and voiceover;
- accessibility and screen reading;
- announcements and instructions;
- character dialogue;
- voice-agent output;
- scripts with names, numbers, or regulated wording.
Review exact transcript, pronunciation, accent, pace, pauses, voice authorization, and audio format. A plausible voice that says the wrong number is a failed TTS output.
Dubbing transfers an existing performance
Dubbing begins with speech already performed. A system transcribes, translates, separates speakers and background sound, synthesizes translated voices, and aligns them to the original timeline.
ElevenLabs Dubbing v2, announced May 28, 2026, conditions directly on the original performance and aims to preserve tone, pacing, delivery, and emotional intent across more than 90 languages. Its sync-aware translation adjusts phrasing for natural speech while aligning starts, stops, and pacing.
As of July 17, the current documentation labels Dubbing v2 Alpha, says it is not real-time, and says the v2 API is not yet live. It is available through Automatic Dubbing in ElevenCreative and through ElevenProductions. These status limits should not be inferred from the general v1 Dubbing API.
Dubbing is the right category when an existing speaker and performance matter. Plain TTS may be sufficient when only a translated script and approved target voice are needed.
Music with vocals is not narration
Music models can produce sung or vocal-like sounds. That does not make them reliable speech engines. Lyrics can be distorted, repeated, or replaced with phonetic approximations. The voice may change across phrases, and the output may resemble a performance without providing a controllable speaker identity.
If a song requires exact lyrics, consider a specialized vocal workflow with explicit rights, then review every word. If a video requires narration over music, generate the instrumental bed and speech separately. Mixing later gives control over intelligibility, ducking, timing, and replacement.
Likewise, do not ask TTS to generate a complete musical bed by describing one in prose. It may supply expressive speech or audio tags, but its output contract is voice.
Latency requirements separate the systems
Real-time agents need speech quickly enough for natural turn-taking. TTS models often have streaming modes, telephony codecs, and predictable text-to-duration behavior. Stable Audio can be very fast on optimized hardware, but its product objective is not conversational turn-taking or exact verbal response.
Offline production has different priorities. A music model may spend longer producing a six-minute arrangement, while a narrator’s entire script can be reviewed before final synthesis. Dubbing must process source separation, translation, and alignment.
Set latency targets by asset:
- voice agent: first-audio latency and interruption behavior;
- narration: accepted minutes per hour of production;
- music: time and samples per approved cue;
- dubbing: turnaround per source minute and target language.
Do not compare these on one “seconds to waveform” metric.
Rights differ by asset type
Stable Audio’s provider says its models use licensed and CC0 data and governs weights through the Community License. That does not authorize every source recording or guarantee that an output is clear of similarity concerns.
TTS adds voice rights. A person’s voice should be cloned or used only with explicit permission, defined purposes, and revocation. A library voice has its own terms.
Dubbing combines translation, performance, voice, and audiovisual rights. Permission to publish an original video does not always include synthetic localization into every language. Contracts should address translated performances, territories, edits, and attribution.
Build a routing policy
A production system can classify a request before calling a model:
- Does the output need exact spoken words? Route to TTS.
- Does it translate an existing performance? Route to dubbing.
- Is it music, ambience, or an effect? Route to Stable Audio.
- Does it combine speech and music? Generate separate stems.
- Does it require a visible speaker? Add a later lip-sync or avatar stage.
Require manual review when classification confidence is low. Store asset type, source rights, provider, model, prompt or script, voice ID, output checksum, and approval.
Route ambiguous prompts to clarification instead of guessing. “Make an audio intro” could mean a musical sting, a spoken welcome, or both, and each interpretation invokes different models, rights, and review criteria.
Combine the right stems downstream
For a video, keep music, effects, and speech as separate approved assets as long as possible. This allows a localized narration to change without regenerating music, or a rejected cue to change without resynthesizing a voice.
An authorized Claude or Codex MCP client can use Medux as an external media layer. Stable Audio and the cited TTS systems do not natively route into Medux. The audio-track tutorial demonstrates adding an approved music or effects track to video. The Codex MCP TTS tutorial shows generating a spoken asset through a separate task.
Persist the script, voice consent, music prompt, model versions, audio checksums, input video, and Medux task IDs. If the mix fails, retry the media operation; do not change the underlying audio generation without a new approval.
Stable Audio is for what the world around the speaker should sound like. TTS is for what the speaker should say. Dubbing is for how an existing performance should cross a language boundary. That simple taxonomy makes models easier to evaluate and finished media easier to control.