Dubbing short videos into Thai: from subtitles to voice cloning
Turning a Chinese product video into a Thai version takes more than translation. The translated lines have to finish in the time the original took, the voice should still be the same person's, the subtitles have to fit, and the lip movement should not be obviously off. This article follows the processing order and covers the stages a video goes through from Chinese to Thai, and what tends to go wrong in each.
1. The six stages
A video goes through these stages, from upload to the finished file:
- Upload: the video is sent to the server.
- Speech recognition: sentence-level timeline and speaker separation.
- Translation: keep the terminology, control the length of the translation.
- Voice-cloned dubbing: each speaker keeps their original voice.
- Lip sync: sync lip movements to the new audio.
- Mix and export: mixing, burned-in subtitles and the finished video.
Later stages build on the results of earlier ones. If a problem shows up, go back to the stage that caused it and fix it there. Do not patch it in a later stage.
2. Prepare the original first
The state of the original video already decides half of the result:
- The voices must be clear. Recognition suffers when background music covers speech. If there is a version of the voice-over without music, use it.
- Only one person should speak at any moment. People talking over each other make speaker separation and voice cloning unstable.
- For lip sync, keep the face towards the camera and the mouth unobstructed.
Before processing the whole video, pick a 10-to-15-second clip with two speakers and run it through every stage. Problems in recognition, translation, dubbing and subtitles show up in minutes on a short clip. Fix them, then process the full video.
3. Recognition: a sentence-level timeline
The first step is speech recognition. It outputs the text of each sentence, a timeline, and who is speaking. The timeline is the frame for everything that follows: each translated sentence has to go back into the time window of the sentence it came from.
The sentence boundaries that recognition gives are not always the right subtitle boundaries. Split sentences that are too long and merge fragments that are too short. Translation works sentence by sentence, so how cleanly the sentences are cut affects how well the translation reads.
Brand names, model numbers and figures are what recognition gets wrong most often in product videos. Proofread them one by one in the subtitles before dubbing. Changing one character at this stage costs far less than changing one in the finished video.
4. Translation: consistent terms, controlled length
The translation stage does two things: it keeps terminology, and it controls the length of the translation.
For terminology, brand names, product names and model numbers should either stay as they are or have one agreed rendering, the same throughout the video.
Length needs more care. Each Chinese character is one syllable and carries a lot of meaning, so the same content in Thai usually needs more syllables and takes longer to say. If the translation is too long, the dub can only be sped up, and once the speech is fast it stops sounding natural.
A product video also has content that should be decided in advance: prices, currencies, units and promotional phrases. Whether to change them to what Thai viewers are used to is not something translation can decide for you. Write the decision into the glossary so the translation stays consistent.
So when the translation is too long, rewrite it first: remove repetition from spoken phrasing, split a long sentence into two short ones, use a shorter way of saying it. Whatever can be solved in translation should not be left for the dubbing stage.
5. Voice-cloned dubbing: each speaker keeps their own voice
Voice-cloned dubbing takes a reference voice from the original video for each speaker, and reads the Thai translation in that voice.
The requirements for the reference segment are the same as for standalone voice cloning: one speaker, quiet room, no background music, natural pace. If a speaker only appears in segments covered by music, the cloned voice will suffer. For recording tips, see Voice cloning from an 8-second sample: recording tips.
Dubbing naturalness is measured by Mean Opinion Score (MOS) on a 5-point cross-lingual scale. See the metrics strip on the video dubbing page.
6. Duration alignment
Every Thai line has to finish within the time window of the original sentence. Otherwise the sound and the picture drift apart.
To align one line of dubbing, first work out the ratio it needs:
ratio
= natural length of the Thai line
÷ time window of the source lineA ratio of 1 or less means it fits, and a short pause remains after the line. A ratio slightly above 1 can be handled by adjusting the speed within a natural range, or by using the gap between two neighboring sentences. A ratio clearly above 1 means going back to translation and rewriting that sentence.
Do not speed up the whole track just to make the time fit. Speeding up everything makes every sentence sound rushed, while rewriting touches only the sentence that has the problem.
7. Lip sync
Lip sync matches lip movements naturally to the new dubbed audio. It works best for talking-head and interview videos.
If the speaker's mouth is not the main thing on screen, as in product close-ups and shots of hands at work, lip sync gains little. For a video made mostly of such shots, you can skip lip sync and only replace the audio track.
After it runs, check two things: at the moment speech starts and ends, whether the mouth opens early or stays open too long; and at bilabial sounds such as b, p and m, whether the lips close. Shots of a face in profile, with the mouth covered, or with a very small mouth in frame gain little, and you can skip them.
8. Subtitles and export
The last stage mixes the tracks, burns in the subtitles and outputs the finished video. You can also download only the SRT subtitle file and use it in your own post-production.
When mixing, the Thai dub should be about as loud as the voice in the original, and the music and ambient sound should stay where they were, not covering the voice.
Each subtitle has to stay on screen long enough for viewers to read it. For the same meaning, a Thai subtitle has far more characters than a Chinese one, so Thai subtitles often have to be split into more, shorter pieces.
Thai subtitles also differ from Chinese in two other ways:
- Thai is written without spaces. Line breaks belong between words, never inside a word. Keep each line short.
- Thai tone marks and upper vowels stack above the consonant, and tight line spacing cuts them off. When you burn in subtitles, use a font that supports Thai and leave a little more line spacing than for Chinese.
9. Check before delivery
- Are brand names, model numbers and figures read correctly?
- Does every line finish within the time window of the original sentence?
- Are the voices of different speakers kept apart?
- Are there typos in the subtitles, and do line breaks fall at word boundaries?
- Is the dub balanced with the background music?
- Has a native Thai speaker listened to the whole thing? Correct grammar does not mean it sounds natural.