multilingual tts segmentation

Voice assistants and language tutors often mangle bilingual text by pronouncing Markdown syntax, misclassifying languages, or using the wrong accent for quoted phrases. This skill provides the logic and order of operations to correctly segment multilingual text before it reaches the synthesis engine.

What it does

• Markdown Stripping: Removes asterisks and syntax server-side to prevent voices from literally saying "asterisk asterisk".
• Protected Segmentation: Ensures quoted spans remain unbreakable so punctuation inside quotes doesn't cause premature sentence splitting.
• Phonetic Cleaning: Identifies and removes hallucinated phonetic symbols (like /k/) that models invent but TTS engines misread.
• Language Classification: Applies refined rules for elisions and stop-words to prevent false language switching in multilingual voices.
• Silence Injection: Logic for joining synthesized segments with specific timing to prevent audible "hard cuts" between voices.

How it works

1. Pre-process: Strip all markdown and pseudo-phonetic slashes from the model output.
2. Segment: Split the text into sentences while treating quoted spans as atomic, unbreakable units.
3. Classify & Merge: Identify the language of each chunk and merge adjacent chunks of the same language.
4. Clean & Synthesize: Drop chunks containing no alphanumeric characters and send remaining segments to their respective voices.

Frameworks & tools

The skill provides logic applicable to any TTS pipeline (Azure Neural, AWS Polly, OpenAI Audio) and includes scripts/measure_voice.js for duration-based defect testing.

Why this beats prompting it yourself

Prompting a model to "not use markdown" is not a guarantee, as models frequently ignore these constraints in edge cases. This skill implements the server-side safety nets and segmentation logic required to fix defects that prompts alone cannot resolve.

Use cases

• Building a language tutor that switches between a native and target language.
• Developing a bilingual assistant that quotes foreign text without changing accents.
• Fixing "stuttering" or "spelling out" behaviors in production TTS systems.

Known limitations

Does not include voice cloning or model training logic. The phonetic slash removal is narrowly scoped to avoid breaking dates or "and/or" constructions.