Audio-Driven MV

Developing music videos where the visual performance actually matches the source audio is a high-friction process. This skill automates the coordination between Toonflow infinite canvases and MiniMax H3 Digital Human models to generate lip-synced MVs driven by existing song tracks. It eliminates the guesswork in prompt engineering for specific digital human nodes, ensuring character consistency and environment stability across multiple segments.

What it does

• Lip-sync orchestration using the h3DigitalHuman workflow to synchronize character mouth movements with provided vocal tracks.
• Scene management via Toonflow canvases to handle character refs, scene refs, and multi-segment song structures without losing context.
• Blender previs integration for planning complex movement, camera paths, and blocking before committing to expensive cloud generations.
• Automated post-processing using specialized scripts to trim video tails to match exact WAV durations without re-encoding or audio drift.
• Task recovery by tracking Request IDs and Task IDs to prevent duplicate billing and resume interrupted generation flows.

How it works

1. Asset validation where the skill checks audio durations via ffprobe and aligns lyric segments with individual audio files.
2. Workflow selection by querying current MCP capabilities to identify the correct MiniMax H3 node IDs and parameter schemas.
3. Generation execution involving the creation of localized prompt nodes and submission of tasks to the Toonflow backend.
4. Final alignment using trimToAudio.ts to ensure the final MP4 exactly matches the source audio length.

Frameworks & tools

Built for the Toonflow ecosystem utilizing MiniMax H3 (Digital Human variant). It integrates with Blender for 3D pre-visualization and uses TypeScript-based scripts for FFmpeg operations.

Why this beats prompting it yourself

Writing prompts for digital humans often results in "hallucinated" lip-sync or environmental flickering. This skill enforces strict reference-to-audio mapping and utilizes specific "lip-sync clock" declarations that manual prompting often misses. It also handles the technical overhead of task ID persistence and file-system locking across long-running generation jobs.

Use cases

• Creating multi-segment music videos from a single master song file.
• Re-generating specific segments of an MV while maintaining character and scene parity.
• Planning complex camera paths in Blender to guide AI video generation.
• Batch processing lyric-synced social media clips for virtual idols.

Known limitations

Does not support real-time audio synthesis or post-generation voice swapping. Video generation will fail if the source audio is longer than the generated visual output.