Top AI Product

NAVA — Native Audio-Visual Alignment for Generation — comes out of Baidu’s ERNIE research group and stakes a position the field has been edging toward: audio and video should be learned together as one signal, not two separate streams stapled at the seam.

## Native, not stitched

Most audio-visual generation pipelines today are bolted together. A video model produces frames, an audio model produces sound, and an alignment stage tries to hide the seam — usually with limited success on lip sync, footstep timing, or sound that actually matches what’s happening on screen. “Native” alignment is the alternative: a single model that treats audio and pixels as a joint representation from the start, so the timing relationships are part of what the model learns, not a post-hoc correction.

## Why it matters

The seam shows. Audio-video misalignment is the small thing that makes generated clips feel wrong — a cup hitting a table half a beat after the contact, a voice barely out of sync with the lips. Solving that at the architecture level rather than at the alignment stage is the path NAVA stakes out, and it matches the broader move toward native multimodal models — pixels-and-text natively unified, now extended to pixels-and-sound. The code is open at ernie-research/NAVA; the paper is on arXiv. As generative video gets used for ads, shorts, and game content, getting native audio-visual coherence right turns into product polish that users notice.