PixVid lifts video diffusion into pixel space

Video generation has long been constrained by VAE temporal compression, making real-time interactive generation and world models sluggish. PixVid demonstrates that lifting pretrained latent diffusion transformers directly to raw pixels is practical, computationally efficient, and critical for low-latency applications.

• –Slashes interactive latency: Frame-by-frame pixel generation eliminates multi-frame latent queues, dropping control-to-screen lag from 133 ms to the immediate next frame.
• –High resolution at constant compute: Doubling the patch size and adding an upsampling detailer block produces 2× higher resolution without increasing DiT attention token counts.
• –Clean video prediction stabilizes training: Predicting clean video ($x_0$) instead of velocity along with a larger noise shift avoids modeling full-rank pixel noise and stabilizes training.
• –Efficient finetuning: When initialized from Wan2.1-14B, PixVid reaches baseline latent quality within roughly 20,000 training iterations (~1.3M clips).