Flexible Video Diffusion
Flex-Forcing enables video diffusion models to switch between bidirectional and autoregressive generation using flexible chunking to optimize speed and consistency.
Deep dive
- Unifies bidirectional (global planning) and autoregressive (streaming) generation.
- Employs flexible chunking over both frames and denoising steps.
- Improves video quality and long-range stability on 30s video benchmarks.
- Allows for 'any-order' and 'any-timestep' editing by decoupling global structure from local details.
- Outperforms previous methods like Self-Forcing on both FPS and VBench metrics.
Decoder
- Bidirectional diffusion: A generation process where the model attends to past and future frames simultaneously to ensure global coherence.
- Autoregressive generation: A process where the model generates data sequentially, predicting the next frame based only on previous ones.
- Denoising steps: The iterative process in diffusion models where noise is removed to reveal a clear image or frame.
Original article
5s videos generated by Flex-Forcing — better speed with better quality. Hover any clip to pause & read its prompt.
One Model, Bidirectional & Autoregressive
Flex-Forcing unifies the generation spectrum of both autoregressive generation and bidirectional generation. Drag the slider from one bidirectional chunk to autoregressive small chunks and watch the same prompt generate under every regime in between.
Pick a prompt for each panel, then drag the shared slider. Larger chunks plan more globally (closer to bidirectional diffusion); smaller chunks generate more autoregressively — the same model covers both ends, on every prompt.
Abstract
Recent progress in large-scale generative models has substantially advanced video generation, yet existing methods remain constrained by a rigid inference paradigm. Bidirectional diffusion models excel at global coherence and visual fidelity but suffer from slow inference, while autoregressive models offer efficient and streaming generation at the cost of long-range consistency and exposure bias. We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes. The core idea is a flexible chunking mechanism jointly defined over the temporal dimension and denoising steps. This design allows the model to (1) flexibly chunk to support different device budgets, (2) perform bidirectional inference across chunks for global structure planning while generating frames autoregressively within each chunk for efficient, fine-grained synthesis, and (3) perform any-order, any-timestep autoregressive generation without the strict causal constraint. Extensive experiments on multiple benchmarks demonstrate that Flex-Forcing achieves consistently better video quality and long-video stability than strong baselines while offering faster inference.
Method
A single flexible-chunking mechanism, defined jointly over video frames and denoising timesteps, unifies autoregressive, bidirectional, and hybrid generation in one model.
Flexible chunking over frames and timesteps. Fine-grained chunks recover autoregressive generation (efficient, lower quality); a single coarse chunk recovers bidirectional generation (high quality, less efficient). Flex-Forcing interpolates between them: chunking over frames sets the granularity, while chunking over timesteps lets the granularity evolve along the denoising trajectory (high → low noise).
Speed–Quality Trade-off
Across configurations, Flex-Forcing dominates Self-Forcing on both FPS and VBench score.
Brute-force search over chunk configurations (5s video split into three chunks over 21 latent frames). The Pareto-optimal hybrid [9, 6, 6], a coarse-to-fine pattern, beats uniform [7, 7, 7] and Self-Forcing [3, 3, …] on both FPS and VBench, and can even edge out fully bidirectional inference.
With different configurations, our model achieves superior performance over Self-Forcing on both FPS (frames per second) and VBench score.
Qualitative Comparison
Flex-Forcing (ours) vs. Self-Forcing under same prompts.
Long-Range Consistency and Dynamic Degree (30s Videos)
Autoregressive comparison over an extended horizon — Flex-Forcing maintains stability over time while preserving a higher dynamic degree.
Videos are played at 3× speed.
Applications: Any-Order, Any-Timestep Editing
We restrict editing to low-level refinement timesteps while keeping high-level planning fixed, decoupling global structure from local details for far better consistency than Self-Forcing, where small edits propagate across frames. Attending to both past and future tokens, our model edits any chunk regardless of its temporal order.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi