A²RD: Agentic Autoregressive Diffusion for Long Video Consistency

Abstract

Synthesizing consistent and coherent long video remains a fundamental challenge. Existing methods suffer from semantic drift and narrative collapse over long horizons. We present A²RD (/ɑːrd/), an Agentic Auto-Regressive Diffusion architecture that decouples creative synthesis from consistency enforcement. A²RD formulates long video synthesis as a closed-loop process that synthesizes and self-improves video segment-by-segment through a Retrieve–Synthesize–Refine–Update cycle. It comprises three core components: (1) Multimodal Video Memory that tracks video progression across modalities; (2) Adaptive Segment Generation that switches among generation modes for natural progression and visual consistency; and (3) Hierarchical Test-Time Self-Improvement that self-improves each segment at frame and video levels to prevent error propagation. We further introduce LVBench-C, a challenging benchmark with non-linear entity and environment transitions to stress-test long-horizon consistency. Across public and LVBench-C benchmarks spanning one- to ten-minute videos, A²RD outperforms state-of-the-art baselines by up to 30% in consistency and 20% in narrative coherence.

Terminology

Method Overview

A²RD enables video diffusion models to synthesize and self-improve long videos autoregressively, enforcing temporal consistency and narrative coherence. A²RD is training-free and built upon three pillars:

• Multimodal Video Memory: Existing methods store only visual references, losing narrative context over long horizons. A²RD stores structured contexts from synthesized segments, disentangling each segment into three modalities: Textual States (entity identities, attribute changes, motions, spatial relations, camera trajectories), Frames (global references and boundary keyframes), and Videos (full segments for motion continuity). Online Retrieve and Update operations are enabled for synthesis.
• Adaptive Segment Generation: Prior studies adopt either extrapolation or interpolation as a fixed generation mode. Extrapolation enables natural progression but risks semantic drift; interpolation enforces stronger consistency but risks unnatural video progression when end frames are poorly planned. A²RD adaptively selects the mode per segment to enable both natural video progression and strong consistency enforcement.
• Hierarchical Test-Time Self-Improvement (HITS): A single inconsistent frame can cascade artifacts across the entire horizon. Existing video refinement methods operate only on short clips. A²RD introduces HITS to self-improve long videos hierarchically — first boundary frames, then full segments — focusing on intra- and inter-segment coherence, and video quality to combat errors propagate uncorrected.

The workflow proceeds in two stages:

• Memory Initialization: The agent reasons over the narrative to identify entities and environments, constructs a dependency graph, and synthesizes global reference frames as a form of long-term memory.
• Autoregressive Segment Synthesis & Self-Improvement: For each segment, the agent retrieves context from memory, selects the generation mode, synthesizes boundary frames and video, applies HITS, and updates memory before advancing.

Benchmark: LVBench-C

We introduce LVBench-C (Long Video Bench-Challenge), a challenging benchmark designed to stress-test temporal consistency under complex scenarios where entities and environments appear, disappear, and reappear across long horizons with optional state changes. LVBench-C features multi-shot stories at 3-minute, 5-minute, and 10-minute scales with rich non-linear entity and environment transitions.

SOTA Segment-Based Long Video Synthesis Baselines

Single-scene (VBench-Long): A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.

Direct Prompting, Naive Parallel, Naive Autoregressive, MovieAgent (Wu et al., 2025), ViMax (HKUDS, 2025), VideoMemory (Zhou et al., 2026), A²RD-Par (Ours), A²RD (Ours)

Multi-scene (LVBench-C, 3 minutes, The Scuba Diver's Reef Exploration): Prompt below.

Direct Prompting, Naive Parallel, Naive Autoregressive, MovieAgent (Wu et al., 2025), ViMax (HKUDS, 2025), VideoMemory (Zhou et al., 2026), A²RD (Ours)

A²RD Single-Scene/Multi-Scene Long Video Gallery

A²RD Multi-Scene Ultra-Long Video Gallery

(a) 3-minute: The Master Potter's Creation

1. In a quiet morning living room, a man with a grey ponytail puts on a clean navy blue apron.
2. He walks into his kitchen and packs a small wooden crate with various carving tools.
3. He exits his house and walks down a cobblestone alleyway toward his art studio.
4. Inside the bright studio, he approaches a large bag of wet, grey clay and cuts a large chunk with a wire.
5. He carries the heavy clay to a pottery wheel and slams it down onto the center of the bat.
6. The man sits at the wheel and begins centering the clay, his hands quickly becoming coated in thick, wet slip.
7. As the wheel spins, he pulls the clay upward, forming a tall, elegant vase shape.
8. He picks up a wet sponge and smooths the exterior of the vase, grey water dripping onto his apron.
9. He uses a metal rib tool to shave the sides, creating a pile of clay shavings around the base.
10. The man stops the wheel and uses a thin wire to carefully slice the vase off the spinning head.
11. He carries the wet vase into a drying room filled with wooden shelves and sets it down gently.
12. He walks to a workbench and picks up a leather-hard bowl from the previous day to begin carving.
13. He uses a fine needle tool to etch intricate patterns into the bowl, clay dust settling on his arms.
14. The man carries the carved bowl into a kiln room and carefully places it inside the large industrial kiln.
15. He adjusts the digital settings on the kiln and presses the start button to begin the firing process.
16. He walks to a glazing station and stirs a bucket of deep blue glaze with a wooden stick.
17. He dips a finished, fired plate into the blue liquid, his fingers getting stained with the pigment.
18. He sets the glazed plate on a rack to dry, looking at the transformation of the surface.
19. The man walks to a large utility sink and begins scrubbing the thick clay from his hands and forearms.
20. He removes the navy blue apron, which is now heavily stained with grey clay and blue glaze spots.
21. He hangs the apron on a wall hook and picks up his wooden crate of tools.
22. He walks back through the cobblestone alley as the evening streetlamps flicker on.
23. Entering his home, he places the tool crate on the table and sighs with satisfaction.
24. He stands in his living room stretching leisurely, a look of deep satisfaction on his face.

(a) 3-minute: The Scuba Diver's Reef Exploration

1. A diver stands on the deck of a boat in the ocean.
2. The diver is wearing a black neoprene wetsuit.
3. The diver puts on a heavy air tank and harness.
4. The diver fastens a weight belt around their waist.
5. The diver sits on the edge of the boat deck.
6. The diver pulls a rubber mask over their eyes.
7. The diver puts a regulator mouthpiece in their mouth.
8. The diver falls backward into the blue water.
9. The diver sinks beneath the surface of the ocean.
10. Bubbles rise from the diver's regulator as they breathe.
11. The diver swims down toward a colorful coral reef.
12. The diver sees a school of bright tropical fish.
13. The diver hovers near a large sea turtle.
14. The diver checks the air pressure gauge on their tank.
15. The diver begins to swim slowly back to the surface.
16. The diver breaks the surface of the water.
17. The diver swims to the ladder on the side of the boat.
18. The diver climbs up the ladder onto the deck.
19. The diver removes the rubber mask from their face.
20. The diver takes the regulator out of their mouth.
21. The diver removes the heavy air tank and harness.
22. The diver enters the boat's cabin and changes into dry clothes.
23. The diver hangs the wet wetsuit on a drying rack.
24. The boat begins to drive back toward the harbor.

(b) 5-minute: The Stage Fright (Clara)

1. Clara, wearing an oversized wool sweater and glasses, sits at a piano in a dusty attic.
2. Her hair is messy and tied back with a simple rubber band as she hums a melody.
3. She stops to scribble notes onto a piece of crumpled sheet music with a pencil.
4. Clara wipes a layer of dust off the piano keys, her fingers trembling slightly.
5. The grand theater lobby is filled with socialites in tuxedos and evening gowns.
6. Ushers in gold-trimmed uniforms hand out glossy programs to the arriving guests.
7. A large poster in the lobby features a silhouette of a pianist with the name 'CLARA' in bold.
8. Stagehands move a massive black grand piano into the center of the stage.
9. The conductor of the orchestra adjusts his baton, looking at his pocket watch.
10. The audience begins to file into the rows of red velvet seats, whispering in anticipation.
11. [Full 40-scene narrative continues...]

(c) 10-minute: The Great Museum Heist

1. Victor and Saffron sit in a dim basement, wearing casual hoodies and jeans.
2. They study a holographic blueprint of the Royal Museum, glowing blue on the table.
3. Saffron points to the laser grid in the North Gallery, her eyes narrow and focused.
4. Victor checks the internal mechanism of a miniature glass-cutting device.
5. They clink two mugs of cold coffee together, finalizing their silent pact.
6. The Royal Museum stands majestic under the moonlight, guarded by tall stone lions.
7. A security guard walks his patrol, the beam of his flashlight cutting through the dark.
8. The museum's grand clock strikes midnight, the sound echoing through the empty streets.
9. [Full 80-scene narrative continues...]