MiniMax-H3 on 8×H200: 1.95× Lossless, Up to 6.24× at 0.76–0.91 SSIM

TL;DR

We benchmarked MiniMax-H3 video generation on 8× NVIDIA H200 with SGLang Diffusion, holding prompts, seeds, resolution, frame rate, and denoising steps fixed across six workloads.

• SGLang's dense, lossless path is 1.85–1.95× faster than Diffusers with no approximation: the same denoising work, on a faster runtime.
• Stacking step reuse and sparse attention reaches up to 6.24×, at 0.76–0.91 mean SSIM. The fastest tested profile, SubBlock 0.80 + Cache-DiT stride, delivers 5.06×/5.72× on 5 s/10 s T2VA and 5.86×/6.24× on FL2VA. The cost is not uniform: FL2VA holds 0.85–0.91 SSIM there, while T2VA drops to 0.76–0.78.
• For a quality-first default, use Cache-DiT alone (up to 2.99×, mean SSIM 0.90–0.92). For a balanced trade-off, SubBlock 0.75 + Cache-DiT stride gives 4.90–5.93× at SSIM 0.79–0.90.
• The gains come from three layers that compose: fused kernels, step reuse (Cache-DiT skips redundant denoising steps), and SubBlock sparse attention (NVIDIA's block-sparse forward, which cuts the cost of the steps that do run).

Background

While SGLang Diffusion already delivers a fast lossless path for MiniMax-H3, faster lossy generation of high-quality video has long been sought after by the community. Video diffusion is dominated by two costs: the denoising loop runs the same transformer dozens of times, and each step spends most of its budget in attention over a very long token sequence.

Three accelerations attack that from different directions, and they compose:

• Fused kernels cut the fixed cost of every step without changing its math.
• Cache-DiT reuses results between denoising steps, so some steps never run.
• SubBlock sparse attention reduces the cost of the steps that do run, by skipping attention blocks whose contribution is below a threshold.

Detailed Results

T2VA

FL2VA

Where the Speedup Comes From

Fused kernels reduce the cost of each step that still runs. The H3 path fuses indexed AdaLN updates, gated residuals, SwiGLU activation, and QK RMSNorm with 3D RoPE.

Cache-DiT attaches one DBCache context to MiniMax-H3's shared DiT block stack. After the warmup steps, it evaluates the configured boundary blocks and compares the normalized residual change with the previous cached state. All cache modes use Fn=1, Bn=0, and four warmup steps.

SubBlock sparse attention reduces the KV blocks read on computed steps. It uses n_k=n_q=4; the first ten denoising steps use dense attention, and SubBlock is enabled afterward. The minimum sequence length is 4096.

The Kernel Layer

How SubBlock Sparse Attention Works

SubBlock is a training-free router for block-sparse attention. It divides the sequence into 64-token query and key blocks, then splits each block into four 16-token sub-blocks on both sides. A lightweight pooling and log-sum-exp score estimates each key block's unnormalized softmax mass for each query block and head. The router keeps the highest-scoring key blocks and passes their indices to the block-sparse attention kernel.

Acknowledgement

• SGLang Diffusion Team — wrote the first version of this post, drives the SGLang kernel work these results build on, and provides the diffusion runtime, the fused kernels, and the parallelism measured here.
• Ji Huang (@IPostYellow), Ant Group — ran the H200 benchmark, brought SubBlock sparse attention into SGLang Diffusion, and revised this post.
• Cache-DiT Team — @DefTruth and the vipshop.com team, for Cache-DiT and for support integrating its cache profiles into SGLang Diffusion.
• MiniMax — for open-sourcing MiniMax-H3, the model every measurement here runs on.
• NVIDIA — for the underlying SubBlock sparse attention support, including the block-sparse attention forward these results depend on.