DEVOURED
DFlash 2: Keep Drafting Parallel
Inco AI's DFlash 2 enables speculative decoding that is 3x faster than standard inference by introducing parallel path selection.
What: DFlash 2 improves upon the original DFlash speculative decoding technique by adding a 'path selector' and dynamic short convolutions. This allows the drafter model to pick more coherent token sequences in parallel without autoregressive bottlenecks. It is compatible with engines like SGLang, vLLM, and llama.cpp, providing a 16-25% improvement in token acceptance over the previous version.
Why it matters: This reflects the move toward making LLM inference cheaper by optimizing the 'speculative' stage, where small models guess token sequences that large models verify, essentially reducing the amount of high-compute work required per generated word.
Takeaway: If you are serving LLMs with vLLM or SGLang, you can test the DFlash 2 drafter by updating to the latest GitHub branches (e.g., vLLM pull request #52816) and using the provided Hugging Face model IDs.
Deep dive
- DFlash 2 maintains a one-pass parallel design while resolving coherence issues.
- A lightweight selector scores adjacent token pairs to ensure the drafted block is logically sound.
- Dynamic depthwise convolutions mix information between token positions, addressing the 'suffix decay' common in earlier parallel drafters.
- The method requires only 16.5M additional parameters (approx. 3% overhead).
- Performance benchmarks on models like Qwen3.8-27B demonstrate 2.7-3.4x throughput increases compared to standard autoregressive decoding.
Decoder
- Speculative Decoding: An optimization technique where a smaller, faster 'draft' model predicts a sequence of tokens, which a larger 'target' model then verifies in a single forward pass.
- Autoregressive Decoding: The standard inference method where a model generates one token at a time, using that token to predict the next.
- Inference: The process of running data through a trained model to make predictions or generate text.
- Drafter: The smaller, faster model used in speculative decoding to propose potential output sequences.
Original article
Full article content is not available for inline reading.
Explore all 57 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi