Back to Devoured - November 12, 2026
DEVOURED
Don't Write Kernels, Derive Them

Don't Write Kernels, Derive Them

AI Muni.works
Tang replaces hand-written operator kernels with an IR that derives and fuses kernels on-the-fly, allowing performance parity with expert-tuned engines.
What: Tang's tensor IR uses views and comprehensions to derive fused kernels at runtime. It matches the performance of hand-tuned engines like those for 80B MoE models on H200s by replacing operator-specific code with algebraic rewrites.
Why it matters: The shift from manual kernel writing to derived kernels moves the burden of optimization from humans to compilers, enabling better performance across diverse hardware without specialized library maintenance.
Deep dive
  • IR constructs: Views (affine maps) and Comprehensions (reductions over a monoid)
  • Derived kernels outperform hand-written ones in specific regimes by eliminating materialized intermediate buffers
  • Fusion is treated as a single substitution rewrite rule; flash attention is derived, not hardcoded
  • Backward pass is generated automatically via reverse-mode differentiation of comprehensions
  • Supports multiple targets: CUDA, WGSL, Metal, C, OpenCL, and GLSL
  • Validated on M4 Max, RTX 3090, H200, and Android-based humanoid robot SoC
Decoder
  • Monoid: A mathematical structure used here to define how reduction operations (like sum or max) are combined.
  • Affine Map: A transformation used in views to express memory access patterns like transposes or slices without explicit pointer arithmetic.
  • FlashAttention: A memory-efficient attention mechanism that computes attention scores in blocks to minimize memory traffic.
Original article

Full article content is not available for inline reading.

Read the original article →

Explore all 61 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi