Back to Devoured - September 22, 2026
DEVOURED
AI Comes for the If Statement

AI Comes for the If Statement

AI backendperformance Tom Tunguz
Specialized 'decision models' like Jev2 and SemIf replace generative AI for basic if-then logic, slashing inference costs by 99%.
What: Instead of generating text, these models run attention math once to determine the probability of specific allowed outcomes. Tests on email classification showed a jump from 47% to 80% accuracy compared to general-purpose generative models while costing significantly less.
Why it matters: This marks a move toward a bifurcated AI economy where massive frontier models are used for discovery, while highly optimized 'primitive' models handle repetitive production logic.
Takeaway: Review your application code for simple classification or if-then routing calls and consider replacing general LLM calls with a specialized decider model.
Deep dive
  • Jev2 and SemIf optimize coding primitives like if-then statements.
  • These models bypass the multi-layer feed-forward network to evaluate outputs directly from logits.
  • Achieved near 80x cost reduction for standard classification workflows compared to frontier models.
  • Demonstrates that specialized models can be more accurate than general models for narrow classification tasks.
Decoder
  • Logits: The raw output scores produced by a neural network before they are converted into probabilities or text tokens.
  • Primitive: A fundamental building block in programming, here referring to basic control flow logic like 'if-then'.
Original article

In short : Machine-native models replace human-facing text generation with zero-token typed execution, cutting inference costs by orders of magnitude for basic programming primitives.

Working in the back of a grocery store has its own set of rules : if the food is a banana, send it to produce ; if it is a cookie, the snack aisle ; if it is cumin, shelve it with the spices.

But what if the load of bananas has spoiled, the cookies have crumbled & the cumin is caked? These rules & exceptions govern every grocery store & neighborhood mart. At the beginning they are rigid ; over time, more exceptions are discovered : is that a plantain? Dubai chocolate : dessert or baking supply?

Coding literally means encoding these rules & exceptions into software. Pre-AI, these programs were rigid. AI handles the exceptions : an image search identifies the unfamiliar fruit as Musa paradisiaca, a brother of the banana.

But we do not need the world’s most brilliant model to handle if-plantain-then-produce logic.

The newest wave of AI is a robust if-then decider. Jev & SemIf answer questions like these in hundreds of milliseconds at a 99% reduction in cost compared to traditional AI. They simplify the existing models by running the attention math once & then determining the probability of each allowed answer, based on a few tokens of output.

I went looking for the if-then statements in my own code, the ones I had handed to AI. Within a few minutes I had replaced about a quarter of those calls in one of my agents. On 98 hand-verified production email threads, the specialized deciders nearly doubled the classification accuracy of the production model, jumping from 47% to over 80%.

Software is composed of primitives : the if-then statement is one of them. By optimizing this single primitive, we see nearly two orders of magnitude in cost reduction alongside higher accuracy.

These advances raise the question of which other programming primitives will benefit from the same specialization. They also highlight the bifurcating economics of AI : state of the art models for discovery & optimized models for production. We use the largest frontier models to train new models, & the most capable models to architect systems. But once a system is engineered & hardened, running it thousands or millions of times through a workflow benefits from narrower AI.

If this is the first of many primitives specialized for production, then harnesses are about to capture a lot more margin.

Explore all 44 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi