Kev
Jared Palmer released Kev, a family of four open-source 'decision models' that score answer options without generating text.
Deep dive
- Kev uses a 'pointer head' architecture that scores multiple-choice options instead of generating tokens.
- The model family includes 0.8B, 4B, 9B, and 27B parameter variants.
- Kev-27B leads overall performance, while smaller models are optimized for consumer complaint classification and developer tool workflows.
- Supports MLX for running on Apple Silicon.
- Caching mechanism reads documents once and branches computation for multiple questions, significantly reducing latency for multi-question tasks.
- Fine-tuning tools are provided to calibrate models on domain-specific support tickets or structured data.
- 27B model requires 80 GB+ VRAM (H100/H200/B200) for long contexts; smaller models fit on consumer GPUs or 32GB Mac hardware.
Decoder
- Decision Model: A specialized language model trained to output probability scores for discrete choices (e.g., yes/no, classification) rather than generating conversational text.
- Pointer Head: A neural network layer that maps model representations to specific output choices by assigning probabilities to each option.
- Calibration: The measure of how closely a model's predicted confidence matches its actual accuracy on tasks.
Original article
Today I'm releasing Kev 1.0, a family of four open source decision models, from 0.8B to 27B parameters. This release includes a new Kev-27B and an updated Kev-9B.
TypeSafe's Jev takes text, questions, and possible answers and returns a probability for each answer. I wanted that interface with weights I could run and fine-tune myself. Kev uses the same API, so applications built with TypeSafe's SDK can switch by changing the endpoint and model name.
All four models have Apache-2.0 weights on Hugging Face. The code is on GitHub, including a server that runs on GPUs and on Apple Silicon through MLX. You can try Kev-4B in your browser or use the agent skills to deploy and fine-tune your own.
The Kev family
Kev-27B leads overall in my evaluations. On consumer complaints, Kev-9B and Kev-4B are within a percentage point of it; on developer-tool decisions, Kev-9B matches it. The table separates tasks excluded from Kev's fine-tuning from held-out examples of task families it trained on.
Overview of accuracy across six evaluation panels for Kev-27B, Kev-9B, Kev-4B and Kev-0.8B. Kev-27B leads overall; Kev-9B and Kev-4B are within one percentage point of it on consumer complaints, and Kev-9B matches it on developer-tool decisions. All results are from the author's evaluation harness, not an independent leaderboard.
| Kev-27B | Kev-9B | Kev-4B | Kev-0.8B | |
|---|---|---|---|---|
| Tasks excluded from Kev's fine-tuning | ||||
| Generalization14 public datasets, 3,089 test questions | 75.7% | 69.8% | 69.0% | 58.6% |
| Short-text decisionssources excluded from fine-tuning, 656 development questions | 85.1% | 82.0% | 81.7% | 64.8% |
| Knowledge and evidenceexams, buried facts and missing evidence, 1,046 development questions | 82.0% | 78.0% | 77.4% | 58.6% |
| Held-out examples of trained task families | ||||
| Policy and rule reasoningpolicies, logic, numbers and missing evidence, 1,088 test questions | 91.8% | 83.4% | 80.3% | 66.5% |
| Sorting consumer complaintsproduct and issue, 936 questions | 90.8% | 90.0% | 90.3% | 85.1% |
| Developer-tool decisionscode review, commits and tools, 1,071 questions | 79.0% | 79.1% | 75.6% | 63.7% |
I recommend Kev-27B if you want the highest overall scores and have an 80 GB H100 for shorter inputs, or an H200 or B200 for long documents. Otherwise, start with Kev-4B and measure it on your own examples.
| Model | Best for | Runs on | Req/s | GPU $ / 1M | Model time |
|---|---|---|---|---|---|
| Kev-27B | Best overall results in my evaluations | H200 (or H100, B200) | 29 | $44.09 | 67 ms |
| Kev-9B | Workflow accuracy close to Kev-27B on a smaller GPU | H100 (or L40S) | 80 | $13.80 | 24 ms |
| Kev-4B | A starting point for local use and fine-tuning | L40S, or a 32 GB Mac | 51 | $10.54 | 42 ms |
| Kev-0.8B | High-throughput, narrowly defined decisions | L4, or any Apple Silicon Mac | 63 | $3.54 | 23 ms |
For local use, Kev-0.8B and Kev-4B run on Apple Silicon through MLX. On a 32 GB M5, Kev-4B answers five questions about a short text in 721 ms, or 136 ms when the text is cached; Kev-0.8B takes 149 ms for new text. Kev-9B is expected to fit a 32 GB Mac, and Kev-27B a 96–128 GB Mac. I haven't measured those two on Macs yet.
How Kev works
Kev adds a pointer head, a layer that scores the supplied answer options, to a language model. It supports yes/no questions, multiple choice, and ordered scales. The document is processed once, and each question uses that cached computation without seeing the other questions.
- A generated answer: A chat model generates a sequence of tokens, even when its output is constrained to JSON.
- Kev scores options directly: A pointer head returns probabilities without generating answer tokens.
- Kev reads the text once: The document prefix is computed once and cached.
- Every question branches off that one read: Ask three questions or thirty: the ticket is still read once.
- A probability for every answer: Each <decide> token scores the options. A softmax turns those scores into probabilities.
- Your code sets the threshold: Here, only billing clears the illustrative 80% threshold. The other decisions take the fallback path.
Caching saves more work on longer documents. On an H200, Kev-27B takes about 9.4 seconds of model time to read a new 64k-token document and about 0.7 seconds for subsequent questions using its cached state.
Training the new models
What's new in Kev-27B
The smaller Kevs use Qwen3.5 backbones with LoRA adapters: small sets of additional weights trained while the backbone stays fixed. For the new Kev-27B, I used Qwen3.8-27B, Qwen's post-trained release, and fine-tuned every weight of its text backbone.
Training the whole backbone on the previous Kev-27B's data did no better than the adapter-trained version. The new release also uses a broader corpus: about 146,000 examples and 337,000 questions, with documents up to 32k tokens. It combines Kev's existing data with licensed public tasks, synthetic decisions written by open-weight models, and documents and agent logs generated by code with exact answers.
I added examples for tone, grounding, prompt injection, and personal-data detection, and screened the training records against the frozen evaluation sets for exact and near matches. No Jev outputs were used. The model card documents the data sources and training recipe, including the final weight blend with the previous checkpoint. The full training corpus is not public.
What's new in Kev-9B
Kev-9B had fallen behind Kev-4B on policy and rule reasoning, developer tools, and consumer complaints because it hadn't received the same additional training. One more fine-tuning pass, about three hours on one GPU, closes that gap.
| Kev-9B (previous) | Kev-9B (new) | |
|---|---|---|
| Policy and rule reasoning | 58% | 83% |
| Developer-tool decisions | 64% | 79% |
| Sorting consumer complaints | 83% | 90% |
| Generalization index (0–100) | 40 | 41 |
Evaluation
Generalization
To test tasks excluded from Kev's fine-tuning, I assembled 14 public datasets spanning intent classification, retrieval, language understanding, tool selection, knowledge, and subjective judgments. They include choosing among 150 customer intents and deciding whether a contract supports a claim.
Calibration
Calibration measures whether a model's probabilities match how often its answers are correct. On the pooled 14-dataset test, Kev-27B's answers in the roughly 85%-confidence bin are correct about 85% of the time.
Fine-tuning on your data
I packaged that workflow into an official kev-finetune skill. Your coding agent helps define the questions, prepares labelled data or generates examples, fine-tunes a released Kev, fits its temperature on a separate calibration split, and compares the result with the starting checkpoint before deployment.
The companion kev-deploy skill makes it easy to deploy Kev to Modal behind an authenticated HTTPS endpoint.
Limitations
Kev only scores the options you supply. It doesn't generate explanations or retrieve missing facts. You need to supply the relevant policy or evidence and handle the possibility that none of the options is correct. Answer order can also shift probabilities slightly.
Get started
The weights are on Hugging Face, and the code, model cards, and serving instructions are on GitHub. You can try Kev-4B in your browser before setting anything up.
npx skills add jaredpalmer/kev@kev-deploy
npx skills add jaredpalmer/kev@kev-finetune
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi