DEVOURED
Kev (GitHub Repo)
Kev provides small, local decision models built on Qwen3.5 that return calibrated probabilities for classification and scoring tasks.
What: Jared Palmer released Kev, a family of 0.8B, 4B, and 9B parameter models that run locally on CUDA, ROCm, or Apple Silicon. The models use a pointer-head architecture to output probabilities for yes/no, multiple-choice, and rating questions independently, matching the TypeSafe System One API.
Why it matters: This project highlights a shift toward using small, task-specific models for automated decision-making rather than relying on massive general-purpose LLMs, particularly when calibration and latency are paramount.
Takeaway: If you need to automate routing or escalation logic with probabilistic confidence, test the Kev-4B model locally using `uv run -m kev.serve --run jaredpalmer/kev-4b`.
Deep dive
- Features 0.8B, 4B, and 9B models based on Qwen3.5.
- Uses a rank-16 LoRA adapter and pointer head for decision tasks.
- Supports independent question processing without context contamination.
- Provides calibrated probabilities instead of just top-k labels.
- Runs on Apple Silicon via MLX and on GPUs via PyTorch/Flash-Linear-Attention.
- Offers a playground for testing option order and delimiter effects.
- Includes tools for fine-tuning on custom domain data using a JSONL workflow.
- Optimized for low-latency inference, with sub-50ms latency for many requests.
Decoder
- Brier score: A metric that measures the accuracy of probabilistic predictions by calculating the mean squared difference between predicted probabilities and the actual outcome.
- Calibration: The alignment between a model's predicted probability (e.g., 80% confidence) and its actual success rate (e.g., being right 80% of the time).
- Pointer Head: A neural network architecture component that selects between specific output options by comparing their hidden states against a decision token.
- LoRA (Low-Rank Adaptation): A parameter-efficient fine-tuning technique that freezes the base model and trains only small, rank-decomposition matrices.
Original article
Full article content is not available for inline reading.
Explore all 44 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi