DEVOURED
The Counting House
The 'Counting House' pattern offers a deterministic, stateless alternative to agentic software factories for handling high-volume enterprise customer service workflows.
What: Author Jeff Yegge introduces 'Counting House,' a design pattern where AI models act as stateless, transient clerks for specific tasks. It uses rigorous data science evals, confusion matrices, and the Saga pattern via Temporal to manage costs and quality for massive customer service workloads.
Why it matters: This approach represents a shift from 'YOLO mode' agent systems to hardened, measurable pipelines, addressing the enterprise need for auditability, regulatory compliance, and cost control in AI integration.
Takeaway: If you are managing high-volume tasks, build a 'policy table' of intents and score your models against a human-verified 'gold set' before moving them to production, rather than attempting end-to-end automation.
Deep dive
- Stateless clerks: Models handle exactly one request before exiting, preventing credential exposure and drift.
- Policy table: A configuration-driven approach defining which models handle specific 'intents'.
- Autonomy tiers: Progression from 'shadow' (silent evaluation) to 'transact' (state-changing with orchestration).
- Gold set: A collection of 200-1,000 human-scored cases per intent used for validation.
- Confusion matrix: A tool to measure model mistake costs, prioritizing 'customer-harm' errors over simple accuracy.
- Orchestration: Using durable execution (e.g., Temporal) to handle multi-step sagas instead of allowing models to write directly to systems of record.
- Model welfare: Implementing 'clerk welfare' principles (clear roles, truth in prompts) reduces token waste and improves output reliability.
Decoder
- Saga pattern: A design pattern for managing distributed transactions by breaking them into a sequence of smaller, local transactions with compensating actions to revert changes if any step fails.
- Intent: A specific, granular category of customer contact (e.g., 'flight status query' vs. 'rebooking request').
- Gold set: A curated subset of historical data manually scored by experts to serve as the ground truth for evaluating AI performance.
- Idempotency key: A unique value that ensures a process (like a refund) can be safely retried without performing the same action multiple times.
Original article
Full article content is not available for inline reading.
Explore all 40 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi