DEVOURED
SafeChat: Building AI-Powered Safety Systems at Scale in a Real-Time Marketplace (42 minute video)
DoorDash replaced an expensive LLM-only moderation pipeline with a hybrid system using fast internal models to filter safe content, cutting verbal-abuse incidents by 50%.
What: Bruna Pereira, a software engineer at DoorDash, described a content-agnostic moderation platform where a small, fast internal classifier acts as a pre-filter, while an LLM scores nuanced content across multiple axes. This system processes millions of daily chat, voice, and image interactions while providing configurable, no-code workflows for other internal teams.
Why it matters: This validates the 'human-in-the-loop' architectural pattern where large language models are reserved for complex decision-making, while high-throughput traffic is handled by purpose-built, smaller models to optimize latency and operational costs.
Takeaway: When building moderation systems, avoid requesting boolean labels from LLMs; instead, ask for numerical scores to enable granular, threshold-based automated actions.
Deep dive
- Use a tiered architecture: small internal models for obvious cases, LLMs for high-severity nuances.
- Optimize cost and latency by processing 90%+ of traffic through lightweight models.
- Implement a 'score' rather than a 'boolean' to allow flexible, graduated responses (e.g., censoring vs. blocking).
- Build a platform-agnostic orchestrator so internal teams can configure their own moderation logic without custom code.
- Use backtesting on historical data before deploying new moderation rules to production.
- Ensure multi-modal support by standardizing ingestion for voice, image, and text pipelines.
Decoder
- Graduated action: A policy where the system response scales with the severity of a violation (e.g., from a warning to an account ban).
- Moderation agent: A configurable workflow pipeline within the platform that executes specific safety rules on content.
Original article
Full article content is not available for inline reading.
Explore all 59 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi