Back to Devoured - October 21, 2026
DEVOURED
Voxtral 3: Voice Agent SDK & Speech Models in 50+ Languages

Voxtral 3: Voice Agent SDK & Speech Models in 50+ Languages

AI Mistral
Mistral's Voxtral 3 introduces a multilingual voice agent SDK that integrates speech-to-text, reasoning, and text-to-speech into a single low-latency streaming pipeline.
What: Voxtral 3 supports 50+ languages with state-of-the-art transcription and expressive text-to-speech. The SDK allows agents to perform multi-step tool calls, verify identities, and handle booking tasks via SIP, WebRTC, or browser interfaces, with deployment available on Mistral's EU-hosted API or on-premises.
Why it matters: This indicates a shift toward end-to-end sovereign voice AI, where companies can process sensitive audio entirely within their own infrastructure while gaining conversational capabilities previously restricted to proprietary, cloud-only black boxes.
Deep dive
  • Voxtral Small Transcribe 3: State-of-the-art multilingual ASR with context biasing and diarization.
  • Voxtral Mini TTS 3: Expressive, context-aware speech generation supporting 50+ dialects and custom voice cloning (30 mins of audio).
  • Voice Agent SDK: Built on LiveKit, unifying listening, reasoning, and speaking into one sub-500ms pipeline.
  • Sovereignty: Full on-premises deployment options enable data control for regulated industries like contact centers.
  • Cost: TTS pricing is $13/1M characters, significantly lower than OpenAI or ElevenLabs alternatives.
  • Functionality: Natively handles end-of-turn detection and emotional adaptation based on conversational context.
Decoder
  • Diarization: The process of identifying "who spoke when" in an audio recording.
  • Context biasing: Providing a model with domain-specific keywords or phrases to increase the probability that it recognizes those specific terms during transcription.
  • Prosody: The patterns of stress, timing, and intonation in speech that convey emotion and intent.
  • SIP (Session Initiation Protocol): A protocol used for signaling and controlling multimedia communication sessions in VoIP telephony.
Original article

Voxtral 3

Voxtral 3 introduces a cutting-edge voice agent SDK and state-of-the-art speech models designed for real-time, multi-step conversations across 50+ languages. It combines accurate transcription, natural speech synthesis with emotional adaptability, and deep tool integration for seamless workflow automation. Available via Mistral’s EU-hosted API or self-deployment, it ensures data sovereignty while delivering low-latency, human-like interactions for contact centers, virtual assistants, and hands-free copilots.

Speech transcription and voice generation have come a long way, but building end-to-end voice agents that can listen, reason, interact, and handle complex workflows remains difficult.

A reliable voice agent needs to do four things very well (and simultaneously):

  • Listen accurately across languages, accents, speakers, and interruptions,
  • Reason deeply about what users mean in specific contexts,
  • Speak naturally with emotion and expression, and
  • Take action over multiple steps through connected systems.

To address this, we introduce Voxtral 3: a new family of state-of-the-art speech-to-text and text-to-speech models, alongside a voice agent SDK that brings listening, thinking, tool calling, and speaking into a single real-time streaming pipeline.

Voxtral 3 is built to create natural conversational experiences, from contact-center agents and service desks to virtual assistants and hands-free copilots. With support for dozens of languages, expressive speech, complex reasoning, and deep integration with your own tools and data, agents built with Voxtral can interpret nuanced intent, reason through sophisticated requests, and execute multi-step actions across connected systems.

Highlights

  1. Accurate transcription across 50+ languages. The speech-to-text model, Voxtral Small Transcribe 3, is state-of-the-art in multilingual transcription, identifying who said what (diarization), using keyphrases to improve its accuracy (context-biasing). It detects when someone has finished speaking so agents can respond without delay (end-of-turn detection).
  2. Natural speech that adapts to the conversation. The text-to-speech model, Voxtral Mini TTS 3, introduces new voices that adjust their emotion, pace, and emphasis based on conversational context, with support for 35+ languages and 50+ dialects.
  3. Human-like intelligence at your (spoken) command. The new voice agent SDK combines listening, reasoning, tool calling, and speaking into a real-time experience. Agents can retrieve information, interact with business systems, and complete tasks during live conversations over phone or web.
  4. Pro Voice for distinctly accurate voice adaptation. For professional use cases where basic voice cloning does not suffice, Voxtral’s Pro Voice adaptation is able to reproduce a speaker’s distinctive character, emotional range, and speaking style across languages, using just 30 minutes of audio.
  5. Your voice AI, on your infrastructure. Use Mistral's EU-hosted API or deploy Voxtral 3 on your own infrastructure. Keep sensitive conversations within your environment and control whether any audio is retained.

It hears 50+ languages, with state-of-the-art accuracy.

Voxtral Transcribe 3 represents a significant milestone across transcription, diarization (who spoke when) and context biasing capabilities. The supported list of languages is expanded to 50, including 24 official EU languages and many low-resource Asian and Indic languages.

The Small version achieves state-of-the-art results across diarization and multilingual transcription, including in English, French and Spanish. It outperforms Whisper, Scribe v2 and Gemini 3.5 Transcribe. The Mini variant is 4B parameters, enabling fast and accurate transcriptions at a price of $0.0035/min.

Both models support context biasing, where key terms and free-form text can be given to bias the transcription and improve recall on technical terms.

It answers with the right emotion, in your or your customer’s brand's own voice and local dialect.

Voxtral Mini TTS 3.0, available in batch and realtime, achieves state-of-the-art expressiveness and instruction following for 35+ languages and 50+ dialects. At $13 per 1M characters, Voxtral TTS costs less to run than comparable models from OpenAI TTS-1, Deepgram, ElevenLabs Flash and Gemini Flash.

The speech model implicitly interprets the semantic meaning of context better, producing natural speech across a diverse range of emotions and languages. New and improved features include voice cloning from 30+ minutes of sampled audio, custom prompt instruction following (IF), contextual TTS, and a vocabulary library.

The voice agent is in control of the whole conversation.

Voxtral 3 introduces a voice agent SDK, built on LiveKit, that runs speech in, reasoning, and speech out as one streaming pipeline. Voxtral Mini Transcribe Realtime 3 listens and detects when the caller has finished speaking, a general model—like Mistral Small—reasons and calls tools, and Voxtral TTS replies in the tone of the conversation so far. No stage waits for the one before it, so each turn completes in under one second, and the agent stops talking when the caller cuts in.

During a call, the agent can verify a caller's identity, check a bill, or book an appointment, using function calling, web search, and retrieval over your own systems.

Explore all 55 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi