Introduction - Google AI Edge Gallery now supports on-device speech recognition and TTS
Google AI Edge Gallery has added new features that let devices process speech entirely on their own. These features include automatic speech recognition (ASR) and text-to-speech (TTS). They work offline and with low delay, making interactions more natural. This update means devices no longer need to connect to the internet for voice tasks.
Running speech models locally improves privacy and reduces dependence on network speed. It also allows faster responses, especially in areas with poor internet. Developers can now build apps that transcribe speech and generate spoken replies directly on the device. This makes voice interactions more reliable and private.
Advances in speech architectures - Compact models enable accurate, multilingual speech processing
Recent progress in speech model design has made it possible to run accurate, multilingual speech recognition and synthesis on small devices. These models are lightweight but still deliver high quality results. They can understand and produce speech in many languages at a speed that feels instant.
These compact models are built to fit within limited memory. They are optimized to run efficiently on devices like phones and wearables. This means developers can add voice features without increasing app size or draining battery life. The models are designed to work well across different hardware, including devices with limited processing power.
The models are also capable of handling multiple languages at once. They can recognize speech in over 100 languages, making them useful worldwide. Their speed allows real-time transcription and speech synthesis, which is essential for natural conversations. These advances open new possibilities for voice-based apps that work offline and across many languages.
Android integration - ML Kit provides fast, accurate speech models with no APK size increase
Google’s ML Kit now includes the latest on-device speech models. These models are integrated directly into Android, making it easy for developers to add speech recognition and TTS to their apps. The models are managed by Google’s systems, so developers do not need to handle downloads or updates.
ML Kit’s speech models are designed for speed and accuracy. They can transcribe speech in over 100 languages, reaching a Word Error Rate of around 8.0% to 9.0% on a standard benchmark. They process speech ten times faster than real-time audio, with a delay of about 300 milliseconds for three seconds of speech. This quick response is important for natural conversations.
The TTS models are also fast. They start speaking in about 100 milliseconds and produce high-quality audio at 24 kHz. They can generate a 27-second speech in about 500 milliseconds and run at up to 50 times real-time speed. These models do not increase the size of the app’s APK, keeping the app lightweight. They also reduce power consumption by over 50%, which helps extend device battery life.
Cross-platform solutions - MediaPipe Tasks and LiteRT support speech on various devices and OS
For apps that run on different operating systems, Google offers MediaPipe Tasks and LiteRT. These tools support speech recognition and synthesis on platforms like iOS, Windows, Linux, and web browsers. They provide ready-to-use APIs that simplify the process of adding voice features.
MediaPipe Tasks handle complex audio processing tasks automatically. They include features like Voice Activity Detection (VAD) and support for real-time streaming. Developers can choose from open-weight models like Parakeet-TDT 0.6B for recognition and PocketTTS 100M for synthesis. These models are optimized for low latency and small memory use.
LiteRT is a high-performance inference framework that packages models into a single file format. It supports a wide range of open-weight models, including multilingual and lightweight options. Developers can deploy custom models or use community-shared models like Whisper or Qwen3. This flexibility allows building voice apps tailored to specific needs and hardware.
Both MediaPipe Tasks and LiteRT enable developers to create cross-platform voice applications that work smoothly across devices. They abstract away the low-level details of audio handling, making it easier to focus on app features. These tools help ensure consistent performance and quality regardless of the device or operating system.
XR Language Teacher showcase - Multimodal voice tutoring in real-world environments
The XR (Extended Reality) Language Teacher is a new demo that shows how voice and vision models can work together offline. It runs entirely on a device, without needing internet access. The demo helps users learn languages while traveling in real-world settings like underground stations or markets.
In this showcase, users point their camera at objects or signs and ask questions aloud. The device streams the spoken question in real time to on-device speech recognition models. It then processes the visual input and provides an explanation in the user’s language. It also pronounces foreign words with authentic accents.
The system can explain scenes, translate phrases, and switch voices seamlessly. It tags foreign-language parts of speech and switches between voices and accents during speech synthesis. This creates a natural, immersive learning experience that works anywhere, even without Wi-Fi. The demo demonstrates how multimodal models can enhance language learning in practical situations.
Streaming voice dictation and read-aloud playback in AI Chat
Google has integrated on-device speech models into its AI Chat feature. This allows users to speak and listen offline, making voice interactions more natural and private. The models support live transcription and speech synthesis directly on the device.
Users can choose different voices and preset speech settings within the app. During a chat, they can select “Auto-play TTS” to hear responses read aloud. The system highlights each sentence as it is spoken, helping users follow along. Tapping any sentence jumps to that part of the conversation.
The implementation also includes a reference codebase that shows how to handle speech output efficiently. It removes non-spoken content like code blocks and schedules synthesis at the sentence level. This prevents memory overload and ensures smooth performance during long conversations. These features make voice-enabled AI Chat more responsive and user-friendly.
Technical details - Model sizes, speeds, and open-weight options for developers
The speech models in Google AI Edge Gallery are designed for efficiency. The ASR models, like Gemini Nano, are small enough to run on most devices. They deliver high accuracy across many languages and process speech ten times faster than real time, with a delay of about 300 milliseconds.
The TTS models, such as Gemma 4, are also compact. They start speaking in about 100 milliseconds and generate high-quality audio at 24 kHz. They can synthesize a 27-second paragraph in just half a second. These models run at up to 50 times real-time speed, saving power and reducing latency.
Open-weight models are available for developers to customize or deploy on their own hardware. Examples include Parakeet-TDT 0.6B for recognition and PocketTTS 100M for synthesis. The models are packaged into a single file format, making deployment straightforward. The open ecosystem includes models from the Hugging Face LiteRT community, offering options for multilingual, lightweight, and high-quality speech processing.
Developers can choose models based on size, speed, language support, and quality. The models are optimized to work well across a range of hardware, from high-end phones to low-power wearables. This flexibility helps developers create tailored voice experiences for their users.
Background - Why local speech models matter for privacy and low latency
Running speech models on the device offers several benefits. It keeps user data private because speech does not need to leave the device. This is important for sensitive conversations or applications with privacy concerns.
Local models also reduce delays. Since speech is processed directly on the device, responses are faster. This improves the natural feel of voice interactions. It is especially useful in areas with poor internet or in situations where connectivity is limited.
Another advantage is reliability. Offline models do not depend on network quality or server availability. They work consistently in any environment, whether indoors or outdoors. This makes voice features more dependable and accessible everywhere.
Finally, local processing helps conserve bandwidth and reduces server load. This can lower costs for service providers and improve overall system efficiency. As models become smaller and more capable, local speech processing is becoming a standard feature for modern devices.
How OpenSmartRoute helps
Routing requests through OpenSmartRoute offers a way to access speech APIs quickly and reliably. It can send each request to the best-fit model or tool from a predefined catalog. The router scores options based on quality, cost, speed, and safety, and adjusts traffic accordingly.
Using OpenSmartRoute, teams can set rules to keep personal data on-premises or avoid crossing region boundaries. The system learns from outcomes, directing more traffic to models that perform well. New models can be added easily and start competing immediately. This setup helps optimize costs and performance for voice applications.
OpenSmartRoute also provides visibility into request costs and accuracy. It can measure routing quality and flag issues before deployment. For teams building voice apps with local models, it ensures requests are handled efficiently and securely, making the most of the new speech features.