Today, we’re publicly launching the AirCaps Audio Research Lab.

Real-world audio is 10x harder to understand than the digital world, and voice is far from “solved.”

We’re building fully streaming speech and audio models for the physical world: target speaker extraction, far-field enhancement, diarization, separation, and speech-to-text.

Our models are a step function ahead of leading cloud speech models in the hardest acoustic environments, while running on edge on consumer hardware.

See real demos: https://research.aircaps.com

AirCaps Audio Research Lab

Source: https://research.aircaps.com/

Giving machines human-level audio and speech perception.

AirCaps Audio Research Lab explores how intelligent systems can better understand and interact with complex, real-world acoustic environments in real time.

Our work spans multichannel audio processing, spatial filtering, source separation, sound localization, target-speaker extraction, automatic speech recognition, and speech synthesis.

We focus specifically onmessyaudio found only in the physical world: far-field, reverberant, low signal-to-noise ratio (diffuse and directional noise), overlapping speech (babble), and multi-speaker scenarios.

1. 01 ### Real-worldaudio is 10x harder to understand than the digital world. We’ve been developing voice technologies for consumer electronics for 8 years. Before the era of ElevenLabs, or even OpenAI. We realized early on that all the leading speech models (at the time, Google and Microsoft) were optimized for the “digital” realm: transcripts for call centers, meeting notes for Zoom calls, captions for podcasts and movies. As soon as you deploy them in the “messy” physical world—far-field, reverberant, low SNR, overlapping speech—they degrade rapidly. Part of the problem is the scarcity of real-world, noisy audio data. The largest datasets and evaluation benchmarks for speech models are not even close to the complexity of the real world. A sample fromLibrispeech, the de-facto reference benchmark for ASR. Carefully segmented, clean, single-speaker audiobook speech. Real-world sample from our proprietary dataset. Background noise, distance, interfering speakers all compete with the target voice. The gap remains surprisingly wide - Ayllon et al. (2026) evaluated leading 40 ASR models and found that median word error rate (WER) reached 20.45% on speech with background noise - the same models report 2-3% on clean benchmarks. This study also found evidence of optimization for benchmarks (“benchmark-maxxing”).[1]
2. 02 ### One model is not enough. Speech recognition (“ASR” or “STT”) answers one question: what words were spoken? Simply slapping an STT model onto a machine does not make it good at comprehending real-world audio. A useful system must also infer who spoke, where the voice is, whether it was directed at the device, which speaker to focus on, all while enhancing speech and suppressing interfering noise. This is known as the “cocktail-party problem.” Humans can do this in a few milliseconds. Machines have not been able to do this well, despite efforts since 1953[2]. We don’t just develop STT models. We also build dedicated models for: - Target speaker extraction - Directional speech enhancement - Speaker diarization - Speaker identification - Sound source localization (SSL) - Continuous moving speaker tracking - Overlapping speech separation - Speech enhancement and dereverberation
3. 03 ### Build forstreamingfrom the ground up Human-machine interaction cannot wait for an upload. A real-time system must be causal: at timet, it can use only samples at or beforet. It never sees future context that would make the current decision easier. Thus, batch (async / offline) model accuracy therefore overstates what an interactive (streaming) system can do. Causality changes architecture, training, and evaluation. Every stage must keep bounded state, recover continuously, and process audio faster than it arrives (a real-time factor “RTF” below 1.0) to keep end-to-end latency low enough for natural turn-taking.
4. 04 ### Theedgeis ready. Intelligent machines should not lose the ability to communicate when they lose internet. Your car’s voice assistant shouldn’t stop working in a tunnel; voice dictation on a smartphone shouldn’t be handicapped when you’re on a hike. Nor should every spoken interaction incur a metered charge per word. Edge intelligence used to lose on two fronts: devices lacked compute, and capable models were too large. Both constraints are moving quickly. Apple’s A19 Pro (iPhone 17 Pro) combines a 16-core Neural Engine with Neural Accelerators in every GPU core and reports up to 40% better sustained performance than its predecessor. Novel specialized audio model architectures concentrate capacity on one task, delivering high intelligence density where the microphones are. Recent wave-based simulation methods can generate controlled mixtures of rooms, motion, reverberation, noise, and overlapping speakers, leading to higher quality training data. All our models are optimized for edge deployments on consumer grade devices (modern smartphones, laptops, wearable NPUs) while maintaining RTFs < 1. This enables private, low-latency, and cost-effective voice AI that works everywhere, anytime.

Current work

1. Noise
2. Interfering speech
3. Multi-speaker separation

Existing providers tackle real-world environments by training an STT model on noisy audio, which works well up to a degree. But in highly reverberant, far-field (when the target sound is far from the microphone) noisy conditions (SNR < 5 dB), their performance degrades significantly.

Instead, we build specialized models that first process raw audio and outputenhancedaudio before feeding it to the STT model. We train the enhancement and STT models jointly. Our models utilize spatial cues from multi-channel audio signals to “zoom in” on speech frequencies from a noisy mixture. Watch the difference for yourself:

Often, a machine must be able to distinguish desired speech (a user) vs unwanted speech (e.g., a group sitting at nearby table at a restaurant, a TV playing in the background). Existing models are trained to simply focus on the most “dominant” voice in the mixture, which leads to either jumbled transcripts or missing the target speaker completely.

This is a hard problem. Another voice is harder to remove than ordinary noise: it has the same structure as the signal we want to keep.

Our work combines spectral (what the voice sounds like) and directional (where the sound is coming from) information to isolate a target voice and suppress competing speech.

03

Multi-speaker separation

In many cases, simultaneous overlapping speech must not be discarded as all participants are targets. Across natural conversations, simultaneous speech occupies as much as 20.8% of total conversation time[3].

Group meetings lose interruptions, confirmations, and decisions; interviews and live panels lose the quieter speaker; cars, smart rooms, and multi-user voice agents discard target speakers entirely. A single isolation or suppression model would extract only a single target, what is needed here is multi-speaker separation.

Our models separate multi-speaker mixtures into stable, speaker-consistent streams so transcription, speaker identification, and intelligent systems can retain and attribute several target voices.

Voice interface for home robotics

Models used: Noise suppression, speech enhancement, target speaker isolation, speech-to-text

Robust wake-word detection

Models used: Background speech suppression, target speech enhancement, speech-to-text

Multi-user machines

Models used: Sound source localization, directional speech isolation, speaker identification and verification, speech-to-text

Automotive: in-cabin voice interfaces

Models used: speech enhancement & dereverberation, noise suppression, directional speech extraction, speaker verification, personalized voice activity detection, speech-to-text