Speaking is one of the fastest ways to communicate with a computer, but it comes with an obvious drawback: everyone nearby can hear what you are saying. Whether you are dictating notes in a coffee shop, issuing commands on a train, or asking an AI assistant sensitive questions at work, privacy disappears the moment you open your mouth. Researchers have explored everything from lip reading to electromyography (EMG) in an attempt to solve that problem.
Now, a team at a research lab called Aleph has demonstrated a different approach that looks beneath the surface by watching the tongue. Rather than relying on cameras or electrical signals from facial muscles, the system places a small ultrasound probe beneath the speaker’s chin to capture video of tongue movements while the person silently mouths words. Those images are then processed by an AI model that reconstructs what was intended to be said without requiring any audible speech.
Despite being an early research project, the results are already looking positive. The team reports a 15.6% word error rate on open-vocabulary speech, approaching the 12.5% error rate achieved by state-of-the-art lip-reading systems that were trained on roughly one million hours of data. Aleph’s model, by comparison, was trained using just 50 hours of ultrasound recordings collected over the course of about a month.
A major reason for the system’s excellent performance is that ultrasound observes one of the primary articulators of speech directly. While the lips can only form around 10 to 14 visually distinguishable shapes, the tongue is involved in approximately 34 distinct phoneme classes across the 40 phonemes used in English. That richer information gives the AI much more data to work with than a conventional camera looking only at a speaker’s face.
To build the dataset, participants were asked to read synthetically generated short stories aloud while holding the ultrasound probe under their chin. Speaking audibly allowed the researchers to verify that each sentence was read correctly using speech transcription, while real-time quality monitoring ensured that the tongue remained visible and the probe stayed properly positioned. The researchers observed that tongue movements during spoken and silent speech were similar enough that models trained on vocalized speech generalized well to silent speech.
The processing pipeline is made up of several established AI models combined in a novel way. A ResNet-18 video encoder learned to interpret the ultrasound footage, while OpenAI’s Whisper Base speech model provided a pretrained decoder capable of turning speech-like embeddings into text. By teaching the video encoder to produce embeddings similar to those generated from corresponding spoken audio, the system gradually learned to translate tongue motion into words.
The work is still a prototype that is not yet ready for everyday use. As it stands, the setup requires an ultrasound probe and coupling gel beneath the chin, though the researchers believe both can eventually be replaced with smaller wearable hardware and a hydrogel. Still, the team has already shown that new users can successfully use the technology without personalized training, suggesting that silent, private conversations with AI assistants may be closer than they appear.This setup allows computers to recognize silent speech (📷: Aleph)
The training approach (📷: Aleph)
The model can learn from small datasets (📷: Aleph)
Source link
