In a groundbreaking achievement, researchers at the University of Technology Sydney (UTS) have developed a portable system capable of translating silent thoughts into text. This innovative technology, created by the GrapheneX-UTS Human-centric Artificial Intelligence Centre, has the potential to revolutionize communication for individuals who are unable to speak due to conditions such as stroke or paralysis. Additionally, it holds the promise of seamless interaction between humans and machines, enabling tasks such as operating bionic arms or robots.
Presented as the spotlight paper at the prestigious NeurIPS conference held in New Orleans on December 12, 2023, this study marks a significant breakthrough in the field of artificial intelligence and machine learning. Led by Distinguished Professor CT Lin and his team, the research involved participants silently reading passages of text while wearing a cap equipped with an electroencephalogram (EEG) to record their brain’s electrical activity. This data was then decoded and transformed into written text, as demonstrated in a video showcasing the technology’s capabilities.
Unlike previous methods that required invasive surgical procedures or large MRI machines, the UTS system offers a non-invasive approach. By utilizing EEG signals recorded through a cap worn on the scalp, the researchers ensure a more versatile and user-friendly experience. Importantly, this study’s extensive scope involved 29 participants, yielding more robust and adaptable results compared to previous single-subject studies.
While achieving state-of-the-art performance in EEG translation, the model still exhibits room for improvement. Currently, the accuracy score stands at around 40% on the BLEU-1 scale, which measures similarity to high-quality reference translations. The researchers aim to enhance this metric, with the long-term goal of achieving translation and speech recognition capabilities comparable to traditional language programs, which typically reach levels as high as 90%.
Distinguished Professor CT Lin expressed enthusiasm about the study’s groundbreaking approach to neural decoding. He highlighted the integration of discrete encoding techniques and the subsequent fusion with large language models as a significant achievement. The resulting system successfully converts raw EEG waves into comprehensible language, thereby expanding the horizons of both neuroscience and AI research.
Lead author Yiqun Duan and fellow PhD candidate Jinzhou Zhou shed light on the model’s limitations. While more proficient in matching verbs, the model tends to produce synonymous pairs rather than precise translations when it comes to nouns. The researchers attribute this behavior to semantic similarities that generate comparable brain wave patterns during word processing. Despite these challenges, the model yields meaningful results by aligning keywords and constructing similar sentence structures.
As technology continues to advance, the potential applications for this revolutionary system are boundless. Beyond aiding individuals with speech-related disabilities, it could facilitate improved human-machine interactions, making tasks such as controlling bionic limbs or communicating with robots more effortless and intuitive.
The research presented at NeurIPS embodies a significant stride towards harnessing the power of AI to enhance and empower human communication. As this technology evolves, the future holds the promise of transforming thoughts into words, unlocking new frontiers in accessibility and interaction between humans and machines.
Reference: Lin CT, Duan Y, Zhou J, et al. DeWave: Discrete encoding of EEG Waves for EEG to text translation. Presented at NeurIPS 2023, December 12, 2023, New Orleans.
