In a groundbreaking development, researchers at the University of Technology Sydney (UTS) have unveiled a portable, non-invasive system that can translate silent thoughts into written text. This innovative technology, developed by the GrapheneX-UTS Human-centric Artificial Intelligence Centre, has the potential to revolutionize communication for individuals who are unable to speak due to conditions like stroke or paralysis. It could also facilitate seamless interaction between humans and machines, enabling the operation of bionic arms or robots.
The significant breakthrough, selected as the spotlight paper at the prestigious NeurIPS conference held in New Orleans on December 12, 2023, demonstrates the capabilities of the new system. Led by Distinguished Professor CT Lin and assisted by first author Yiqun Duan and PhD candidate Jinzhou Zhou, the research involved participants silently reading text passages while wearing a cap that recorded their brain activity using an electroencephalogram (EEG).
The captured EEG waves were then processed using an artificial intelligence model called DeWave, developed by the researchers. DeWave is capable of translating the EEG signals into words and sentences by learning from vast amounts of EEG data. By employing discrete encoding techniques, the system introduces an innovative approach to neural decoding and opens up new frontiers in neuroscience and AI.
Unlike previous technologies that required invasive procedures or bulky MRI machines for brain signal translation, the UTS system utilizes an EEG cap, making it more accessible and user-friendly. Moreover, it can be used with or without additional aids such as eye-tracking.
The UTS study involved 29 participants, making it more robust and adaptable than previous decoding technologies tested on only one or two individuals. Although the EEG signals captured through the cap may be noisier, the study reported state-of-the-art performance, surpassing previous benchmarks.
Despite the system’s significant achievements, translation accuracy remains a work in progress with a current score of approximately 40% on BLEU-1, a metric used to evaluate machine-translated text. The ultimate goal is to reach a level comparable to traditional language translation or speech recognition programs, which typically achieve a score of around 90%.
Duan noted that the system excels in matching verbs but struggles with nouns, tending to provide synonymous alternatives rather than precise translations. This phenomenon is likely due to similar brain wave patterns being produced by semantically related words. Nonetheless, the model still produces meaningful results, aligns keywords, and constructs sentences with similar structures.
This latest research builds upon UTS’s previous brain-computer interface technology, developed in partnership with the Australian Defence Force, which employed brainwaves to command a quadruped robot. These advancements highlight the university’s commitment to pushing the boundaries of scientific exploration and practical applications.
As the GrapheneX-UTS Human-centric Artificial Intelligence Centre continues to refine and enhance their mind-reading system, the possibilities for transforming communication and interaction in various fields are limitless. The implications for individuals with speech-related difficulties, as well as the potential for seamless human-machine collaboration in areas like prosthetics and robotics, are truly remarkable.
