Alibaba Cloud’s AI Sign Language Interpreter ‘Xiaomo’ Enhances Accessibility at Asian Games
Alibaba Cloud, the digital technology and intelligence unit of Alibaba Group, has developed an AI-powered virtual sign language interpreter named Xiaomo. Xiaomo is currently being deployed at the ongoing Asian Games in Hangzhou, China to provide support and enhance accessibility for individuals with hearing impairments. This innovation aims to ensure that everyone can participate fully in the regional sports competition.
According to the World Health Organization, over 1.5 billion people, which accounts for almost 20% of the global population, live with hearing loss. With the integration of Xiaomo into Alipay, Alibaba Cloud’s payment platform, users can now access a mini-app that supports two-way translation between sign language and Chinese spoken language.
The mini-app allows users to input voice messages, which Xiaomo will translate into sign language for hearing-impaired individuals. Conversely, Xiaomo can also interpret sign language into spoken language, facilitating seamless communication between hearing and hearing-impaired individuals.
To tackle the complexities of sign language, Alibaba Cloud compiled a Chinese sign language translation dataset containing 25,000 signs. This dataset was gathered and annotated from sign language practitioners and hearing-impaired people across Zhejiang province in China. Leveraging advanced visual recognition algorithms and motion-tracking capabilities, Alibaba Cloud can accurately capture the movements of the hearing impaired and translate sign language into natural language.
Additionally, Alibaba Cloud developed a deep neural network and a lip movement generation module to convert natural language into sign language. This ensures that Xiaomo can accurately perform sign language gestures.
During the Asian Para Games, which will take place from October 22 to 28, Xiaomo will assist participants with hearing impairments in interacting with volunteers at the venues. This includes asking for directions, seeking medical help, and receiving assistance with game viewing.
In addition to Xiaomo, Alibaba Cloud has introduced a sustainable lifestyle web application at the Asian Games. This application encourages participants to adopt low-carbon behaviors and reduce their carbon footprint through digital engagement. Athletes, journalists, and staff members can earn carbon points by engaging in eco-friendly activities such as taking public transport, recycling waste, and reducing food waste.
Alibaba Cloud’s interactive web application has already recorded over 310,000 visits and over 7 tons of carbon reduction since the opening of the Asian Games villages on September 16. Participants can accumulate carbon points and redeem them for limited-edition Asian Games-themed pins and low-carbon products.
To further promote sustainability, Alibaba Cloud has partnered with 10 companies and brands to go sustainable within the Asian Games villages. The sustainability platform Energy Expert provided carbon footprint measurement, carbon neutrality calculation, and certification services, helping companies optimize their operations and reduce their environmental impact.
Alibaba Cloud’s efforts to enhance accessibility and promote sustainability at the Asian Games highlight the power of technology to create an inclusive and eco-friendly environment. With AI-powered innovations like Xiaomo and the sustainable lifestyle web application, Alibaba Cloud is making significant strides toward a more accessible and environmentally friendly future.
