Toyota, in collaboration with MIT and Columbia Engineering, has made a significant breakthrough in AI learning for robots, showcasing a method that dramatically accelerates the acquisition of new skills. The approach, analogous to a ChatGPT moment for robotics, aims to usher in the general-purpose robotics era, with humanoid robots autonomously navigating workspaces and taking over various tasks from humans.
While early use cases primarily involve robots handling physical tasks, such as lifting and moving objects, Toyota envisions a future where robots can quickly adapt and learn a wide array of tasks based on human instructions or demonstrations. The key to this breakthrough is Toyota’s new learning approach called Diffusion Policy, developed in partnership with MIT and Columbia Engineering.
Diffusion Policy enables robotic AIs to observe and learn from human actions in the real world, programming themselves to perform tasks flexibly. Unlike some approaches that use virtual reality telepresence, Toyota’s method focuses on haptics. Human operators receive haptic feedback from the robot’s soft, flexible grippers through hand controls, allowing them to feel the robot’s interactions with objects.
The learning process begins with a human demonstrating a set of skills through teleoperation. Toyota’s AI-based Diffusion Policy then learns in the background over a matter of hours, creating its internal model of task success and failure. The robot then runs thousands of physics-based simulations based on its internal models to refine its techniques for the task.
The team has successfully trained robots in over 60 small tasks, including kitchen-based activities like spreading a slice of bread evenly with a knife or flipping a pancake with a spatula. Toyota aims to have hundreds of tasks mastered by the end of the year, with a target of over 1,000 tasks by the end of 2024. The ultimate goal is to create the first Large Behavior Model (LBM), a comprehensive AI-generated model of how robots can interact with the physical world to achieve various outcomes.
While Toyota’s breakthrough showcases the rapid evolution of robotics, experts emphasize that it’s not a complete solution to the climate crisis. The technology represents a crucial step in enabling robots to quickly learn new tasks, potentially revolutionizing industries and applications. As the team constructs the LBM, it sets the stage for future robot owners and operators to rapidly teach their bots new skills, upgrading entire fleets of robots with unprecedented dexterity and adaptability.
This breakthrough has significant implications for a wide range of industries. As Professor John Leonard of the MIT Department of Mechanical Engineering states, This represents a key step toward a new generation of flexible, dexterous robots that can work alongside humans to augment our capabilities, increase productivity, and enhance safety. He further adds, This research paves the way for robots that can adapt to unforeseen circumstances or changing environments using skills learned from human operators, which significantly expands their potential applications.
The implications of this breakthrough go beyond just physical tasks. Professor Hod Lipson from Columbia Engineering explains, What makes this work so important is that it enables robots to learn from real-world human interventions; it opens the door to robots that can understand how humans are using their bodies in social settings and to engage humans more intuitively.
Toyota’s breakthrough is a major stepping stone towards advancing the field of robotics. With robots gaining the ability to quickly learn and adapt, the potential for increased automation and efficiency in various sectors is immense. While challenges remain, the collaborative effort between Toyota, MIT, and Columbia Engineering has unlocked a promising pathway to a future where humanoid robots seamlessly integrate into our daily lives, taking on diverse tasks and augmenting human capabilities.
