Two-World Training Approach Earns Roboticist NSF CAREER Award

Through his novel approach to combine simulation data with real-world ground truth data to train more capable robots, Assistant Professor Sehoon Ha has earned the National Science Foundation (NSF) CAREER Award.

The award is one of the most prestigious honors a junior faculty member can be given and typically comes with five years of funding for their proposed research.

Machine learning has opened the door for robots to enter the real world and learn about their environments and how to function within them with no pre-training data.

But having access to the ground-truth data for artificial intelligence isn’t a luxury that all robots have access to. For some robots, such as heavy-duty manufacturing machines or self-driving vehicles, it would be dangerous to introduce them to real-world environments and let them operate on trial and error. 

That’s one of the reasons many robots are trained in simulations and then deployed into the real world. But while programmers seek to mimic the robot’s real-world environment as accurately as possible, robots often have trouble overcoming what’s referred to by roboticists as the sim-to-real gap.

“Ultimately, the simulation is typically an inaccurate blueprint of the real world,” said Ha, who researches robotics and computer graphic simulations in Georgia Tech’s School of Interactive Computing. “Sometimes it works, sometimes it doesn’t. We need a structured approach to mitigate the gap.”

In his proposal to the NSF, Ha offers a solution to narrow the sim-to-real gap. He’s combining data simultaneously gathered from simulations and real-world learning to create new algorithms to maximize the potential of artificial intelligence (AI), which will lead to more capable and safer robots. 

“Typically, programmers will use simulation to pre-train their policies, with a little fine-tuning in the real world,” Ha said. “But there are no seamless approaches for integrating them into the same process.”

Ha calls his approach concurrent robot learning, which takes advantage of the scalability that comes with simulation and the ground-truth data found in real-world learning.

“On the one hand in simulation, you can collect massive amounts of data,” he said. “You can have as many agents as you like navigate the environment. But not all the data is accurate. For accuracy, we look at the real world. In the real world, you typically can only have one embodied agent, and their experience is limited, but they are very accurate. We can take the best from both worlds by integrating them into a single learning algorithm.”

Ha said his goal is to seamlessly combine the data provided from both worlds and to have each world “talk to one another” to create more insightful algorithms.

Ha’s research proposal involves two key stages. The first will focus on creating safe real-world learning environments, especially for bipedal humanoid robots, which because of their instability have larger sim-to-real gaps than quadrupedal robots.

In the second stage, Ha will develop various versions of concurrent robot learning algorithms.

Ha is the second faculty member from the School of Interactive Computing to receive the CAREER Award this year. Associate Professor Matthew Gombolay received the award for his research on home assistive robots in February.

Top photo by Nathan Deen/College of Computing