Professor to Use NSF CAREER Award to Advance Open-World Computer Vision

School of Interactive Computing assistant professor Zsolt Kira believes the future of machine learning and computer vision exists in continuous open-world learning.

The National Science Foundation seems to agree.

Kira’s research received a boost in January when he learned he had received an NSF CAREER Award for his proposal, Visual Learning in an Open and Continual World.

“The goal is to move beyond current machine learning and computer vision where there is a closed-world assumption,” Kira said.

According to the National Science Foundation website, the NSF CAREER Award is given to early career faculty who show potential to serve as academic role models and advance the mission of their organization. The award is the most prestigious one given by the NSF to early career faculty.

Kira said in closed-world computer vision, a human annotates every piece of data into a fixed set of categories that a machine learning model uses to function. Eventually, the machine will encounter a new scenario or object that humans didn’t anticipate.

“Self-driving cars, once you deploy them, inevitably they’ll encounter new types of data such as new objects,” Kira said. “The idea is how can we detect if it’s seeing something new? Once we detect it, how can we add it to the knowledge that the AI model has and be able to automatically and continuously update it to learn new classifiers and object detectors?

“An open world means that if the car encounters some completely new thing, it should be able to handle that new type of object that it might encounter.”

Kira is an associate director of the Machine Learning Center at Georgia Tech and branch chief of the Machine Learning and Analytics group at the Georgia Tech Research Institute. The NSF CAREER Award, which is worth more than $500,000, will allow Kira and his students to create more robust open-world machine learning models over the next five years.

In the first year of his five-year proposal, Kira will focus on untangling different types of distribution shifts in out-of-distribution detection from current machine learning models. Out-of-distribution detection happens when the model detects an entity outside of its training data.

In the second year, he’ll use natural language processing to help his models detect and label new entities.

“There’s a lot of buzz about these language models or multimodal models that tie language and vision together,” Kira said. “So, we’re leveraging those advances so that after it detects something new, it will automatically be able to label it.”

In the third and fourth years, Kira will refine the semantics and build a hierarchal structure for his new models to help them become more continuous in their learning. In the final year, he hopes to have models ready for generation and deployment.

Kira said his work will advance AI education through Meta’s AI Learning Alliance, an initiative to increase diversity and equity in the field of artificial intelligence. Kira’s work will also expand into K-12 education initiatives through Georgia Tech’s Constellations Center for Equity in Computing.

“It’s a great honor to get it, and it’ll drive a lot of great work,” Kira said of receiving the award. “The foundations of the proposal were built from the work of my students. Most credit goes to them for doing great work in my lab. I think this proposal really ties together a lot of their work to really try to revolutionize computer vision where it can really be in an open world.”