CSE Graduate Takes High-Performance Computing Expertise to Top Tech Corporation

As another semester wraps up at Georgia Tech, new alumni will soon take the next step in their professional journeys. 

One of those graduates is Mikhail (Michael) Isaev, who earned a Ph.D. in computer science from the School of Computational Science and Engineering (CSE). After he walks across the stage and accepts his diploma at McCamish Pavilion on May 2, Isaev’s next move is to work at NVIDIA Research as a research scientist.

Advised by School of CSE Professor Rich Vuduc, Isaev’s research interests lie at the intersection of computer architecture, high-performance computing (HPC), and deep learning. He focuses on deep learning workload analysis and software/hardware co-design of large-scale deep learning systems.

Image

Isaev received notable recognition for his co-design research at ModSim’22, where he won the Dr. Sudhakar Yalamanchili Award. He earned the “Sudha” Award for his research on ParaGraph. 

ParaGraph provides an automated way to emulate application software in ways that a network simulator can understand. The tool makes co-design a bilateral process, facilitating better supercomputing applications and closing the gap for hardware and software experts.

While the award recognizes researchers for their contributions to the computer modeling and simulation field, it carried much more sentimental meaning to Isaev. Yalamanchili was a Georgia Tech faculty member who died in 2019. Isaev and his collaborators personally knew and worked with Yalamanchili.

“I felt very honored to receive the award,” Isaev said. “I had the pleasure to meet and talk to Sudha, so it felt great to bring home this award in his name and, in a way, give back to Georgia Tech.”

Another meaningful project Isaev worked on was Calculon, a tool for co-design optimization of large language models (LLMs). 

Calculon analyzes large co-design spaces of hardware and software configurations. This ability  progresses the discovery of new, and sometimes surprising, configurations that might outperform current methods.

By focusing specifically on LLMs, Calculon modeled more aspects of performance optimization at greater accuracy and speeds several orders of magnitude faster than ParaGraph.

As tech companies train and retrain LLMs on tens of thousands of graphics processing units (GPUs), search spaces grow larger and become more vast as larger systems are introduced.

Isaev’s work on Calculon is timely since there was no open-source tool that could quickly navigate this space and determine the best configurations. Calculon is a simple, yet effective, tool that can do the job fast and efficiently, sparking interest from many companies and research community

Isaev presented Calculon at conferences like Supercomputing 2023, ModSim’23, and the ASSYST workshop at ISCA 2023. He also gave talks at NVIDIA, Google, Microsoft, IBM, and the Department of Energy to share his research.

The talks at NVIDIA, Google, and Microsoft were notable since Isaev interned as a Ph.D. student at the companies. He has also interned at HP Labs and Meta. 

Isaev interned at NVIDIA four times, three with the company’s Network Research Group. That is where he worked on ParaGraph and Calculon under one of his mentors, Nic McDonald. 

“While Calculon won no award, I feel it was more well-received and got better traction with the HPC community,” Isaev said. 

“This is partly due to Calculon being a product of collaboration between my internships and conference presentations. It truly is a tool created by HPC researchers for HPC research.”

Image