Group Optimizes Fluid Dynamics Simulator on World’s Fastest Supercomputer

From the sky to the sea, and even inside our bodies, a simulator from Georgia Tech is helping us better understand aerodynamic fluid flows. And thanks to Frontier, the world’s fastest supercomputer, the simulator is even better.

Spencer Bryngelson’s research group manages the Multi-Component Flow Code (MFC) software package. His group was one of ten teams selected by Oak Ridge Leadership Computing Facility (OLCF) to optimize their simulator in a hackathon held Oct. 31 – Nov. 3.

During the hackathon, the group crafted MFC to run on Frontier, the world’s only exascale supercomputer. With newfound experience on Frontier, the group is poised to work with exascale machines scheduled to come online soon, like Aurora and El Capitan.

The team used the hackathon to refine MFC, making it more capable for medical, aeronautical, and defense applications.

MFC simulates compressible multiphase flows, a key issue in many engineering problems. MFC’s high quality simulations help engineers improve technologies.

Potential applications include:

• Needle-free drug injection
• Improved artificial heart pumps and valves
• Erosion-resistant aircraft surfaces
• Quieter submarines

These simulations require a blend of software, like MFC, and the largest supercomputers the government can bring to bear.

“MFC is a versatile solver that accounts for flavors of fluid flows like acoustics, surface tension, phase change, high Mach shock waves, and so on,” said Bryngelson, an assistant professor in the School of Computational Science and Engineering (CSE).

“MFC has other tricks. We can model solid and porous materials, which have been applied to simulations of the breakup of kidney stones for new lithotripsy therapies.”

Before the hackathon, Bryngelson's group already scaled MFC to the entirety of Summit, another OLCF supercomputer. Summit is based on NVIDIA GPUs, and was the world's fastest supercomputer from November 2018 to June 2020. Testing MFC on Frontier showed the group how the software performs at its peak on over 30,000 AMD GPUs. As a result, the group configured software to the industry’s two leading hardware vendors.

During the hackathon, the group resolved a compiler issue that slowed performance by five times. They squashed the bug by hacking a fix in the machine bytecode.

The issue caught the attention of Cray, a manufacturer of Frontier and its compilers. The performance bug likely infests other applications using Frontier. So, MFC’s experience will help Cray further study and prevent similar issues for future users.

[Related: New Hardware Brings Students Closer to Exascale Computing]

“Running on Frontier is about discovering the most challenging engineering problems we can solve, “Bryngelson said. “We’ve tooled MFC to perform exceptionally large simulations on the next generation of leadership-class supercomputers, like Frontier.”

Oak Ridge National Laboratory (ORNL) and the U.S. Department of Energy jointly operate Frontier. It jumped to first place on the Top500 in June 2022 soon after becoming operational and proving its exascale capability that May.  

Frontier can compute over one quintillion calculations in a second as an exascale supercomputer. If a person completed a simple math problem every second, that person would need 31.6 billion years to match what Frontier can do in a second.

The group accessed research scientists and engineers as mentors throughout the hackathon. Notable assistance came from Steve Abbott of Cray, Brian Cornille of AMD, and Reuben Budiardja of ORNL.

MFC stems from Bryngelson’s time at Caltech, dating back to 2018. He worked on the project with Tim Colonius, and the project has collaborators around the world.

Bryngelson’s students maintain MFC and participated in the hackathon. These included Ph.D. students Ben Wilfong and Anand Radhakrishnan and undergraduate students Henry Le Berre and Ansh Gupta.

“Through the hackathon, the group acquired an education that may not be possible otherwise. They now have experience using a specialized tool to use toward their research,” Bryngelson said.

“Everything we’re talking about right now is very new, and there is little track record to fall back on, so we’re helping find problems and hacking fixes as we go.”