Northeastern team breaks records at supercomputing competition

July 26, 2017

Zachary Marcus, a junior computer engineering and computer science major, recently returned home with his team from the Student Cluster Competition in Frankfurt, Germany. The supercomputing competition is designed to give undergraduates, from around the world, an immersive experience in high-powered supercomputing.

Northeastern joined teams from across the country and around the world, including Germany, China, and Poland, after the lengthy process that qualifies teams to compete, “teams need to find industry sponsors to help them in selecting their hardware to run scientific applications and benchmarks,” Marcus explains. “Eight months before the competition, teams apply to the committee and are soon chosen from the applicant pool. In the following months, the teams must work with their chosen sponsor (companies like IBM, NVIDIA, AMD, and HPE) to select hardware to best run the applications the competition is based on.

“Once the system is decided on,” Marcus explains, “the students must build and optimize their applications, which involves profiling the applications, leveraging new libraries, and rewriting portions of the code where necessary. The students are expected to gather a large understanding of the applications and the field of high performance computing; this is tested both in interviews and during the competition when they introduce a mystery application that the students have to quickly get up to speed on, optimize, and run.”

Northeastern’s path to involvement in the competition originated in the Computer Architecture Research Lab, which sent a team to Austin, Texas for the SCC in November 2015. Dave Kaeli, College of Engineering Distinguished Professor of the Department of Electrical and Computer Engineering is advising the team and supporting their efforts. “I was working as an undergraduate research assistant in the lab that summer,” Marcus says, “so I joined the team and began preparing. This recent competition was a bit strange” in that it involved an unusual amount of collaboration between universities: Northeastern partnered with Purdue University, with each university sending three competitors to form a six-person team; Northeastern’s competitors included Zachary Marcus, with a major in computer engineering and computer science, and Spencer Hance and Carter McCardwell, computer engineering students. “The students were selected based on high-performance computing experience,” Marcus says, and “all three NU students had been to two competitions previously. We competed against teams from across the world with varying levels of expertise.”

The competition included two days of setup and testing plus three days of competition. “We set up our hardware on Saturday, June 17th,” Marcus says, “and began testing. On Sunday, we tuned the performance of our applications. One wasn’t compiling, two weren’t performing as well as expected, and my application would break the rules of the competition if left to run unchecked” due to the amount of power that each team’s cluster is allowed to pull. “I had to tune the hardware’s frequencies and set up dynamic power scaling to keep the power levels acceptable,” he says.

“Starting on Monday, we began running our applications to submit actual results. This continued into Wednesday, with breaks for sleep and the introduction of a mystery application to be run with no time to prepare. By the time they introduced the mystery application, my applications had already been run, so I set out to run the new challenge with the help of the other two Northeastern students. The competition ended with a set of interviews about our team, the hardware, the applications, and how we modified them.”

Their team broke competition records for two applications, netting the highest High Performance Compute Gradients score any team has ever received, and the highest High Performance Linpack score that any US team has achieved.

In December, Northeastern will send a full team of six students to the next competition in Denver, “with some exciting new hardware from AMD,” Marcus says. Their record-holding “bodes well for the next competition.”