From quarks to the coronavirus: Jefferson Lab dedicates a portion of its supercomputing resources to help in the fight against the COVID-19 coronavirus
Around the world, organizations are finding ways to contribute to the fight against the COVID-19 coronavirus. Now, the scientific computing group at the Department of Energy’s Thomas Jefferson National Accelerator Facility has begun making a real impact in the effort by re-purposing computing downtime on the lab’s supercomputers toward finding a cure for the disease.
“This is one of the biggest ways that we can contribute, because, as you know, we don't have expertise on the biological side of it, but we do have expertise in computing,” said Graham Heyes, who heads Jefferson Lab’s scientific computing group.
While Jefferson Lab’s supercomputers are typically focused on calculating the building blocks of matter, now those same machines are also calculating the building blocks of the COVID-19 coronavirus as part of the Folding@home project.
“We knew that we had a big computing resource that we could make use of while everybody was away from the lab, so I started to look into making our resources available to the Open Science Grid and what that would entail,” Heyes explained. “It turned out that what they wanted to do was to run this thing called Folding@home.”
The Folding@home project simulates protein dynamics, or specifically how a protein’s components are arranged and how they move. Folding@home takes advantage of compute time donated by citizen scientists and organizations to run simulations of these proteins, such as those present in COVID-19. It is currently simulating the dynamics of COVID-19 proteins to aid in the hunt for new therapies to combat the virus.
“We contacted Folding@home, and what they needed most of all were our graphics processors, our GPUs. And one of our compute clusters is equipped with 256 GPUs, and we have a machine learning project that has more GPUs. So, we downloaded the software ourselves to see if we could get it to work,” Heyes said.
Jefferson Lab’s supercomputer clusters are powered by two different technologies: traditional central processing units (CPUs) and newer graphics processing units (GPUs), such as those found in home gaming consoles. Both technologies are installed in the laboratory’s supercomputer clusters.
“Bryan Hess, Jie Chen and I worked out how to get the software, which would normally run as a screensaver, to run on our big cluster computers. After a little bit of effort, we originally got this thing to run by re-authorizing a portion of the GPUs for use only by this project, about 10% of that compute cluster,” Heyes said.
Once they successfully demonstrated that the software could run on the supercomputer, the team began tweaking the configuration to determine how to make it as productive as possible without impacting the laboratory’s scientific output. They soon found a way to run the software on the GPU-based supercomputer cluster in the free time between its scientific runs, which also amounts to about 10% of the computer’s total up-time.
Then, Heyes and his group researched donating additional time on the other supercomputers based at the laboratory.
“The Folding@home project wanted the GPUs specifically, but we've also got 7,000 CPU cores sitting in our main clusters, so it's a much bigger resource, even if it wasn't what they critically needed,” Heyes said. “So, after we had done the hard work to get Folding@home to work on our GPU clusters, we then worked out how to run it on the rest of the clusters, so that we could make prudent use of all of the resources at Jefferson Lab to benefit the project.”
Once again, Heyes stressed, the project only runs in the background on these clusters, so that it does not negatively impact the laboratory’s experimental or theoretical research programs. Even so, Jefferson Lab’s supercomputers are making a real impact on the effort. The lab’s contribution to the COVID-19 effort can be tracked on the Folding@home website.
“Our scientific computing group rose to the occasion during this challenging period, and I’m proud of the team for the work they did make to Folding@home run well on our systems,” said Jefferson Lab Chief Information Officer Amber Boehnlein. “We have participated in Open Science Grid projects before, and we are proud to give back to the research community via this appropriate venue.”
In addition, Heyes said that individual members of the Jefferson Lab community can also donate computer resources to the project via their personal computers.
“Anybody who wants to can download the software themselves and use it on their own PC at home,” Heyes said. “If they would like to contribute as part of the Jefferson Lab effort, they can click on our team number, and any processing they do on their home machine will go toward the total.”
Contact: Lauren Hansen, Jefferson Lab Communications Office, 757-269-7689, lhansen@jlab.org