Green 'Oakley Cluster' to double OSC computing power

COLUMBUS, Ohio (Mar 19, 2012) — 

Researchers using Ohio Supercomputer Center (OSC) resources can now conduct even more innovative academic and industrial research by accessing Ohio’s newest energy-efficient, GPU-accelerated supercomputer system.

"This strategic investment in a powerful, state-of-the-art computational system will boost the vital research being conducted at industrial facilities and in university laboratories across Ohio,” said Chancellor Jim Petro of the Ohio Board of Regents. “The Oakley Cluster also will help Ohio colleges and universities in their development of a highly skilled workforce by invigorating the state’s innovative computational science education programs.”

Photo: OSC's Oakley Cluster

OSC's HP-built Intel Xeon Oakley Cluster Specs

  • 8,328 total cores
    • 694 nodes, 12 cores/node, and 48 gigabytes of memory/node
  • Intel® Xeon® x5650 CPU processors
  • HP SL390 G7 Nodes
  • 128 NVIDIA® Tesla™  M2070 GPUs
  • QDR IB Interconnect
    • Low latency
    • High throughput
    • High quality-of-service
  • Theoretical system peak performance
    • More than 88 teraflops
  • GPU acceleration
    • Additional 66 teraflops
  • Total system peak performance
    • 154 teraflops
  • Memory increase
    • Increases memory from 2.5 gigabytes per core to 4.0 gigabytes per core
  • Storage expansion
    • Adds 600 terabytes of DataDirect Networks Lustre storage for a total of nearly two petabytes of available disk storage.
  • System efficiency
    • 1.5x the performance of current systems at just 60 percent of current power consumption. 

OSC’s new $4.1 million HP-built, Intel® Xeon® processor-based supercomputer, dubbed the Oakley Cluster, features more cores (8,328) on half as many nodes (694) as the center’s most recent flagship system, the IBM Opteron™ 1350 Glenn Cluster. The Oakley Cluster can achieve 88 teraflops, tech-speak for performing 88 trillion calculations per second, or, with acceleration from NVIDIA® Tesla™ graphic processing units (GPUs), a total peak performance of 154 teraflops.

The new system provides nearly twice the memory per core (4 gigabytes) and three times the number of graphic processing units or ‘GPUs’ (128). The Oakley Cluster also provides researchers with one and a half times the performance of the Glenn Cluster at just 60 percent of Glenn’s power consumption and will expand OSC storage to nearly two petabytes with the addition of 600 terabytes of new DataDirect Lustre storage.

OSC is a state-funded high performance computing center that provides Ohio’s universities, industries and other clients with computation, software, storage and support services. OSC’s centralized support increases the opportunities for researchers statewide to innovate and successfully compete for grants and national supercomputing resources. Major users of OSC’s resources have focused on research in the areas of the biosciences, advanced materials, energy and the environment.

The new system arrived Dec. 28 on the west campus of The Ohio State University, where engineers have been working to assemble the complex system, install various software packages and test-run the system by allowing access to a few selected “power users.” Earlier, the engineers made room for the Oakley Cluster by removing the older half of the Glenn Cluster, which in December logged its 150 millionth hour of computational use by OSC clients.

“A recent survey of Ohio universities indicated that researchers who leverage OSC resources annually garner more than $140 million in research funding – a huge return on investment for the state,” explains Ashok Krishnamurthy, interim co-executive director of the center and senior director of research. “And, as we deploy each powerful, new computational system, we see our researchers react by leveraging even more sophisticated applications to analyze even larger data sets to solve even more complex science and industrial problems.”

Among many OSC initiatives that will benefit from the features of the new system are the Blue Collar ComputingTM (BCC) program and the virtual Ralph Regula School of Computational Science. One of the center’s most pioneering projects, BCC provides high performance computing for small- and mid-sized companies that do not have the time, money, or expertise to invest in supercomputing resources. The Ralph Regula School, developed through funding from the National Science Foundation, coordinates computational science education programs statewide that lead to a baccalaureate minor, an associate degree concentration and a workforce certification.

“The school empowers the workforce with the computational science knowledge and skills for additional employment and advanced education opportunities,” said Steven Gordon, interim co-executive director of OSC and executive director of the Ralph Regula School. “Following the wider deployment of the Oakley Cluster, OSC will offer training with the new system, including such topics as parallel programming and various languages.”

The Oakley Cluster’s 2012 deployment coincides with the 25th anniversary of the establishment of the center in 1987 to place Ohio universities and industry at the forefront of computational research. OSC is a member of the Ohio Technology Consortium, the technology arm of the Ohio Board of Regents, and is located on the west campus of The Ohio State University.

THE OAKLEY NAME: The Ohio Supercomputer Center’s newest cluster pays tribute to the legendary Ohio sharpshooter and social advocate Annie Oakley. Recent OSC systems have been named after important Ohio pioneers: The Glenn Cluster honors astronaut and statesman John Glenn and the ARMSTRONG research portal honors astronaut Neil Armstrong, while the Csuri Advanced Photo: Annie OakleyGPU environment recognizes pioneering computer artist Charles “Chuck” Csuri.

Oakley was born Phoebe Ann Moses in a log cabin in western Ohio’s Darke County in 1860 and died in Greenville, Ohio, 66 years later. She learned to shoot a gun at the age of eight and performed for 16 years in a traveling Wild West show with “Buffalo Bill” Cody.

During WWI, Oakley gave shooting demonstrations to raise money for medicine and supplies. Annie overcame poverty, mistreatment and physical injury, helped to break barriers for women and showed great compassion and generosity to orphans, widows and young women.
(Darke County Historical Society, OH)


Examples of recent research being conducted on OSC systems

Image: Biosciences

Image: Advanced Materials

Image: Energy Image: Industry

BIOSCIENCES: Seeking a new antidote for nerve agents, Ohio State chemist Christopher Hadad accessed OSC resources to model paraoxonase bound to a ligand. 
More - (Hadad/OSU)

ADVANCED MATERIALS: Backed by growing industrial interest, Sadhan Jana’s University of Akron team used OSC systems to simulate organic tie-molecules on multi-walled carbon nanotubes. 
More - (Jana/UA)

ENERGY/ENVIRONMENT: Ohio State students used OSC systems to simulate the airflow throughout the wheel well of their Buckeye Bullet land speed racecar as it travels at 300-mph. 
More - (OSU Center for Automotive Research)

INDUSTRY: Specializing in automotive fluid dynamics, Total Sim USA in 2009 landed in Dublin, Ohio, over potential sites in North Carolina and Indiana in part due to the close proximity of resources available through OSC. 
More - (Total Sim USA)


OSC Oakley Cluster Client Testimonials

We are combining models and observations together to describe and understand the human-caused climate change happening in the Arctic. This study is being done at unprecedentedly high spatial resolution. The project, called the Arctic System Reanalysis (ASR), is a massive computational task. The much faster processor speed, the increased number of cores, and higher performance I/O that will be provided by Oakley Cluster in comparison to the system we are currently using will result in ASR being completed on time and in budget.
David Bromwich, Ph.D., Senior Research Scientist
Byrd Polar Research Center, Polar Meteorology Group, The Ohio State University


We started a U.S. sister company to a British engineering firm that specializes in automotive fluid dynamics and were deciding whether North Carolina or car racing mecca Indianapolis was a better fit than Central Ohio. But the proximity of auto manufacturers, support from TechColumbus and, perhaps most important, the ability to tap into the state-run Ohio Supercomputer Center for complex simulations drew our firm to the Dublin Entrepreneurial Center in 2009. The impact of a the speed improvement on Oakley is two-fold; it’s simply getting the answers back quicker, but it’s value too because of the way you pay for the compute resources on a time basis. To have a resource like OSC ... it was kind of the key part of putting the company together.
Ray Leto, President 
Total Sim LLC, Dublin, Ohio


One of the hardest issues for a faculty member is to make new infrastructure improvements on a regular basis. An experimentalist might outfit their lab when they first start, knowing that in 5-7 years, or maybe less, they’re going to have to go buy a new spectrometer for this and a new one for that, continuously worrying about how to get the right amount of capability to remain state-of-the-art. I don’t have to worry about that with computing, because the Ohio Supercomputer Center is improving their hardware and my computing capability on a regular cycle, through acquisitions like the new Oakley Cluster, continuously giving me more firepower than I had before. That keeps moving my research program into areas I never thought possible, because now I can accomplish so much more in less time. I simply write more proposals that can get more people to be supported so I can use the compute cycles at the OSC in advantageous ways to get more exciting science done. It’s a tremendous advantage.
Christopher Hadad, Ph.D., Professor of Chemistry
Associate Dean of Natural and Mathematical Sciences
College of Arts and Sciences, The Ohio State University


The Ohio Supercomputer Center (OSC), a member of the Ohio Technology Consortium of the Ohio Board of Regents, addresses the rising computational demands of academic and industrial research communities by providing a robust shared infrastructure and proven expertise in advanced modeling, simulation and analysis. OSC empowers scientists with the vital resources essential to make extraordinary discoveries and innovations, partners with businesses and industry to leverage computational science as a competitive force in the global knowledge economy, and leads efforts to equip the workforce with the key technology skills required to secure 21st century jobs. For more, visit

The Ohio Board of Regents is the state agency that coordinates higher education in Ohio, and its Chancellor, who is a member of the Governor of Ohio’s cabinet, directs the agency. The Chancellor, with the advice of the nine-member board, provides policy guidance to the Governor and the Ohio General Assembly, advocates for the University System of Ohio and carries out state higher education policy.For more, visit