Supercomputers given optimal conditions

Computer calculations Data analysis Hardware and components Systems and data security

Passing Risø DTU Campus, one is immediately drawn to the impressive row of wind turbines. However, something equally impressive and significant is now beginning to take shape at the northern end of the campus.

Behind the modest exterior of two containers lies the ‘Computerome’ supercomputer. Specializing in life science research, it can compete with best in the world. And Computerome marks only the start of DTU’s new high performance computing centre, which will house supercomputers for every imaginable area of research—from wind turbulence to neutron radiation—not to mention providing the added bonus of supplying district heating to Risø Campus.

Supercomputing or high-performance computing (HPC) helps researchers to combine large and complex data volumes. By connecting many computers and programming them to perform calculations based on carefully planned algorithms, for example, we can answer the question of how certain diseases are linked to particular genes, create models for how the wind moves around a wind turbine blade, or learn more about the emigration of our early African ancestors. Supercomputing is simply the most effective way to derive maximum benefit from the wealth of data we are now able to gather.

For a long time, DTU has used supercomputing in several research areas and installed different computer systems to serve the departments. But now, a completely new infrastructure specifically dedicated to supercomputing has been inaugurated at the northernmost corner of Risø Campus. The 3,500 m2 area can accommodate a minimum of eight large containers, each capable of holding up to several hundred computers connected in strong networks.

Photo: Iben Julie Schmidt
 

 

It does not draw much attention to itself, but behind the doors in the container stands one of the world’s most powerful and state-of-the-art supercomputers. 

 

Maximum security and flexibility
With such vast data volumes and sensitive data covered by the Danish Data Protection Act, security is a top priority—both physical security to prevent unauthorised access and computer theft—and IT security to ensure that the various data owners do not have access to other owners’ data unless the parties have entered into a specific agreement.

Several HPC containers can quickly and easily be connected to the existing infrastructure, and if new users so desire, the site can be upgraded to a higher level of security of supply.

Cooling converted into heat
At present, Computerome consumes 200 kW, with the entire site dimensioned to six megawatt. Energy, however, does not simply vanish into thin air, but is converted into heat at Risø Campus.

“The cooling system has both a warm a cold side to it. On the hot side facing the computers, we have temperatures of 45-50 degrees. Using a heat pump, we increase the water temperature to 70 degrees so it can be used in the heating system. The system is designed so that each new computer that is connected immediately supplies district heating to Risø Campus,” says Allan Murphy, Section Head, CAS Campus Service.

“With our current capacity, we can heat the entire campus four months of the year. As the site expands with the addition of more supercomputers, our district heating contribution will increase, and looking ahead we will easily be able to cover all our heating needs and more.”

The built-in flexibility and sustainability of the system has attracted attention from outside DTU. For example, the Danish Ministry of Foreign Affairs paid a visit in connection with the negotiations with Apple to establish a data centre in Denmark. One of the reasons why the agreement was reached was DTU’s experience in channelling excess heat into the district heating network. So while DTU’s HPC centre is clearly a good business case, it also helps to reduce the University’s carbon emissions.

Article from DYNAMO no. 42, DTU’s quarterly magazine in Danish.

Computerome

  • DeiC National LifeScience supercomputer was established by the Center for Biological Sequence Analysis, DTU Systems Biology in collaboration with the Danish e-Infrastructure Corporation and the University of Copenhagen.
  • Denmark’s biggest supercomputer with a capacity of approx. 8,024 interconnected PCs .
  • In November 2014, it was ranked 121 on the top 500 list of the world’s largest computer systems.
  • Size: 16,048 CPU cores. • Memory: 92TB DDR4 RAM. • Storage: 3 Petabyte = 300,000 gigabyte storage.
  • Backup: 1 petabyte – one PC can typically store 250-500 gigabyte. • Performance: 410.8 TeraFLOPS (flops = floating point operations per second).
  • Bandwidth: Can transfer 75 Gigabyte per second between the server and storage facilities. It can analyse 41 genomes in eight hours, for example.
  • Security: All data are stored in a cloud to which each data owner has exclusive rights. The system complies with the USA’s strict security requirements.
  • Reliability: There is redundancy in all critical systems so that users are guaranteed uptime around the clock.
  • Demand: Currently, there are 400 users and queued 20,000 jobs.