Big leaps forward in bandwidth

22 Nov 2011

This week, two US research organisations have unveiled networks for research scientists capable of delivering data at speeds of 100 Gigabits per second.

To put that in perspective, that’s ten times faster than the speeds provided by Internet service providers and 50,000 times faster than the average iPhone connection, according to a DOE statement.

The new DOE network links scientists working at the National Laboratories of Lawrence Berkeley (in California), Argonne (Illinois) and Oak Ridge (Tennessee). Called the Advanced Network Initiative (ANI), it was unveiled this week at SC11, the annual shindig for the supercomputing/high-performance computing community.

ANI is just the first stage of a wider DOE initiative to roll out 100 Gbps services across ESnet, the national network that links thousands of government researchers at more than 40 different facilities in the US as well as research partners in other countries. Other data-intensive laboratories will be the next beneficiaries, with all DOE national lab sites to be integrated into the 100 Gbps infrastructure by the end of 2012.

The new network at HHMI’s Janelia Farm Research Campus, meanwhile, is based on technology from networking company Brocade, which claims it is the “world’s largest single-site deployment of 100 GB Ethernet” and will “significantly increase operational efficiency” for some 250 biomedical research scientists.

"We have 200 Gigabits connections to each of our network wiring closets," said Spartaco Cicerchia, director of network infrastructure systems for the Janelia Farm Research Campus in Ashburn, Virginia. "Then we have 10 GbE connections going directly down to researchers' systems in order to support the massive volumes of data they have collected. This new infrastructure enables our research team to obtain results 10 times faster than before. This performance advantage will allow us to set a new standard for research process cycles."

Of course, both networks cater for a rather niche audience. High-performance computing is a highly specialised area that has grown up around the need of scientists and researchers working on advanced computation problems, involving massive data sets and often performed across large clusters or grids of machines.

For example, ESnet is used today by physicists who need access to data generated by the Large Hadron Collider in Switzerland; by climate scientists, to interrogate some of the world’s largest databases of meteorological information; and by biofuels experts, to access genomic data on energy-relevant microbes and plants.

But the DOE also claims that the launch of ANI, which uses new optical technology to reduce the number of routers needed on the network, will accelerate “by several years” the commercialisation of 100 GbpS network technologies. In other words, regular IT teams will eventually have access to this technology, too.

“With the establishment of this high speed network, the United States is once again blazing a path for the future of Internet innovations,” said US Secretary of Energy Steven Chu. “Initially, this breakthrough will make sharing information between our labs much more efficient and pave the way for new discoveries, but it also holds the potential to change and improve our lives much like the original commercialisation of the Internet did in the mid-90s.”

That may sound like hyperbole, but it’s worth remembering that the World Wide Web originated with physicists at CERN who needed to share data with colleagues worldwide – and that several DOE laboratories were among its earliest pioneers.

For now, however, 10Gbps connectivity is the goal for most commercial organisations – the speed that ESnet already offers today. In a report released this week by IT market analyst firm Dell’Oro Group, quarterly shipments of 10Gbps Ethernet controllers and adapters – which connect servers to these networks - grew 47 percent year-on-year, to 1.3 million in 3Q11.

“We are still in the early stage of 10 Gbps server adoption and we expect that Intel's Romley platform will fuel even more migration towards 10 Gbps in 2012," said Dell'Oro Group analyst, Sameh Boujelbene. Romley is Intel’s new workstation/server platform, originally expected in 2011 but now likely to appear in 2012.