Researchers from the Argonne National Laboratory and DataDirect Networks (DDN) have transmitted 65 terabytes of data within 100 minutes, a transfer which usually requires two days with a 10 Gbps connection.
The record data transfer was demonstrated on 19th November during the SC14 supercomputing conference.
During the demonstration, scientists transmitted 85 Gbps of data — with peaks at over 90 Gbps — between storage systems in Ottawa, Canada, New Orleans and Los Angeles through a 100 Gbps wide-area network (WAN) connection.
The project was supported by Ciena, Brocade, and ICAIR.
For the high volume data transfer, the researchers used embedded file system and virtual machine capabilities of the DDN storage controller, the high-speed wide-area data transfer capabilities of the Globus GridFTP server, and an advanced 100G wide-area network.
Argonne National Laboratory principal software development specialist Raj Kettimuthu said, "Embedding the GridFTP servers in virtual machines on DDN’s storage controller eliminates the need for external data transfer nodes and network adapters."
"We sustained a data transfer rate of 85 Gbps for over 60 minutes — and occasionally for as long as 90 minutes — several times during the SC14 conference."
"This demonstration was aimed at bringing together the experts and latest developments in all aspects concerning disk-to-disk WAN data movement, including network, storage, and data movement tools," Kettimuthu added.
The team expects that the approach can be used to achieve 100+ Gbps wide-area transfer rates between storage systems using multiple WAN paths and additional storage resources in the end systems.
