According to Jay Bretzmann, director of eServer products at IBM, the company could have put a 32-way Xeon server in the field with its Summit-II EXA chipset. But with Intel running late with its Potomac and Cranford 64-bit Xeon processors (which were originally due at the end of 2004) and Microsoft slipping on its 64-bit Windows Server 2003 operating system and Yukon SQL Server database release, putting a 32-way machine in the field that could only address a maximum of 64GB of main memory was not a good balance for the server.
This is why IBM held off on delivering its 32-way machine until the Hurricane chipset was ready, Intel got the 64-bit chips out the door, and Microsoft got the 64-bit version of Windows out the door for servers. (The company is still working on getting 64-bit Yukon done, however.)
With the new xSeries 460, IBM is supporting 32-bit and 64-bit variants of Windows Server 2003 and 32-bit and 64-bit variants of Linux 2.6, specifically the Linuxes from Red Hat and Novell, which are IBM’s key Linux distributors on X86-X64, Power, and mainframe servers. The xSeries 460 is based on a four-socket, 3U chassis, and to scale up to 32-way processing, it uses fiber optic interconnections to link the four-way boxes into a single system image.
Like IBM’s Power-based servers and prior Summit and Summit-II chipsets, the Hurricane chipsets use NUMA technology to glue multiple server cell boards together to create a single system image that looks more or less like a big SMP server to operating systems and their applications. While the Summit-II chipset was based on an eight-socket, 4U chassis, IBM decided with the current round of enterprise-class xSeries servers to use the same chassis in the xSeries 366 (a four-way that only supports the Cranford Xeons and not the Potomacs) as a building block component.
Cramming eight processors and their memory in the box was possible with 3GHz, 32-bit Gallatin Xeons, but with faster 64-bit Xeons that have higher clock speeds and bigger caches as well as support larger chunks of memory – and with dual-core Xeons on the way next year – IBM decided that it would be best to go a little less dense and make the thermals not so constrained with this line of xSeries products. To make a fully loaded 32-way, customers have to link together eight of these rack-mounted chassis.
The xSeries 460 supports all three variants of Intel’s Potomac chip: a 2.83GHz version with 4MB of L3 cache, a 3GHz version with 8MB of L3 cache, and a 3.33GHz version with 8MB of L3 cache. These 64-bit Potomacs have little bit more clock performance and twice the cache of the fastest 32-bit Gallatins, but the Potomac processors can support 64-bit extended memory as well as older 32-bit modes; they also have a 667 MHz front side bus.
It is reasonable that a 32-way box should be able to do about 2.5 to 3 times the work of the eight-way box – and might be able to do more if IBM’s Hurricane chipset can reduce the latencies between memory in the nodes in the server cluster. We’ll have to see. Compared to the 32-bit Gallatin servers using the Summit-II chipset from IBM, the xSeries 460 has about 60% more oomph, which is a pretty big jump in performance for a jump to a new server generation.
As for future scalability, don’t expect IBM to push it beyond 32 sockets any time soon. According to Mr Bretzmann, both Windows Server 2003 and Linux 2.6 top out at support for 64 simultaneous threads. With HyperThreading turned on, that means a 32-way xSeries 460 can’t do any more threads running Windows or Linux.
By this time next year, Intel will be shipping and IBM will be using dual-core Xeon processors with HyperThreading that will put as many as 128 threads in a top-end 32-socket server. If the market requires it, the Hurricane architecture probably can expand more, but a 64-core Xeon server with 512GB or 1 TB of main memory is a very big machine, and a lot more than most customers will need. The current xSeries 366 and xSeries 460 machines will support future Intel processors, by the way. We have created an architecture that can run the next several generations of Intel chip technology, Mr Bretzmann boasts.
IBM is also not interested in using the Opteron processors from Advanced Micro Devices in the servers that support the Hurricane chipset, and as we reported earlier this year, Big Blue is not going to support the Itanium processor with Hurricane, either. While the Summit family chipsets had an Itanium variant, IBM believes that by tightly linking to the Xeon X64 architecture, it can build a server that meets the needs of Windows and Linux customers without having to support Itanium, too.
