By Timothy Prickett Morgan

The applicability of the TPC-C benchmark in today’s evolving e- business world can certainly be called into question, but it is still the only industry-standard online transaction processing benchmark that IT managers have to compare the relative performance and price/performance of Unix, NT and other proprietary servers. Although the Sun Microsystems Starfire Enterprise 10000 servers have been on the market for two years, the company has yet to run the TPC-C benchmark on them. After aggressively running tests in 1997 and early 1998 to show how its less powerful Unix servers compared to their midrange competitors, Sun has always stopped short of running the tests on the E10000s. The company did this probably because, like HP, it has found out that the TPC-C test makes the machines look like they aren’t as powerful as they seem to be based on raw performance metrics.

For one thing, the TPC-C test, which is based on the rudimentary online and batch data processing associated with running a warehouse (a real one, with forklifts, not a data warehouse), is a relatively light workload compared to, say, SAP’s R/3 ERP suite. A big mainframe-class server like a Hewlett-Packard V2500 or a Sun E10000 might be able to handle only thousands of SAP R/3 users, but as much as 90,000 or 100,000 TPC-C users. There aren’t many companies that want to support that many users on one system inside their corporate walls, so the validity of TPC-C is, at this point, suspect.

Perhaps that is why Sun Microsystems, in the aftermath of its Starfire+ announcements in February when the company announced support for its 400MHz UltraSparc-II processors in the E10000 line as well as a faster 100MHz Gigaplane-XB crossbar interconnection, has started talking about other types of performance metrics for its machines. For one thing, Sun is constantly throwing around mainframe-equivalent MIPS numbers which may or may not be meaningless since the Starfires don’t run IBM’s OS/390 mainframe operating system or its DB2 and IMS data bases. Sun is claiming that a 64-way Starfire+ has about 3,000 aggregate S/390 MIPS of processing power, compared to the 1,069 MIPS of IBM’s ten-way G5 series 9672-RY6, and is similarly claiming that Starfire+ servers cost only $1,000 per MIPS compared to $3,700 per MIPS for real mainframes.

Sun has also told potential customers that a 64-way Starfire+ can deliver more than 1,400 HTML-to-database transactions per second to web clients. When you do the math, that’s 84,000 transactions per minute or 120 million transactions per day. Those numbers are derived from a set of inhouse benchmark tests based on e-business applications built in Java using its recently NetDynamics 5.0 web application server. The Java applications managed by NetDynamics accessed information stored on Oracle 8i data bases and displayed their pages using Netscape Enterprise Server V3.6. These numbers also suggest that if the Sun NetDymanics benchmark bears any relationship at all to the TPC-C test in how it loads up a system that a Starfire+ server will not be able to break the 100,000 TPM barrier, much as HP and Sequent have, as yet, been unable to do. á