By Rachel Chalmers
It was about the worst news the Linux evangelists could imagine: Microsoft Windows NT Server 4.0 is 2.5 times faster than Linux as a File Server and 3.7 times faster as a Web Server, announced a Mindcraft Inc study released on April 13. What the company didn’t disclose was that the study was commissioned by none other than Microsoft Corp. Mindcraft had conducted similar surveys for Microsoft in the part, comparing NT – always favorably – with Solaris and NetWare. The NetWare survey in particular had attracted the ire of Novell Inc, which accused Mindcraft of unfair practices and published a detailed rebuttal of the study and its findings.
That was all the encouragement Linux evangelists needed to argue the case for their own operating system. Through hacker site Slashdot and Usenet newsgroups, they set to work at picking the report to pieces. It was quickly established that Mindcraft had seriously misconfigured the Linux server used in the benchmark. The company used version 2.2.2 of the Linux kernel, even though 2.2.3 was available and had fixed the earlier version’s problems with TCP. Adjustments known to improve the performance of Samba systems were not performed. Mindcraft used a RAID controller, which is not well supported under Linux; another controller would have yielded better results. None of this would have mattered, perhaps, if the company had not spent so much effort optimizing NT. NT even got a 1012MB swap file; Linux got nothing.
What annoyed the Linux community most, however, was not the obvious unfairness of the respective configurations and the radically different levels of performance tuning. It was Mindcraft’s complaint about lack of support for Linux. We posted notices on various Linux and Apache newsgroups and received no relevant responses, the study claimed. The documentation on how to configure the latest Linux kernel for the best performance is very difficult to find.
Since the open source software community prides itself on its generous provision of information and help, this comment was calculated to sting. But it seems Mindcraft was stretching the truth, to put it mildly. The company called Red Hat, once, pointlessly seeking high-end performance tuning from a novice- level installation help line. Similarly, Usenet postings were sent to inappropriate newsgroups. Others tried to help, but to no avail. Mindcraft did in fact get a useful answer to its request for help tuning the Linux system, says open source evangelist Eric Raymond, but they did not answer the request for more information, neither did they follow the tuning suggestions given… Evidently Mindcraft’s efforts to get help tuning the system were feeble – at best incompetent, at worst cynical gestures.
Raymond sees an upside to the Mindcraft debacle. Microsoft’s underhanded tactics seem (as with its clumsy ‘astroturf’ campaign against the DOJ lawsuit) likely to come back to haunt it, he writes. And it’s hard to see how Microsoft will be able to credibly quote anti-Linux benchmarks in the future. His optimism assumes, however, that everyone who saw and was impressed by the original study also had access to the open source community’s extensive rebuttals. Not a hint of the controversy has shown up on Mindcraft’s site, or Microsoft’s for that matter. Both sides continue to preach to the converted.
