Further proving itself as a body to which the Unix industry should be paying a little more attention, the Association Francaise des Utilisators d’Unix has this month published a set of tests for benchmarking Unix and MS-DOS-based systems on which it has been working for over two years. It is called the Suite Synthetique de Benchmarks de l’AFUU, and the group hopes that it will be adopted by a new International Standards Organisation Posix workgroup on benchmarking, in much the same way as its model for applications portability. The tests have already become something of a standard in France – the French government uses the suite in making procurement decisions – and manufacturers have been falling over themselves to get their systems benchmarked. So far over 500 machines have been put through their paces. In addition to industry-wide performance standards such as Dhrystone, Whetstone and Linpack, the suite has a variety of tests proposed by users and developers at the French Unix user group meetings, which have been incorporated in the final selection of 200 programs. All the tests have been rigorously designed so that manufacturers cannot achieve performance benefits by tinkering with their systems. Indeed the Association is highly critical of the Systems Performance Evaluation Council, which is pushing to get its own benchmark suite acknowledged as a standard. The French Unix club reckons a pervasive combination of system manufacturers, and a limited range of company-proposed tests make the suite biased in favour of the member companies with little relation to performance users can expect from applications and packages at their screens. The Association says its own suite sets up over 30,000 processes on systems when the tests are run, and can take up to three hours to run, giving manufacturers little opportunity for intensive tweaking – cheating.
