View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
November 17, 1993


By CBR Staff Writer

In response to challenges about the Transaction Processing Council’s credibility, brought about The Standish Group’s claims earlier this year that Oracle Corp had hyped up TPC-A results, the Council has now published a new set of guidelines for benchmark specifications. Called Clause 0, the guidelines have been designed to attempt to prohibit over-aggressive implementations of any part of the benchmark specification by test sponsors. This means that the Council now requires all companies to comply with certain restrictions to stamp out special benchmarks. In the past, some manufacturers have designed benchmarks to pretty up performance ratings by using special switches during testing, such as the discrete transction option. The new Clause 0 guidelines demand that all benchmarks include documented and publicly published results; and that the product being tested is used or can be bought by end-users in the market area that the benchmark represents. In another announcement the Council has also made it mandatory for suppliers to have independent audits done of their results. The Council will license its own set of auditors who will provide independent verification of results before they are published. Previously, the Council only recommended the use of auditors. Manufacturers will also have to bear the extra cost of using an auditor. In a letter to Council members, Kim Shanley, the Council’s chief executive, says that the organisation’s credibility had been undermined by being too lax in the enforcement of the spirit of benchmarking. Recognising this, the Council has now banned all benchmark specials and benchmark validation now has to go through a three tier submission process. First the benchmark has to comply with the new Clause 0 restrictions and then the results are validated by an external auditor. The results must then be registered with the Council, where they will be reviewed over a 60-day period by the Council member review board for final ratification.

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.