Next generation data mining, exploiting the internet and performing mining in a distributed mode is one step away from cost-effective imlementations, according to Chicago-based Magnify Inc. Working with two academic institutions in the so-called Terabyte Challenge, Magnify has been demonstrating high performance mining and intensive computing on massive data sets. Magnify’s Model Interchange Format specification, soon to be published, enables data mining models to be built and stored in an open format, as opposed to the centralized operations of most development efforts. The model appears to be highly applicable to the web , and a raft of developers are working on how to exploit this extra-wide technology that, apparently, locates data, integrates it and makes end usage easier the previously. Robert Grossman, president of Magnify Corp, explains the rationale, saying: It is common today to have data warehouses that are over a terabyte in size, but there are open issue of how best to perform critical tasks like data management, analysis and mining.
