Sungard is teaming up with Google as it targets a lucrative contract with the U.S. Securities and Exchange Commission.
Sungard aims to use a prototype cloud system from Google that could store six years’ worth of stock and financial trading data. The system would have to allow regulators and stock traders to scrutinise the data.
The Consolidate Auditing Trail for the SEC is aimed at providing more transparency into financial markets, partly in response to the computer-driven 2010 "flash crash" which briefly crated U.S. stock prices.
It is expected that the system would cost anywhere between $350 million and $1 billion to build, according to the SEC.
Once operational, it would be required to record every quote and every trade from every financial company which is participating in the public U.S. markets.
This data must be submitted on a daily basis and the system would be required to store the data for six years. This means that on a daily basis it would be required to ingest 50 terabytes of data from about 100 billion events.
Sungard estimates that the six-year window would mean that 30 petabytes will be actively kept.
The prototype uses Google’s Cloud Storage to hold the data, BigTable to structure the information and Dataflow to validate it. Big Query can then be used to publish the data and additionally provide ways for users to analyse it.
The first round of testing showed that the prototype could process 10 billion events an hour, which equals 3 gigabytes of data a second.
Other companies which are competing for the contract are Epam systems, Thesys, FINRA, AxiomSL with Computer Sciences Corporation and a consortium which includes HP and Booz Allen.
