Sungard Financial Systems is relying on Google’s cloud expertise in a bid to land a multi-million dollar contract from the U.S. Securities and Exchange Commission. The two firms are working together to build a prototype cloud computing system that can store six year’s worth of financial trading data, as dictated by U.S. law.

Sungard is competing with six other firms to win the Consolidate Auditing Trail (CAT) contract from the SEC, which has tasked firms with building a system that adds more transparency to U.S. financial markets.

“It is the biggest big data problem in the financial industry today,” said Neil Palmer, Sungard’s chief technology officer, to ComputerWorld.

Sungard’s and Google’s prototype cloud was unveiled at the Google Next conference in New York last Friday, and Palmer said the cloud provides Sungard with the flexibility it needs to land the mega bucks contract. He said the project was advantageous when compared to building an on-premises system that comes with “too many unknowns”.

When it’s up and running, the CAT will generate masses of Big Data, recording every single quote and trade made by every company that participates in the U.S. financial markets. This data must be kept for at least six years, and companies are required to submit their trading activity on a daily basis.

Sungard said the system, once built, will ingest 100 billion events each day, amounting to roughly 50 terabytes of data. In total, that’s some 30 petabytes of data that will be built up over six years, Palmer said. Naturally, such a system won’t come cheap – the SEC estimates a cost of between $350 million to $1 billion.

Besides storing the data, Sungard will also have to provide analysts with the tools to dig into it. To that end, the company plans to use Google Cloud Storage to store the data, Google BigTable to structure the data, and Google Big Query to analyze it, either directly or with third-party business intelligence software.

The finished prototype was impressive, capable of processing 10 billion events per hour, or around 3GB of data per second. Even so, that’s not nearly fast enough – Sungard and Google said the prototype needs to process data four times faster in order to meet the SEC’s requirements.

Google’s Carl Schachter, vice president of the cloud platform, said at the event it would mark a landstone for the acceptance of cloud technologies in U.S. financial markets if Sungard wins the CAT contract, following in the footsteps of cloud disruptors like Uber Inc., and Airbnb Inc.

Not only that, but the system would obviously come in quite handy for the financial firms that actually use it. Palmer said it would mean companies would no longer need to store data in house, and there would be other advantages too, like being able to test algorithms against historical market data.

Epam Systems, Inc.; Thesys Technologies LLC; the Financial Industry Regulatory Authority (FINRA); a team comprised of AxiomSL and Computer Sciences Corp; and a consortium that includes Hewlett Packard Co. and Booz Allen Hamilton Inc., are the other bidders.

Photo Credit: Rusty Russ via Compfight cc