Big Data Search Engines are Big Right Now – Stremor Announces Heuristic Engine

Big Data is all the rage right now. Search engines value appears to be polluted with SEO and social networks are booming. Graph search is hot and Big Data is smoking hot. Startups are constantly trying to add Big Data to their pitch deck.

Companies looking at a few thousand data points or want to put Big Data and analytics in their business plan to appeal to investors. Stremor Corp, a startup based in Scottsdale, AZ thinks a little bigger.

Stremor is announcing a new kind of search engine. They call it a language heuristics engine. Truly the “Holy Grail” of Big Data, a system capable of taking unstructured data from emails, web pages, and any document that uses sentences and converts it to structured, minable, normalized data.

Many startups are hitting the scene with new ways to handle search in this new modern era of the web. Stremor took that approach. So they didn’t launch with enterprise products, they launched with consumer products with massive scale. “We wanted to prove we could do enterprise scale, so we skipped enterprise and went for Internet scale,” says Stremor CEO, Bill Irvine, “Crawling every page on the Internet seemed like the best way to prove we could handle large document libraries.”

In talking to Brandon Wirtz, Stremor’s search engine goes beyond than just searching the web. It creates abstracts of the content it finds. Unlike Google, which gives users sentence fragments with the keyword highlighted, Samuru returns actual summaries of the content.

“On the backend we are doing a lot of very complex things,” explains CTO, Brandon Wirtz. “We are looking at writing styles, presence of facts, statistics, sentiment, author purpose, subject matter, vocabulary, and about 40 facets of language. We don’t put all of that in the UI; it would be overwhelming. But we are making decisions in the results based on that information.”

By crawling the entirety of the web, Stremor plans to offer brands more information about market research than any other company has ever been able to do. Today research about sentiment, user feedback on features, and metrics about Key Performance Indicators from marketing campaigns, rely on search to find places where a product was mentioned.

“We cut out the middleman, which gives us access to a lot more data,” says Wirtz. “We think big, and if you only look at the data you can discover you aren’t getting the full picture. We felt that had to be addressed.”

My Angle: Search Engines Better Get Busy Living or Get Busy Dying 

Everyone knows that I love anyone who is attacking the “old” search market.  We need to see a better user experience via search technology for our new connected always on social web/internet. The old way of keyword search with a polluted set of results has to go.

We live in a distributed environment where people, content, applications, and data are nodes on a network.  Constantly changing so cloud, social, and mobile are converging (as I predicted in 2009) where these new nodes have to managed, rebalanced, and optimized behind the scenes.  That’s compute and big data in action all under the covers.  Brandon’s vision with Stremor offers a glimpse into the science and possibly the automation of this new search model.

Exciting times in search.  The old guard will have to innovate or die.  Just look at Microsoft.  They spend $12 billion dollars on Bing to build an “old” search engine and they didn’t even put a dent in the market.  Meanwhile Google, Yahoo, and Facebook are retooling and looking good right now.