Whether it involves online search or troubleshooting a thorny systems error, the name of the game is speed.

Yet, in a world where a navigation bar can deliver an answer for “today’s weather” or “the derivative of square root by sine X of first principles” in less than the blink of an eye, finding the cause of system failure in log data can take minutes or longer, an eternity in the world of information technology management.

This is the central issue that Scalyr Inc., a server log monitoring tools provider based in Silicon Valley, has sought to address since its founding in 2011. As log data has grown exponentially, so too has the need to manage it. And the key part of data management has been to find answers to problems and resolve them as fast as a user can locate the nearest ice cream parlor.

“We like to think of speed as the time from a question in someone’s head to the answer on someone’s screen,” said Steve Newman (pictured), founder and chief executive officer of Scalyr. “The question that someone wants to ask can be a little more complicated or the data needs a little bit of massaging. We wanted to take a step forward and create this more-advanced language for more-advanced queries.”

Newman spoke with John Furrier (@furrier), host of theCUBE, SiliconANGLE Media’s mobile livestreaming studio, during the Scalyr On The Ground event in San Mateo, California. Furrier also spoke with Scalyr executives Casey Clark, chief customer officer; Jeff Mathis, software engineer; and John Hart, principal software engineer, in separate interviews. They discussed how Scalyr’s technology was derived from Newman’s previous work at Google LLC, recently introduced features for Scalyr’s platform, and customer interest in the log management technology. (* Disclosure below.)

Watch the interview with Newman below:

Roots in Google Drive

Scalyr’s mission is embodied in its mantra — “Speed To Truth” — a reference to the use of its technology in empowering developers and operations staff to find the answers they need quickly among a mountain of complicated system log data. It’s a complex task, but Scalyr’s founder had seen it all before when his previous company became the primary document management application for the largest search engine in the world.

Newman’s former startup — Upstartle LLC — developed a document management tool called Writely, which drew the attention of Google. Writely was ultimately acquired by the search giant in 2006, and the technology became Google Docs and Google Drive, under Newman’s direction.

By 2018, Google Drive usage had reached 1 billion users, but long before then, Newman had become frustrated by a fundamental problem: It often took months to identify an issue in the platform’s complicated indexing system.

When Newman and his team needed to create new indexes for document sharing on Google Drive, the system collapsed upon going live because they had grossly underestimated the amount of data being generated through the Google Doc system. And one of the biggest clues for the failure was found in the signup sheet for the Google company picnic where metrics hid the complex ways that the document was shared by employee users, according to Newman.

“We would never have found that until we got way down into the details of the logs, which we didn’t have the tools to do, because we didn’t have Scalyr,” Newman recalled.

Powered by thousands of servers

Scalyr addresses this issue by essentially creating a way to use the massive compute power of a centralized system to serve its customers. Each customer gains access to a log management engine that is equipped to handle queries on a scale far greater than anything that might be available in-house.

It’s a model very similar to what happens when anyone around the world queries Google. “Literally thousands of servers in that tenth of a second that Google is processing that query, 3,000 servers on the Google side may have been involved,” Newman explained. “Those aren’t your 3,000 servers, because you’re sharing those with 50 million other people in your data center region. But for a millisecond, those 3,000 servers are all for you.”

This massive use of compute power represents a form of brute force to dramatically speed up the time needed to sort through log data and obtain an accurate and satisfactory answer to resolve a system issue. This kind of log management system must be flexible, exploratory and fast.

“Most of the time you’re going to your logs, you’re trying to troubleshoot a problem; and today’s problems are different than yesterday’s problems,” Hart stated. “The key to brute force is you’ve got to bring a lot of force as quickly as you possibly can. Our median performance is still better than 100 milliseconds.”

Watch the interview with Hart below:

New product in beta

To capitalize on that speed, Scalyr has recently unveiled two additional features designed to let users create advanced search capabilities. One is PowerQueries, launched in beta earlier this year.

Users can perform complex actions to group, sort and filter large amounts of information, combining real time use of data with ease-of-use and scalability.

“PowerQueries provide new insights into data with a powerful and expressive query language that’s still easy to use,” Mathis said. “The vision for Scalyr is to become the platform users trust when they want to observe their systems. PowerQueries is a really important step along that journey.”

The other enhancement, announced in November, involved an update to the Scalyr platform that allowed users to monitor issues in Kubernetes and Amazon Web Services CloudWatch. The added Kubernetes features enabled DevOps personnel to visualize logs at the cluster level.

Large companies drive adoption

Scalyr’s attention to speed and advanced tools for log management have allowed the company to build a customer base that includes over 300 companies. The current list includes firms such as NBCUniversal, OkCupid and Tivo.

“We were fortunate to land some very large customers that pushed us when we were seven employees three or four years ago,” Clark said. “That forced us to mature very quickly, and we immediately found traction. The product just stood up.”

Watch the interview with Clark below:

Scalyr’s founder walks the halls of his company wearing a T-shirt bearing the slogan “Love Log and Prosper,” a play on the signature phrase coined in the 1960s by the megahit TV series “Star Trek.” One of its central characters — Mr. Spock — played a Vulcan and would deliver the line “Live long and prosper” while forming a “v” salute with his fingers.

Like the “Star Trek” spaceship, Scalyr is seeking to enable log management to move at warp speed in keeping with the pace of IT demands across the enterprise.

“Whatever business you’re in, it has a digital element and your competitors are building sophisticated systems, adding functionality, and moving quickly. You need to be able to do the same thing,” Newman concluded.

Watch the interview with Newman and Mathis below, and be sure to check out more of SiliconANGLE’s and theCUBE’s coverage of the Scalyr On The Ground event. (* Disclosure: TheCUBE is a paid media partner for the Scalyr On The Ground event. Neither Scalyr Inc., the sponsor for theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE