Mobile computing and cloud technologies are two strong trends with the potential to help companies to become more competitive. The aim is to enable continuous delivery software solutions allowing companies to take advantage of market opportunities and better meet customer demands. Improving development processes for cloud applications is critical for success and for that reasons companies have created solutions to improve and streamline these processes.

The need to deliver more continuously is becoming more prevalent. Developers are trying to be more agile, trying to deliver software with agility. Real time traffic and log analysis is an important step in that direction. Log Analysis analyzes terabytes of data in IT asset logs to give developers insight into their applications’ performance.

Increasingly the companies who produce and support these products have to deal with not only the volume of data that comes out from these software components and connected devices, but the trend in 2013 is shifting to variety of other information that is needed to understand customer usage, problem diagnosis, installed base analytics etc.

Transitioning beyond log analysis

Splunk continues to enhance its flagship machine data search engine so it can be used by business analysts and managers. Splunk added Amazon Web services to its Cloud service this year to define the meaningful relationships in the underlying data, enabling business users and analysts to easily manipulate and visualize data. The service also includes access to Splunk Enterprise apps, APIs, alerting, and role-based access controls.

Splunk’s great success in providing the tools for a system administrator to delve into previously inaccessible log files has opened up the market for deeper analysis on data in log files in 2013.

IBM has created solutions to improve and streamline log data. The company introduced Log Analysis, part of IBM SmartCloud Analytics. Log Analysis can bring the power of automated analysis to records of IT assets from the terabytes of unstructured data produced from infrastructure and applications. Log Analysis lets you power the automated analysis to records of IT assets. It is a solution that helps companies monitor the performance and availability of applications hosted in a cloud in real time.

IBM Rational Test Workbench is an another solution that developers can record, modify, reproduce and evaluate the test scenarios to automate hundreds or even thousands of tests to which each model is subjected to a mobile device.

In an era where data analytics is giving businesses the insight to optimize their decision-making processes, log data provides an important layer. For Loggly, the San Francisco-based start-up, the focus is on serving companies that build applications in the cloud or for the cloud like AWS users. The company offers its software as a service (SaaS) to make it easier for developers to get started with log file data monitoring and analysis.

Logging is simply not enough

There is more to log analysis than IT logs and operational intelligence as logging is simply not enough for today’s complex business process. Modern applications require application performance management to enable application owners to stay informed to minimize the business impact of performance degradation and downtime. Many companies in 2013 soared to expand traffic analysis beyond log data.

Glassbeam is working to expand its machine data solution beyond log analysis. Glassbeam introduced a new cloud-based service for tapping operational data. The service called SCALAR leverages search and visualization capabilities to improve troubleshooting and create a central data repository for compliance and audit purposes.

The Mountain View, Calif.-based company, Sumo Logic, uses advanced machine learning algorithms to whittle down mountains of log file data into common groupings, making it easier for administrators to synthesize the information. Sumo Logic has access to log data from all of its customers, which can be mined in aggregate to identify security threats and other important behavioral patterns for the benefit of all.

AppFirst has developed a SaaS-based dashboard especially for DevOps who want to know what is going on in their application, systems and business. The dashboard monitors in real-time all data streams and collects a wide range of data sets, from log files, process data, Nagios plug-ins as well as StatsD. All data is stored, analyzed and aggregated in a clear and understandable dashboard.

Microsoft also jumped to log analysis business and has released a tool in 2013 that can be used by developers and IT professionals alike to check the performance of .NET applications. The Microsoft Monitoring Agent, is the successor product to Microsoft’s Operations Manager Agent. It includes the full functionality of the .NET Application Performance Monitoring tool in System Center, as well as the IntelliTrace collector capabilities. The Monitoring Agent can be used to monitor applications in production environments, or it can be used to collect system diagnostics, including event logs, traces and performance information.

Real-time monitoring and performance management

Real-time dashboard and visualization capabilities receive a great attention for companies this year. Some deployment tools, such as Jenkins, use plug-ins to monitor the executions of repeated jobs. It can also monitor the execution of job runs on remote machines, capturing their output for analysis by administrators.

Boundary, an application performance-monitoring provider, uses a software-as-a-service (SaaS) delivery model. Boundary’s cloud software is designed to allow for detection and resolution of failures in application and infrastructure. Boundary can peer deeply into a company’s environment and instantly understand its health in current and historical context, helping IT promote business agility in ways it couldn’t do before.

Other commercial tools that entered into real time trafficking and management include Nagios and Librato, Amazon CloudWatch (provided for and by the AWS environment), and Splunk. Open source tools include Graphite, Kibana, logstash, and more.

Logging is simply not enough information to get to the root cause of problems in modern distributed applications. Expanding the horizon to new level, developers in 2014 can expect something that integrates the information that comes from various functions, such as change management, version control, issue tracking, and monitoring, and that can be used by developers, operators, testers, and others.