Marketing folks love click stats, and tiny URLs spread clicks across numerous spaces in social media. Google, Facebook, Twitter and others already have their respective shortening service. URL shorteners give them even more data and makes them even smarter repositories of consumer information. It’s useful for Google’s search algorithm, and as Twitter and Facebook get more into paid services, it gives them even more data.

Bitly, one of the most popular URL shortener services and most well-known independent URL shortener, has some ideas on that.  The service has added several features that allow users to save, customize and update their shortened links to be public and private. At the recently concluded Bacon conference, Sean O’Connor, Lead Application Developer at bitly, shared some insights into Bitly’s distributed system and how Bitly is using big data analytics to make money.

Bitly distributed analytics system

When Twitter announced the launch of its own URL shortener in June 2010, many people thought that Bitly would close it services. Instead, the service has been innovative in offering its expertise to companies. To make marketing lemonade from all this data flowing in, the company offers a dashboard for businesses to track their links on the web.

Bitly distributed systems process 6 billion clicks a month, 600 million URLs every month shortened with the service and hundreds of millions of web pages crawled from the seller. The platform uses HDFS, S3, Nagios, the real time distributed messaging system Nsq, hostpool and a few different databases.

Distributed systems can often be challenging to build and operate, but they can offer significant benefits in terms of availability, cost effectiveness, and capacity. Specifically, Sean speaks to the benefits of Service Oriented Architecture, using asynchronous streams, scaling, dealing with failure, and monitoring.

Many services on asynchronous communication

In the case of Bitly, the company has 50 people, amid whom are approximately 20 engineers. The distributed system operates 400 servers including nearly 30 servers for handling all incoming traffic from the outside world including shortens, redirects, API requests, web UI, etc.

Connor shared that the URL shortener does not rely on a central application. Rather, the different services work independently and communicate asynchronously with each other. For example, service box A does not wait for a response from service B, but assumes that it has received the message. If service B have problems, the message is queued and executed at a later time.

This approach makes the handling of errors in one of the participating systems easier because of the failure of a single service is not a major problem for the rest of the system. However, the actual URL shortener service operates synchronously. This is to increase the speed.

When a Bitly URL is decoded, an HTTP redirect reflects the message to multiple services: an archive of the message is saved in HDFS and S3, a real-time analytics service receives the message, and a longer term history analytics platform also digests the data.

The Bitly Metric system again operates asynchronously, since a slight delay in the evaluation of analytics data does not constitute a broken leg, according to O’Connor. Data about the conversion is popped onto a queue for the analytics and other downstream systems.

In addition, Bit.ly relies on the self-developed software HostPool. The tool helps to manage the distribution on different hosts by being informed by the client for failed requests.

Lesson learned

Connor says the inclusion of a service-oriented architecture (SOA) has reduced functionality error while processing requests. In addition, individual components are isolated and different processes can run concurrently with Bit.ly by the combination of SOA, queues and asynchronous communication.

This asynchronous approach isolates components, speeds up the request and helps keep services focused. It also prevents disruptions from cascading through the message chain, since each service can just slot messages into a queue and does not need to time-out waiting for a response, it can go about its business even if something is offline.

Sean O’Connor’s lecture offers some interesting insights into Bitly work with distributed systems. The more you understand the distributed system, the better decisions you can make and the more efficiently you can work.