UPDATED 11:48 EDT / MARCH 26 2012

NEWS

Data Scientist Hilary Mason Speaks about Bit.ly’s Dragoneye and Real Time Social Data

Bit.ly is a URL shortener best known for living in a niche alongside such powerful contenders as Tinyurl, Google’s variant, and many others—the debate over the use of URL shorteners continues to be heated—but with services such as Twitter requiring less than 140 characters they’re here to stay with us. As a result, their use is almost ubiquitous across Tweets, Facebook posts, and other social media.

This means that data experts like Hilary Mason, the chief data scientist for Bit.ly, gets a glimpse at what the world is thinking on a moment-to-moment basis by digging through not just what people are linking, but what they’re clicking through on social media. As a data scientist, she works on projects for Bit.ly that are designed to help highlight and sift through information to permit end users (and enterprise users) better organize and use the tool that social media delivers them.

In an interview, Mason told The Next Web about her company’s plan to release the Dragoneye project to all users and how it might just help change the face of social search,

In the coming months, Bitly will release a few tremendously cool consumer and enterprise products built around its data. The product Mason is most excited about is called “Dragoneye”, which picks up phrases that are “bursting” disproportionately on the Internet. Mason gave me a Dragoneye demo on Friday, March 16th and unsurprisingly, the phrases “iPad” and “St. Patrick’s Day” were bursting in New York City.

While Bitly’s current Search product is only available to enterprise users, Bitly plans to release a new version around realtime links that will be available to all users. In relation to this Search product, Bitly will offer useful data feed products built around desired topics and locations. For example, if you want to know all of the celebrity gossip going on in a specific region or what people in New York City are tweeting about pizza, what’s trending from Etsy on Twitter or which American regions read Fox News vs. The Onion, Bitly has an API for you.

Mason spoke with SiliconANGLE.tv at Strata Conference in 2011 and outlined the necessity of some of these search projects. Such as how Google itself works with old data—sometimes mere hours hold, but in the social-information age that’s old enough when retweets take minutes to span the globe.

“When we think about real-time search, we’re trying to think about helping you discover the information you will want to know as soon as possible,” Mason said, referring to the necessity to be able to pull the signal from the noise in the context of what people are currently talking about. Certainly the Dragoneye project will be a vanguard of this sort of technology.

She described the information coming to individual users as a “firehose” that gushes news, gossip, rumors, trivia, and other bits of data without concern for relevance or organization. As a result, information overload for real-time and social search is becoming a real hassle for everyday people. We don’t have time to sieve and filter everything that comes to us during our lives, so the necessity if not the niche for a service that can learn what we like to see (or what we might be interested in) and float that to the top and skim the cream from the incoming information.

While she mentions in the interview with TNW that this is an awe inspiring experience, knowing what people are thinking during a given moment, she also mentions that it’s a very human experience. While most Bit.ly data scientists hope an expect that their users are high minded individuals looking into space exploration and reading articles on mapping the human brain—actually there’s a great deal who are more interested in trading salacious images of hot actresses. At least Bit.ly is nothing if not a cross-section of Internet humanity.

Undoubtedly, the entire information engagement thousands of individuals will be necessary to use big data to sift through the noise to discern the signal. After all, some users may want racy images of famous celebrities to appear high in their news that day and an information system would have to be content agnostic in order to better suit that need; of course, it also means that other needs would be equally treated for those looking for the newest neutrino research from particle colliders across the world or news from NASA.

“Our job at Bitly and in the data science group is to understand what the world is paying attention to and in real time. We’ve developed new angles on the data set, some are mature and some are a long way from maturity.” In essence: “At Bitly, we don’t just shorten links anymore,” she concludes.

With Dragoneye and other projects, there’s no doubt that Bit.ly is sitting in a good place to provide users a better, more organized less cluttered experience with their social media.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU