Erik Voorhees the CEO of ShapeShift AG, a unique registration-less cryptocurrency exchange, went to the Internet to provide a full post mortem of the hack that recently shut the service down and lost the company approximately $230,000 USD in bitcoins. It is a story of insider betrayal and recovering from a hack told with a level of transparency rarely seen in any security breach.

The early news of the hack is also covered by SiliconANGLE’s Duncan Riley, from the initial shutdown on April 11 to the discovery that it was an inside job (after an inside job).

Names were changed to protect the guilty when Voorhees tells a tale that shows an epic struggle and a coming to understanding about security needs for even a small startup. Many cannot absorb the loss of $230,000 in assets (an amount made “small” by ShapeShift’s unique business model) and it also speaks to the need for a conclusive understanding as to what an attacker can do (especially with inside knowledge).

Although ShapeShift got the shaft (more than once), the company did do some things right. One of them was calling in an outside auditor to get to the bottom of what happened. So after reading the amazing story (summary below), also read the audit by Michael Perklin from Ledger Labs.

To begin, Voorhees fingers a man named “Bob” (name changed) for his role in the attacks, starting with an initial hack that occurred earlier in March. The job Bob was hired to do put him in the perfect position to steal from ShapeShift being that he was hired to build out the company’s server infrastructure—for a small startup this unfortunately means Bob wore many hats: office IT, server system administration, security, and infrastructure management.

In information security matters, insiders are amid the worst dangers because they are given a level of “trust” that outsiders are not. In fact, most hacks work by escalating trust through networks to turn an insider into a conduit for attacks, often through trickery or Trojans. Most hackers don’t sneak in through the firewall: they come in through the front door or via an “authoritative” voice over a phone line.

In this incident, 315 BTC (approx. $138,000 USD) was stolen on March 14, 2016 and placed in address 1LchKFYxkugq3EPMoJJp5cvUyTyPMu1qBR (it’s still there as of writing).

Bob’s case gets even more interesting in that when Voorhees and ShapeShift went to file civil and criminal charges against him he ran—and left his dog behind. Voorhees claims that Bob had a pet dog that he brought with him to the office. When Bob realized he’d been caught he did take his dog with him, but it was eventually discovered that he left the dog with a neighbor (and never came back).

Image via Pixabay.

A blatant inside job and ShapeShift moves to clean up, but it’s not enough

Of course, after Bob’s abrupt departure and mounting evidence that he was the thief, the ShapeShift team went to work in entirely changing their server infrastructure. Switching from the hosted setup the system ran on before and moving to a cloud solution, which Voorhees calls “CloudCo” (to protect the innocent). All seemed to be going well until April 7, 2016.

“Then all hell breaks loose. Again,” writes Voorhees.

In spite of the brand new infrastructure, a totally new hosting company (CloudCo) and no Bob, the exchange gets its hot wallets hit again with bitcoins, Ethereum and Litecoin (two other cryptocurrencies held by ShapeShift) looted by the attacker.

Voorhees manages to track the hacker to an exchange (by following the bitcoins) and finds an e-mail address, to one “Rovion”, which miraculously actually connects to the attacker. The answer is one that shows that while Bob may be excised from the company, his legacy remains.

“Nice job on the hack. How did you do it?” asks Voorhees.

The reply from Rovion: “One word: Bob.”

photo credit: marsmet548 via photopin cc

In which ShapeShift learns it’s very hard to clean out insider damage

Now that CloudCo’s setup was compromised, ShapeShift again switches gears and takes up with a new company (they’ll call them “HostCo”). To make this story short … ShapeShift’s team manages to rebuild the entire infrastructure (again) on HostCo’s servers and re-launches the site again within 24 hours.

And a day later … the site gets hacked again. Starting to feel the déjà vu? Voorhees certainly was—at this point it’s all about confusion and fury. “Is this the fucking apocalypse?!?” Voorhees asks himself.

Needless to say, it’s time to make this stop. For the safety of the company, for the safety of the customers, and for his own peace of mind. Voorhees has the entire operation suspended until the hacker can be uprooted and the security failure understood.

At this point he also does another very smart thing: he calls in expert help from the outside in the form of Michael Perklin, Head of Security and Investigative Services at Ledger Labs, who agrees to fly out to assist in determining what has led to this apocalypse.

At the same time, Voorhees does another interesting thing: he strikes up a conversation with Rovion, the hacker (after all they’re strangely on speaking terms from earlier). During a conversation with the hacker Voorhees manages to pay him for some information (2 BTC or about $880) about how the hack worked. And herein is revealed how pernicious an insider can be as Voorhees learns that had already been attempting to break into the system when Bob pulled his heist and that Bob had sold him information on ShapeShift for BTC.

The conversation with the hacker would have ended there—and Rovion did end it then—but then something amusing happens. Rovion can’t seem to sell his stolen Ethereum because exchanges are freezing the assets (because it was stolen presumably) so he goes back to Voorhees and offers to sell the Ethereum back at an insane discount for BTC and more information on the hack.

Voorhees accepts, of course. “We’d be essentially buying back our own Ethereum, and paying him Bitcoin,” he writes. “Obviously worth it, if we can obtain more information.”

The damage an insider can do and a lesson learned

Rovion reveals that Bob sold him the ShapeShift source code, IP addresses for core servers and an SSH key. Using this information Rovion installed a backdoor on the server (which was discovered by Perklin’s investigation).

However, what’s interesting is that the SSH key is not an old key—it’s a new key added for the CloudCo environment almost a week after Bob’s untimely ejection from the company.

Then the other type of damage an insider can do comes out: Bob had installed an RDP (Remote Desktop Protocol) server on one of his coworker’s machines. Rovion reveals he used this RDP server (giving him access to the internal office network) to hack ShapeShift a second time. This is how Rovion got that newly minted SSH key.

At the end of their conversation, Rovion left this gem:

“EVEN THOUGH I SAID CEASE COMMUNICATION, CAN YOU STILL SEND ME AN EMAIL WHEN BOB GETS SUED/WHATEVER IT IS YOU’RE GOING TO DO? I FEEL IT’S REALLY SHITTY TO STEAL FROM YOUR OWN EMPLOYER.”

Well there may be a quote about honor and thieves, but some people have their own sense of loyalty.

Erik Voorhees’s ShapeShift cryptocurrency exchange requires no log in.

Taking everything down so it can be built up again

There’s a lot of lessons to be learned from this but one of the biggest ones is that when an inside job happens: everything is suspect.

It may be very hard to stop an insider (since, by definition they already have trust), but it is possible to limit the amount of damage done with the proper security protocols. Worse, though is that after an inside job happens everything that the insider had access to could become another vulnerability and that’s what ShapeShift had to learn the hard way with the RDP server.

Voorhees made the decision to shut everything down when it became obvious the company had an unresolved compromise. That led to an audit by an outside expert and that in turn led to the realization that anything that could connect to the servers needed to be wiped and rebuilt clean. This added more down time, but it also meant the job could be done right and avoid any more lurking vulnerabilities.

Another thing to note is that ShapeShift is actually a little less vulnerable than the everyday cryptocurrency exchange when it comes to this sort of hack. The $230,000 USD total damages in stolen currency happened from a hot wallet owned by the company and none of that was coin held for customers.

ShapeShift’s model works by facilitating a trade quickly between people who want to exchange cryptocurrencies in a fashion that Voorhees has described as meaning no customer coin is ever held by ShapeShift. The service also does not require registration or log in. As a result, even if it’s been a day of massive trades, there’s no pockets to rob or records to steal. Due to this the service is not exactly ideal for high frequency or big volume traders, but it’s still a unique service for people who just want to trade.

Finally, this is one of the few times in the industry that customers of any service have gotten such an interesting view of what happened and why it happened.

The Bitcoin community has seen some pretty opaque descriptions of hacks including what happened to BitPay in December 2014 and details of the apparent inside job that took down Mt. Gox. Of course there’s also a long string of exchanges that just up and evaporate, closing after claiming to be hacked but it’s hard to prove such as Cavirtex, BTER.com and others.

In conclusion, ShapeShift and Voorhees appear to have weathered a storm that could have laid them low. IT security is not an easy task and Bitcoin businesses are clear opportunistic targets for hackers because bitcoins have value and they’re stored electronically.

“We learned some of our own vulnerabilities, and our own mistakes,” writes Voorhees. “We are correcting them, and improving upon them wherever possible. Such improvement doesn’t come cheap, but the ShapeShift of today is made better than the ShapeShift of yesterday. The steel is tempered, the machine refined. Though no single organization can ultimately achieve it, we try to approach anti-fragility, and exemplify it as an ideal in our work.”

Still interested in the details? Take a look at the full story as told by CEO Erik Voorhees on Bitcoin.com “Looting the Fox: The Story of Sabotage at ShapeShift” and follow that up with the full audit by Michael Perklin from Ledger Labs. The entire case is extremely illuminating.

As Voorhees says in report about the hack: “Hackers gonna hack.”

And Bob … about that dog.

Image credit: Bitcoin Logo, Flickr.