Level 3 communications raised a lot of eyebrows earlier this month when it outbid rival Content Delivery Network (CDN) provider Akamai to deliver Netflix content to large parts of the United States.  The announcement was a huge win for Level 3′s CDN business and it meant leasing around 2.9 terabits of distributed capacity to Netflix.  But there’s just one little problem: Level 3 won that bid because it intends to break its contractual obligations on peering with Comcast and essentially resell stolen bandwidth to Netflix.  Now it makes perfect sense how Level 3 managed to outbid Akamai since no CDN provider operating legally could outbid hot goods.

So how does Level 3 intend to get away with this?  Just put it under the “Net Neutrality” violation banner and invent a story that Comcast is blocking content when it merely wants to enforce existing contractual agreements with Level 3.  Then as expected, we get ignorant groups like Media Access Project and bloggers like Stacy Higginbotham claiming that “Comcast might break the web” despite the fact that the kind of contractual agreement between Comcast and Level 3 is typical of how the Internet has always worked.  On the contrary, allowing Level 3 to violate their peering agreement under the banner of “Net Neutrality” is what would actually break the Internet and turn peering and Internet investment economics on its head.

What makes the Internet work

The Internet is comprised of privately built and privately operated networks that connect on a contractual basis.  These contracts are mutually agreed upon and there are generally two types of contractual agreements.

Transit agreements

Peering agreements

A transit agreements is when a network carrier delivers data between two other networks, and both end point networks have to pay the central carrier to carry that traffic at a metered rate.  So if Comcast and Level 3 communications had no direct method of connecting to each other, they must each pay a third party network carrier to connect their two networks.  Transit agreements are usually more expensive than peering agreements for every packet delivered and they’re generally slower because of distance and congestion.  According to DrPeering.net, typical Internet transit rates are $3 to $10 per Megabit per second (Mbps) per month for server bandwidth.

Peering agreements occur when two networks have a direct physical connection to each other.  A physical connection exists when it is financially feasible and when there is a business agreement in place.  Physical connections are often practical to construct since many end point networks meet at Internet Exchange Points within the same physical building and that merely involves connecting a few in-building Ethernet cables on some gigabit or multi-gigabit switched ports.  The business agreement requires that both parties come to mutually agreed upon terms on a financial settlement scheme.

The settlement scheme can involve a fee based agreement or it might involve no money changing hands.  When one network sends more traffic to another network than it receives, it usually has to pay a metered fee on the extra traffic that it sends.  When the two networks are more or less sending the same amounts of traffic to each other, the two networks can agree on settlement free terms where no money changes hands.  These terms of settlement might sound arbitrary and unfair to outside observers that smacks of “might makes right”, but it is based on sound economics that ensures continued investment in the Internet.

The network sending more traffic does so because it has cheaper-to-serve customers than the network receiving more traffic.  The network receives more traffic because it has more “eyeballs” i.e., broadband consumers who are hundreds of times more expensive to connect than the sending network’s customers which are the Netflix and YouTubes of the world.  Providing a gigabit per second (Gbps) of capacity to a server in a data center might cost a few hundred dollars of capital expenditures but an additional gigabit of last-mile broadband capacity could cost millions in capital expenditures.  Another example of one network that might charge another network to peer is when one network operates more long distance lines which are more expensive to build and maintain, especially those going under the oceans connecting continents.

A network that didn’t build the expensive infrastructure can’t just expect to use those networks without paying to use them.  The network that spent all the money up front in capital expenditures provides value to the network that spent little money building the network and it needs to recoup its costs by charging the network that didn’t build the most expensive part of the network.  Settlement based peering is the mechanism than ensures that all players on the Internet pay their share of expanding the Internet.  This is the only way to ensure continued private investment and growth on the Internet.

How Level 3 intends to steal bandwidth from Comcast

Before Level 3 Communications landed the Netflix CDN deal, it sent approximately the same amount of traffic to Comcast as it received.  Because of that, the two networks exchanged services of roughly equal value and they agreed to a settlement free peering contract.  The two companies agreed not to charge each other for peering connectivity so long as the traffic exchange levels remained more or less the same.

But with the massive new Netflix CDN deal where Netflix is currently the largest source of traffic in North America, Level 3 will likely start sending 5 times more traffic to Comcast than it receives.  That would violate its current settlement free peering agreement and it would require a new fee based agreement where Level 3 has to pay Comcast for the extra traffic it sends to Comcast.  That makes sense as Netflix’s old CDN provider Akamai paid to peer with Comcast, but Level 3 decided that it would simply insist on violating its existing free peering agreement with Comcast which would allow it to outbid Akamai on the lucrative Netflix CDN service.  Level 3 would essentially steal bandwidth from Comcast to outbid Akamai which pays Comcast for bandwidth.

From Comcast’s perspective, this impacts them severely as they’re losing bandwidth revenue from Akamai since Akamai lost Netflix business to Level 3, but Level 3 will refuse to pay for the bandwidth they use from Comcast by free riding on their existing peering connection.  Even without the bogus Net Neutrality claims, Level 3 can cause major problems for Comcast if a peering dispute turns ugly.  If Comcast insists on enforcing its contractual agreement and charges Level 3 and Level 3 refuses to pay, there’s little Comcast can do about it other than to sever the peering connection and send Level 3 traffic to its Internet transit provider that will eventually reach Level 3.  But with most peering disputes, the contract violator will simply refuse to return to the Internet transit path and they’ll keep sending traffic down a broken unpaid pipe to Comcast which breaks network connectivity between Level 3 and Comcast completely.

Each will point the finger while customers on the two networks are cut off from each other and it makes both providers look bad.  The reputation of both companies will be harmed and it will essentially be a game of “chicken” to see who blinks first.  For smaller peering disputes that don’t garner much attention, the dispute is settled in a court or arbitration while the network is cut off for days or weeks.  For larger peering disputes that receive media attention and congressional attention because fuming voters are calling their representatives and senators, it turns ugly and may lead to calls for additional Internet regulations which no one on the Internet wants.

With the added confusion of “Net Neutrality”, the hopeless pro-anything-labeled-Net-Neutrality biased blogosphere and the FCC proceedings on the Comcast NBC merger, Level 3 figured that they had a good chance of getting away with bandwidth theft or at least forcing much more favorable rates from Comcast.  But not only is Level 3 trying to steal bandwidth, they’re a hypocrite because they’ve fought just as vigorously against other bandwidth thieves that tried to violate Level 3′s peering agreements.  Joe Waz of Comcast pointed this out by citing Level 3′s own words when it came to the sanctity of peering agreements.

“To be lasting, business relationships should be mutually beneficial. In cases where the benefit we receive is in line with the benefit we deliver, we will exchange traffic on a settlement-free basis. Contrary to [other ISPs] public statements, reasonable, balanced, and mutually beneficial agreements for the exchange of traffic do not represent a threat to the Internet. They don’t represent a threat to anyone other than those trying to get a free ride on someone else’s network.”

Of course Level 3 wasn’t actually being genuine when they said that.  They only meant it when someone else tried to free ride on Level 3′s network but they think it’s just fine if they free ride and steal bandwidth from Comcast.

A possible solution to free riders

I think there might be a better solution than severing a peering connection that goes unpaid.  What Comcast or any other network encountering a free rider should do is simply enforce a symmetric rate on their peering connection.  If Comcast sends 1 Gbps of traffic to Level 3 Communications, then they should enforce a 1 Gbps settlement free receive rate from Level 3.  This might result in Level 3 failing to deliver the performance they promised to Netflix.  If Level 3 can’t deliver an adequate service level to Netflix because they refuse to pay for the extra traffic going to Comcast, then Netflix is well within its rights to terminate the new agreement with Level 3 and return to honest CDN providers like Akamai.  From a Net Neutrality standpoint, Comcast would not be discriminating against any source or application type in particular and they would only be enforcing existing contractual agreements with Level 3.

[Cross-posted at Digital Society]