The problem with P95

I attended a meeting last week with a customer in the Media & Entertainment space, and one of the topics of discussion was that the standard billing method that most ISP's use doesn't work well for their type of business.

The standard billing method that I am referring to is P95, or 95th percentile billing. What we do is measure the input and output traffic on each circuit connected to our routers. Our equipment collects the interface counters every 5 minutes (approximately) and divides the result by the exact number of seconds since the last collection to get the rate. For example, over a 24 hour period you will have about 288 samples for each direction (in/out). We take out the biggest 14 of those samples and then use the remaining largest sample as the billing rate.

Most of the time, this works great. On your typical large ISP traffic trend you'll cut off the very top of the peak providing a good billing rate to traffic volume fit, like this (the red line indicates the P95, the rate at which the customer is billed):

On a content providers circuit this works pretty well too. Even when they have an "event" where they double their traffic due as the result of a new movie trailer release or one of their other providers going down, they don't pay for the traffic unless it's a regular event.

In the media business, or at least certain segments of that space, it is common to see much more erratic traffic patterns, such as a line rate usage for a couple hours out of the day. Doing some reverse calculation, the 14-15 samples in a day that get discarded comes to 75 minutes, and if you transfer a 200 Gig file in the morning on a DS3, assuming 40Mbps throughput, that transfer will take 83 minutes to complete. After throwing away the top 15 samples showing 40Mbps, the 16th sample (which gets used for calculating the monthly bill) is also 40Mbps, so you got charged for 40Mbps of bandwidth even though you only used it for barefly more than an hour each day. The graph below does not perfectly illustrate the situation, but it is a weekly traffic graph from a real media customer of ours in Los Angeles.

There are two potential solutions for this problem.

Option 1) and probably the easiest option is to develop a pricing model that is by the Gigabyte instead of P95 of the rate. Since we know exactly how many Gigabytes are transfered, all we need to do is calculate a price/gig that will yield a more competive rate and remove some of the calculations from the billing and rating algorythm.

Option 2) and much more complicated is to build an event system where the amount of bandwidth can be automatically re-provisioned for the customer either as the result of an XML data input into our customer portal or as the result of some other automatic feedback mechanism such as link threshhold monitoring with an event trigger. Tie that in with automated network provisioning tools, and a billing system that can support a large number of pro-rated bandwidth purchases over a given month.

Why do the more complicated one when you've got an easier solution at your disposal? To solve the specific problem, option 1 is most certainly the way to go, especially in the short term. Option 2 has the advantage of setting the stage for a massive increase in functionality, particularly when it comes to having applications automatically manage how much of a particulary QoS/CoS they need in order to operate properly.

If you've got ideas on how Option 2 might pave the way for you to do some interesting things for your business, drop an idea in our suggestion box by leaving a comment.

dsiegel – Wed, 2006 – 11 – 01 18:45

Internet – IP VPN

dsiegel's blog

P95

I think Option 2 is very cool from a service providers perspective as it will help draw greater capital effeciency in the longer term..it also has some pretty extensive end user switicing cost implications as it matures...bring on option 2!

Anonymous (not verified) – Fri, 2006 – 12 – 15 11:58

option 1 is easily corrupted, but probably not insurmountable

Option 1 would have a problem out of the box because occasionally stats data is corrupted or missing. A single corrupted sample can ruin a month's worth of reporting if it's averaged in. Missing data wouldn't bother a customer too much, but the provider would want to have established a method for billing if some data is gone. A single missed day could reduce revenues by 3% or so in the month (althought it's possible that any single missed day also reduces the true P95 billed rate). I thought about moving from peaks to averages when I was reporting aggregate network stats, but I felt it was easier to be able to throw out some samples. Any move to averages should include discards for "bad" or interpolation for "missing" samples.

Nels Thompson (not verified) – Thu, 2006 – 12 – 07 13:55

Option 1 is exactly what one of our M&E customers just requested

Hey Dave, This blog couldn't be more timely. We have a potential network we are building for an M&E customer and they said that's how they price their business by GB of transfer. Option 1 would be a great solution. How hard would it be to do this for a customer in the next 2 months?

Greg Leigh (not verified) – Fri, 2006 – 11 – 03 11:37

Re: Option 1 is exactly waht one of our M&E customers...

I did some checking a couple of weeks ago and it doesn't seem that hard to do, but you'll never really know until you try and do it.

My suggestion for a fast turn around is to use the ICB, or Individual Customer Basis, process. They will engage all the proper teams to provide development costs and timeframe. As one of our elite Sales Engineers, I'm sure you have worked with this process before, but if there's anything I can help with, please contact me in email.

dsiegel – Fri, 2006 – 11 – 03 12:23

blurry dots at 5:00 am

:-) I think I've been thinking about Net Neutrality too much, and carriers talking about "walled gardens". But with Option 2... You could "grade" the internet quality with QoS, and sell it appropriately, based on what the customer application was, or what their business model was. Sort of like "wholesale" internet service should bea little "less good" than enterprise/retail grade internet. Or the customer could choose their "grade" themselves, via an XML portal, and be billed appropriately! Arggh. The more I read this, the insaner it sounds. I'm trying to believe that a walled garden is a good idea, and that net neutrality threatens the future of the planet, but golly. It would just be easier to charge the appropriate amount of money for quality access. At BOTH ends, content and consumer. I need a nap now.

jules (not verified) – Thu, 2006 – 11 – 02 20:45

Re: burry dots at 5:00 am

QoS on the Internet, especially across service provider boundaries, is definitely in the realm of net neutrality.

In this case, however, the QoS changes I am referring to apply mainly to our IP-VPN product, which is our only product at the moment that requires the customer to buy bandwidth by class. All Internet traffic is automatically placed into the lowest class on the core, but in general does not support QoS at this time.

Because it is pretty hard to do QoS across provider boundaries, if we ever were to implement QoS on our Internet product, it would most likely be to enable higher classes of service to the applications we offer, like VoIP and Video Conferencing. We wouldn't give that higher class of service to customers, they would have to pay for it, but it would give them a lot more control over the priority of applications on their local loop, which is where they need it the most.

QoS on our core doesn't kick in except in the rarest of circumtances, say a bad fiber cut. We engineer it to deliver a high quality service, even on the lowest class.

dsiegel – Thu, 2006 – 11 – 02 21:51

Re: automatic provisioning

The monitoring and feedback mechanism would be relatively easy. We've got that now. Getting that to something that could automatically provision things would be the challenging part. There was a start on this in sniper, but it could use more fleshing out. I'd really like to see the entire customer provisioning of all network elements driven from a database and templating language. It's a long road to get there, but you've got to start someplace. It's less work in the long run, less prone to error, and more standardized. It'd be really cool. :)

doug – Thu, 2006 – 11 – 02 12:52

Re: automatic provisioning

We did an internal proof of concept test about two years ago with a 3rd party vendor to do the router provisioning for IP-VPN, and it worked well enough. We ended up not deploying the 3rd party solution because the cost/benefit analysis didn't work out when we justified it against operational savings (it really doesn't take a lot of time for a person to change a router config).

The best part of the POC test was the validation of the automated workflow management. Our systems have come quite a long way in this area the past few years, and is a great foundation for this capability.

As for the monitoring situation, our Application Performance Monitoring product which was recently released could definitely play a role in this, but the feedback mechanism doesn't exist yet. Like you say, it might be relatively easy, especially given the Ucommand functionality that currently exists in the area of Web Services and XML interfaces.

dsiegel – Thu, 2006 – 11 – 02 16:41

Option 2 Fits Nicely with Net Neutrality ideas

I like the idea of Option 2 - the self provisioning internet circuit, and it quite fits nicely with the challenges that may be posed by net neutrality debates. But I'm not sure it answers any more questions or problems than Option 1. TELUS lets customers pick between 95P and per Gig usage, usually allowing the customer to change every month, if they so desire (making the billing match up is a bit of a nightmare, but that's another story)....

The appeal of Option 2 is that you don't have to make hard decisions (or guesses) ever month on what your usage is like, and it also gives both the carrier and the customer the knowledge of exact available bandwidth in the network. I think you could even build a buffer into Option 2 - like a floating ceiling, and offer reduced levels of Q0S, depending on how sensitive the customer is to monthly pricing, perhaps they would get 100 Mbps of *clean* bandwidth, and pay accordingly, and if they happened to exceed that ceiling, they could choose to get an additional 50 Mbps of "average, tier 2" bandwidth" at a reduced price.... it wouldn't be as great as the Tier 1, Premium bandwidth, but it could cover their butts, and help them manage costs in the event they have a ridiculous traffic spike.

-jules-dreaming-at-5:00am

jules (not verified) – Thu, 2006 – 11 – 02 06:53

Re: dreaming at 5am

That's interesting to hear that TELUS offers a pay-by-the-byte option. I hadn't heard of any other providers that offer it, but I see that TELUS also offers web hosting, and per-megabyte pricing is more common in that business (it's much harder to calculate bit-rates in a virtual hosting environment), so perhaps that was one of the drivers for them to offer that.

I'm glad you agree that Option 2 is appealing, but I am having a hard time connecting the dots between this capability and the net neutrality debate. It's 11:30am for me, and I guess I still haven't had enough coffee. :-) Care to expand on that?

dsiegel – Thu, 2006 – 11 – 02 14:43

IP Convergence: Beyond VoIP, Beyond Cost Savings

Regional Blogs

Podcasts

Subscribe

Topics

Industry Bloggers

Recent blog posts

User login

Navigation

Terms

Technorati