IP Convergence: Beyond VoIP, Beyond Cost Savings

The release of the proposals by the European Commission for reform of the European regulatory framework for telecom firms is surely just the beginning of what is to be a major political battle within Europe. The proposals garnering the greatest attention at the moment are the ones aimed at the forming a European-wide regulatory authority and allowing national regulators to “functionally separate” the incumbent carriers if other measures fail to introduce competition.

On the subject of the European-wide regulatory authority, I must say I think it is a bad idea and I think the U.S. experience should serve as a vivid example of the problems with such an approach. There was a time, back in the late 1980s and early 1990s when individual states in the United States began experimenting with liberalization of the telecommunications market. New York and Illinois were the early pioneers in this effort. Other states, including Maryland, Massachusetts, Florida, Michigan, California, Washington, and Connecticut, recognized that market liberalization was good for investment and local business and moved quickly to initiate rulemakings or modify their telecommunications laws to open the local telephony market to competition. By the time the Telecommunications Act of 1996 was passed by Congress, more than a dozen states had ended the local telephone monopoly and implemented measures to promote local exchange competition. At least a dozen more states (including Tennessee, Georgia, Pennsylvania, North Carolina, Texas, and Colorado) were in the process of initiating changes to their laws and regulations as well so that basically half of the states were in the process of liberalization.

But once the Telecommunications Act of 1996 passed and the experiment in local competition was nationalized, what started as an orderly progression towards effective market liberalization quickly disintegrated into an orgy of regulatory gamesmanship and litigation. That the Telecommunications Act of 1996 was a mistake was made clear when the FCC issued its first “local competition order” – a nearly 800 page order that attempted establish comprehensive local competition rules, most of which were eventually overturned after years of litigation.

How the FCC could get it so wrong when nearly half the states were getting it right is a lesson on the dangers of centralization that the Europeans would do well to heed. But putting aside the substantive policy problems that a central authority would have, the situation in Europe is a bad idea for the simple reason that it is going to have the effect of freezing progress, not accelerating progress towards liberalization, as parties litigate the contours of the authority of this new pan-European regulator. National regulatory authorities hostile to the idea of a pan-European regulator will simply shut down, forcing interested parties to pursue their cause at the pan-European regulatory authority, ultimately triggering litigation by the losing party. This process could take years to sort out during which time critical issues will remain unresolved. Moreover, this process involves substantial legal and political cost and resources for the parties involved, which lends a decisive advantage to incumbent operators. Most nascent competitors lack the legal and political resources necessary to effectively pursue a legal claim over the course of several years from the national regulator, up to the pan-European authority, and eventually back to the national regulator where the matter will be resolved on some form of remand. So not only will progress be frozen pending the outcome of litigation, but the outcome is more likely to be favorable to incumbent interests since they are the only parties with the resources to see the process through to its conclusion.

My feelings about the proposal to allow national regulatory authorities to impose functional separation on incumbent operators are much more favorable from a substantive perspective, but I still have questions about the procedure. The proposal says national regulatory authorities can impose functional separation as a measure of “last resort” and only after receiving approval from the European Commission. I am not sure European Commission approval is a necessary component to this and it would probably be better if the European Commission simply clarifies the authority of national regulators in this regard.

For the moment, these are just proposals for reform. It remains to be seen whether the European Commission will adopt them, but it demonstrates once again that the Europeans are setting the example for the world on how to effectively liberalize telecommunications markets. Here in the U.S., policy makers seem unable to tackle the primary obstacles to more effective competition and are devoid of any creative ideas as to how to overcome them.

News the other week that Telecom New Zealand is being split into three divisions – wholesale, retail, and network – is further evidence that the folks down under are far more aggressive in their efforts to introduce competition.  The other day, I applauded Australian Communications Minister, Helen Coonan, for her public rebuke of Telstra’s bill insert blaming regulation for its market failures.  Together with New Zealand’s action, these are the strongest examples of courageous policy makers taking the necessary steps to introduce long term, sustainable competition into the telecommunications industry despite the protests of the incumbents. 

Now we also see the European Regulatory Group issuing a report saying functional separation may be appropriate in some markets.  Also, it appears that Eircom Ltd., the incumbent telephone company in Ireland is voluntarily proposing a wholesale/retail split of its operations.  

In the U.S., divestiture was instituted by the hand of a judge.  Will history repeat itself or will U.S. incumbents learn from the past and their overseas brethren?  

A key element of virtualization is the concept of abstraction. Abstraction can take many forms and  many applications with profound  benefits.

This is the first of a  five  part series on the evolution of virtualization, with the following planned articles:

Virtualization – Part 2 - The Abstraction of the Computer
The definition of a computer (CPU, data bus, memory, input/output, and disk)  , the  abstraction of programming a computer (machine code, assembly language,  3rd generation programming languages (3GL),  4GL and  5GL), the  separation of a CPU from disk and the   application  of a Storage Area Network (SAN),  blade servers and the realization of GRID computing.

Virtualization – Part 3  - The Abstraction of Applications
Concepts of a Application Programmers Interface (API),  examples and pitfalls for APIs and the abstraction of  Web Services.

Virtualization – Part 4 – Virtualization of Voice Communications
Telephony basics in the  circuit switched voice network and the evolution of the  packet switched voice network , aka VoIP.

Virtualization – Part 5  -  Real World IT  Examples and Benefits
The anatomy and benefits  of Virtual Data Centers and  Call Centers

We’ll start the series on a model that is near and dear to our hearts – the Internet.

Vint Cerf  is one of the founding fathers of the internet and created a transport model of moving packets from one network to another. Seems pretty straight forward but,  back in the day this was not an easy task as the networks were typically homogenous deployed by a single hardware vendor (DEC, AT&T, NCR, Apollo, Banyan, etc… ). Each vendor had their own proprietary methods of defining a ‘packet’ (that is,   the number of bits in a ‘packet’; the order of the bits in a ‘packet’; the number of bytes in a ‘packet’; and the meaning of each bit/byte in a ‘packet’), and the way these computers spoke to each other (i.e. the protocol).

The first  major challenge was to allow these desperate networks to exchange data. The challenge was met by defining common communication protocols (i.e. TCP/IP, UCP, etc …)  , and   a  common packet &  addressing structure.

This allowed high powered users (Scientists at the world's research institutions) to efficiently communicate across an environment largely made up of  heterogeneous computers.  Within this structure, application developers at each vendor (DEC, AT&T, NCR, Apollo, Banyan, etc…)  developed tools, using the agreed upon common protocols  ,  to transfer files (FTP), login into another computer (TELNET) and exchange email (POP3, SMTP).


Tim Bernes-Lee was one of those high powered users and wanted to find an easier way of navigating thru the internet without having to issue ftp commands or walk thru folders ( i.e. directories).  He created an abstraction between what the internet looked like and the commands that facilitated communications .  Tim created the notion of the world wide web.  The world’s first web site  was brought online in 1991. 

Tim is credited for creating the following words we all use everyday:  www, http and HyperText Transfer Protocol.

These abstractions have lead to an  unprecedented number of users to the internet in very little time.   A concept that some call, “Internet Time” . 

“Internet Time” is defined by Wikipedia   as:

“Internet time was a common catchphrase that originated during the late-1990s Internet boom. In this period, people who worked with the Internet had come to believe that "everything moved faster on the 'net", because the Internet made the dissemination of information far easier and cheaper. Fast-moving developments were therefore said to run "on Internet time".

Efforts in virtualization of the internet are focused in/around layer 3 of the OSI stack,  in the next article in this series we will take a dive into the machines themselves to understand abstraction as it relates to their operation and subsequent evolution, in Part 2 - The Abstraction of the Computer.

We’ll then take a quick look at the applications that ride over them in Part 3 – The Abstraction of Applications.

After understanding the abstraction of the Application,  we’ll look at a very basis of communications – Voice , in Part 4 -  Virtualization of Voice Communications.

In the final article we will put all of these pieces together in understanding the anatomy and benefits of Virtual Data Centers and  Call Centers , in  Part 5 - Real World IT  Examples and Benefits.

Global Crossing's Pan-European duct and fibre network was constructed during 1999 and 2000. It may be hard to believe, but at that time, the Google search engine just came out of the dorm room for beta testing, and digital camera's were using floppy disks (Whatever these were: Ask your grandparents) for storage.

Even though our fibre and wavelength-based IP network will continue to be state-of-the art for many years -- avoiding the horror scenarios related to IP/VOIP conversion and convergence problems that legacy networks have -- this does not mean that there are no improvements to be made to stay aligned with new technology.

One of our Operations Europe "hobby" projects is to display our network using web-based tools such as Google Earth. Both our Carrier and Enterprise customers demand precise geographic information to avoid single-point-of-failures. They further need to have detailed map information for their support staff and suppliers to reach GC network locations for repairs and transport systems upgrades.

Mapping 25,000 km of underground duct and cable systems is not an easy thing to do: The diversity of the European continent (our home-built fibre network runs through twelve countries) has resulted in multiple non-aligned country or even regional grid coordinate systems. Some used a key city as reference point, others used a meridian, or any other arbitrarily selected location. As long as there were no border-crossing network activities, this perspective was not an issue, and it would not have been an issue at all, if GPS and Google hadn't taken over the world.

Merging all vector-based (AutoCAD) network design data into a single view thus required the mapping of dozens of disparate regional grids into one, earth-centered, GPS coordinate system: A cumbersome process requiring many (non-) linear translations [Kudos to our wiz-kid Nivaldo!].
At that point, the network data is ready for use in systems like Google Earth or Yahoo Maps.

The result is remarkable: This image was created as overlay of our duct route near one of our Northern France amplifier sites on Google Earth.
It was shot after zooming in to an approximate altitude of 600 m (2000 ft). Next to the blue line displaying our duct route as converted from AutoCAD, the original trench dig is still visible from on this satellite image, even though the route was completed seven years ago. The image further shows that, after conversion, an accuracy of a few meters is quite feasible. The amplifier station is the white building on the top-right hand side, next to an old chimney (station access lines are not shown here). 


Our fibre customers are pleased: Not only can they trace the exact location of their network, but their families can also see where daddy or mommy will be for their provisioning or system commissioning activities, and they can check out nearby tourist sites using tools like Panoramio (acquired by Google last month).

Bringing historic data in line with today's technology effectively means: Staying behind. Our task is to stay ahead, so we're now working to link the web-enabled network data into an image and connection database repository, allowing both ourselves and our customers a detailed virtual and real-life perspective of routes, sites and buildings, space, footprints, equipment, and most important for a carrier: All fiber allocations and (meet-me) interconnections at distribution frames.
I expect to get a first version ready for beta testing out of the dorm room in the next few months. Once complete, this solution will hopefully last for at least another few years (or until the next Google Earth software upgrade...).

So what's the next advancement, you may ask? My best bet: Operate our virtual network on Second Life.

Gert Nieveld July 4, 2007

Besides for its beautiful palaces, parks, and the Documenta international contemporary art exhibition (starting June 16!), the German town of Kassel is renowned for the Brothers Grimm who lived and worked there in the early 19th century.

Both brothers were professors of language. To preserve historic data, they took the task upon themselves to write down traditional German folk stories. I’m sure you’re familiar with tales such as Snow White, Hansel and Gretel, the Sleeping Beauty, and Cinderella.

A 600 km long Fairy Tale Road runs from the town of Bremen (the Hanseatic city in northwestern Germany) to Hanau (near Frankfurt), passing many sites and gingerbread towns that have contributed stories to the brothers Grimm archives. It is a popular route with both young and old children.

Based on tales in the public domain, these folk stories often also include the less positive side of human nature: Only the Disnificated versions have happy endings.

One of the darker stories is that of the Pied Piper of Hamelin (German: Hameln): A tale about a piper that uses his magic flute to rid the town of Hamelin from the burden of a rat plague. The town magistrates, however, refuse to recognise his effort and deny him his fee. In response, the piper uses his magic to lead all the children away from the town, never to return.

Although related to an actual event from the 13th century, the moral lessons of the Pied Piper story are still applicable in today’s world. The same appears to be true for the rat plague part.

Some weeks ago, our operations organization received fault reports about fiber outages in Germany. Several rats apparently managed to dig or chew their way through the concrete and metal isolated walls of an optical amplifier station.
Once on the inside, they developed an appetite for dark fiber connections, with a special preference for Italian fibers. It’s unclear if these fibers offered the rat family a higher nutritional value than other fibers.

It will not come as surprise to you, that the site of this event happens to be along the German Fairy Tale route, only a short distance away from the town of Hamelin. Luckily, our field support organization has several modern pipers standby on a 24x7 basis, so the damaged fibers have been replaced and repeat will be avoided by installing new overhead duct trays for these tasty fibers.

The only thing that still needs to be done is to get rid of the rats. We will probably outsource their animal-friendly evacuation to a specialised company.
As long as we don't forget to pay the bill we won’t have to worry about our kids.

Gert Nieveld, 07/05/22

In my previous blog on this subject I indicated that HDTV primarily revolves around flat screens and motion picture studio efficiency. In fact, flat screen TV systems are not purchased for their claimed HDTV capabilities but for esthetic reasons: Flat is Cool. Few of the flat screens are compatible with the true HDTV (1920 x 1080) format. The current quality improvements are based on the use of “DVD-quality” signals in digital format via the air, via satellite, or via cable networks. Given the low replacement ratio (in the range of ten years) for the 1.5 billion TV’s on earth, not much will likely happen with “HDTV” for the consumers in the coming years.

Now compare the quantity of TV’s with the 800 million PC’s and 1.5 billion mobile phones that are out there. We’re used to replace these items every three years, so this is the area where fast changes can happen. The 2006 UK regulator Ofcom report shows, that young adults now watch less TV than their parents. Computer games, chatting, and other internet activities are successfully competing with traditional television, and TV programs are watched selectively rather than passively. The end of the couch potato era is in sight! Taking the TV to the PC and the mobile phone is a logical step, and potentially a killer application. The format to do this is IPTV.

The Focus Group of the International Telecommunication Union (October 2006, Korea) defines IPTV as  “Multimedia services such as television/video /audio /text / graphics delivered over IP-based networks managed to provide the required level of QoS/QoExperience, security, interactivity and reliability”

Each line of business positions to get a piece of the potentially very rich IPTV cake: Equipment manufacturers advertise the IPTV capabilities of their routers; Local telecom providers focus on the quality aspects to compete with the cable companies and fill the gap of the ever reducing voice revenue; Cable companies see in IPTV a new way to move towards two-way services and offer video on-demand (VoD) via set-top boxes; Coffee companies advertise that their beans enrich IPTV viewing, and so on. They all plan to incorporate IPTV in their service offering as a new ‘triple”, or “quadruple” play subscription package. They all hurry to state that IPTV is not to be mistaken for free open Web TV.

Of course, Web TV is exactly what we’re heading for on our PC’s and mobile phones. The plans to offer IPTV as subscription service with set-top boxes are, at best, interim solutions that will fail to generate profitable revenue. Today’s PC users (connected either via DSL, Cable, WiFi/WiMax) are rapidly getting used to high-speed internet connections in range of 3 Mbps to 30 Mbps. These internet users are the early adopters of Web TV. The other internet users will join them as soon as their internet access connection allows for the speed, which is a matter of time. Compression technology improvements have resulted in the ability to view TV at data speeds of 500kbps – 750kbps.  Modern internet connections support watching different TV channels on multiple PC’s.

But what about the Quality of Service, you may ask?
Since 2000, various proprietary and open source codecs have been created that allow an acceptable display of TV programs, both for linear TV as well as for canned motion pictures.
Besides the H.264 standards that were agreed upon by the ITU members in the past decade, multiple open-source MPEG-4 codecs evolved to fulfill the market need. Codecs such a DivX, 3viX, and XviD were created, with additional services, such as subtitles, included in new container structures like OGM and Matroska. Simplified versions have been made for the mobile phone, such as 3GP. Check out the Doom9 site for guides and version details. No need to worry about all these new codecs: The PC or SmartPhone media player will detect the required codec and automatically fetch it from the internet in a matter of seconds (except when you’re tied to a proprietary set-top box, of course).

Today, computer and mobile phone users are enjoying mp3 audio on their phones and iPods. This highly compressed  music format has effectively taken over from the high quality audio CD. The same is happening with video. Consumers have embraced free “low-Q” video sites such as YouTube and Google. In the past year, free linear IPTV / Web TV services have seen explosive growth. Dozens of generic TV-portals are now offering free access to thousands of TV channels. Examples are Channelchooser and wwiTV. This number of portals will likely grow two orders of magnitude in the coming year.

Also for the mobile phone side there are TV standards, such as T-DMB. The free air to TV model, introduced in South-Korea late 2005, allows buyers of a T-DMB (Terrestrial Digital Multimedia Broadband) handset to watch TV without requiring a subscription. This is the way to boost the solution! In Germany, the mobile provider Debitel initially charged a fee for the TV service, which resulted in minimal growth. To boost sales, they are now offering a 6-month free introduction. T-DMB is not restricted to mobile phones. Also PDA, portable TV devices, and car navigation systems can make use of this service.

We’re slowly moving away from the Napster days. Consumers are getting used to pay for mp3 downloads via iTunes and its competitors. The same is bound to happen with TV (and games). Viewers will become used to pay a fee for on-demand motion pictures.  VoD is the reason that all providers are rushing towards IPTV. New anti-piracy tools, such as the video watermark system introduced by Philips this week, will help to protect (at least for a while) VoD material and revenue.

What will be the effect of IPTV on the ISP’s, Cable companies and other transport providers?
A recent Gartner report predicts that almost 50 million households will subscribe to IPTV service by 2010.
My perspective: Why subscribe to IPTV if you can get it for free via your regular internet connection?
In Europe, France leads the way: Companies such as FREE are including linear (terrestrial) TV channels in their package at no extra charge, focusing on new revenue via VoD services. According to a recent WorldBusiness article, PCCW in Hong Kong has highest local IPTV market penetration number,  but a fast ROI seems unlikely, especially if consumers can also watch TV via free web portals.

Similar to the music industry, where music producers and artists are bypassing the record companies by posting their music directly on the internet for a moderate fee, we will see traditional TV transport providers being bypassed by the content producers.
Most linear TV stations already serve their audiences with program replay and extended broadcast services over the internet. Best-in-class example is probably the BBC, offering a wealth of on-line information around their programs in over thirty languages, not only for their TV audience but also for mobile phone users.
Motion picture distribution will be done directly by the studios in cooperation with dedicated storage and on-line servers hosted by internet content providers such as Google, and/or via P2P download options.

In the coming years, I expect to see many local "legacy-based" TV distribution / metropolitan cable companies losing their business to the content and internet providers. A shake-out in this area is unavoidable.
For global IP-based network owners and service providers such as Global Crossing, I see only positive effects: Global IP traffic and capacity will increase even faster that it is doing today.

So what's next in the HDTV/IPTV theater? More about that in my third (and last) blog on this subject.
Gert Nieveld

This presentation from the blackhat conference in Europe last year speaks directly to the point about the security issues between IP-VPN and Ethernet that I took issue with in my last blog. 

A couple of the key points that I took from this presentation were:

• In the case of both Ethernet VPNs and IP-VPNs, in order to hack into a customers network from outside the network, the attacker must have access to the provider's core routers. (pg. 26)
  
• If an attacker has penetrated the customer's network through a backdoor or through weak physical security, he has some interesting options with an Ethernet VPN that do not exist on an IP-VPN network, especially in a VPLS environment. (pg. 34 and others)
  
• A reminder about how much I dislike spanning tree (pg. 38)

ps. he refers to as the Juniper M7i as a "big-iron" router, when in fact it is the smallest of Juniper's carrier-line of routers and doubles as a high-end Enterprise router.  An M7i may be used by a carrier in a very small site, but such a site is not likely to have thousands of customers as cited in his example.  A site with thousands of customers is much more likely to be riding an M40e at minimum, or more likely an M320.

Hat Tip to Jim Lippard

Technorati Tags: Ethernet, VPLS, IP-VPN, Security

Just before Christmas, I was honored to present an overview of Global Crossing’s experience in the planning, deployment, and operations of submarine cable systems to the Development Fund Commission of the European Union in Brussels.

In its efforts to reduce poverty in Africa, the European Council defined a strategy to speed up and provide continued support for the African infrastructure. The EU-Africa Partnership was established in 2006. This Infrastructure Partnership encompasses many elements that are considered standard by most of us: Some 42% of the 900 million Africans do not have access to safe water; 20% of Africans have no access to electricity; Less than 6% have access to internet.

In a five year period from 2008 onwards, the European Development Fund will invest €5.6B (US$7.3B) in the African infrastructure. A significant percentage of this amount will be for the expansion of African communications networks and international connectivity. This will greatly assist in closing the Digital Divide, as mentioned by former United Nations Secretary-General Kofi Annan in his opening speech of the World Summit of the Information Society (WSIS) in Tunis in November 2005.

Despite the lack of access to electricty, Africa is the world’s fastest growing market for mobile telephone services. In remote areas, smart entrepreneurs have even started battery-loading services for people with mobile phones that do not have electricity at home. Mobile phones have taken over from wireline services, for which an infrastructure is basically absent outside the main cities. The mobile penetration rate, which I estimate to be around 16% at present, is still massively behind Western-European countries, where penetration in some cases exceeds 100%. Fixed phone penetration is less than 5%. You can find details on the African market at the African Regional Office web site of the ITU.

The growth of the mobile market offers great potential to boost business, but Africa still lacks the international bandwidth to support this growth. To stimulate new business as well as to handle international traffic, key improvements need to be made in the interconnection between African countries and between Africa and the rest of the world. The two main options are: Expansion of satellite links and/or submarine fibre cables. Due to high latency problems with satellite connections (up to 600 ms), submarine connections are the preferred choice for voice and internet services, while satellite connections are selected for broadcast-type services.

Some of you may remember the Africa ONE project, which involved the installation of 39,000 km of submarine fibre cable around Africa. The project implementation, for which Global Crossing was selected for project management and undersea construction, was planned to start in 1999 and be completed in 2002. The project eventually was abandoned late 2001 when international funding evaporated.  At the time, a main reason for Global Crossing not to participate as either investor or owner in this project was based on the absence of deregulation in African countries as well as on difficulties reaching the many landlocked nations. Experience from GC’s other submarine projects in Europe, US, South-America, and Asia is that the full growth potential only emerges once deregulation comes into effect.

Mid-2002, the SAT-3/West African Submarine Cable (WASC) with a capacity of max 120 Gbps was taken into service between South Africa and Portugal. The SAT-3 cable is a club cable owned by a consortium of over two dozen incumbents. Besides in South-Africa, the SAT-3 cable lands in eight other West-African countries. Unfortunately, like many other consortium cables, the SAT-3 cable system suffers from cooperation issues between the club cable owners. At present, few countries have benefited from its existence.

In 2003, an East-Africa Submarine Cable System project (EASSy) was initiated for linking eight East-African countries to the rest of the world. The completion of this new club cable system is targeted for 2Q07: Less than six months to go. According to the EASSy website, some 23 carriers have signed the project MOU. According to other sources, however, the protocol approving the construction of the EASSy cable has only been signed by twelve carriers up to now. Completion of the project before 2009 is unlikely. We can further expect that this club cable will face similar problems as exist with the SAT-3 system today.
EASSy will not provide a direct connection of these eight East African countries to the rest of the world, but will only connect them to the SAFE and SeaMeWe-4 consortium systems. A significant reduction of capacity costs for the consumer is therefore not to be expected. A total investment of around US$300M has been indicated at present for EASSy. In July 2006, the European Investment Bank approved its participation in this investment.

When trying to gather data from the EASSy website in preparation for my presentation to the EC, I immediately perceived the problems that burden so many of these projects: Both content and data access are poor. Many web pages and images do not exist or cannot be loaded due to hosting problems. This phenomenon already shows the desperate need for many African countries to get better internet connectivity. For a future investor, however, it will be difficult to understand why a US$300M project does not offer a professional looking website. Most high school students on this side of the digital divide will be able to create such sites in a few days and have them hosted by an ISP for under US$50 per annum.

My advice to the Development Fund Commission of the EU, and to any other investor in projects like EASSy, is to add the learnings from SAT-3 high on their 2007 Resolutions List: Especially the aspects related to open access to these scarce resources deserve detailed study.  The unique situation that exists in Africa (e.g. the virtual absence of a fixed in-country telecom infrastructure) requires maintaining a careful balancing act between telecom regulation, deregulation and incumbent privatization. A direct connection of EASSy either to countries with a deregulated telecom enviroment and/or to privately owned submarine cable systems is a must. The funding should further be governed by tight control structures to ensure that aid actually ends up where it is needed most: With the African enterprises and consumers. From my perspective, this is the only way that Africa will be able to cross part of the digital divide in the next five years.

Any other suggestions for bridging the digital divide? Don't hesitate to post your comments below.