Could Data Overload 'End' the Internet?
For months there has been a rising chorus of alarm about the surging growth in the amount of data flying across the Internet. The threat, according to some industry groups, analysts and researchers, stems mainly from the increasing visual richness of online communications and entertainment — video clips and movies, social networks and multiplayer games.
Moving images, far more than words or sounds, are hefty rivers of digital bits as they traverse the Internet’s pipes and gateways, requiring, in industry parlance, more bandwidth. Last year, by one estimate, the video site YouTube, owned by Google , consumed as much bandwidth as the entire Internet did in 2000.
In a widely cited report published last November, a research firm projected that user demand for the Internet could outpace network capacity by 2011. The title of a debate scheduled next month at a technology conference in Boston sums up the angst: “The End of the Internet?”
But the Internet traffic surge represents more a looming challenge than an impending catastrophe. Even those most concerned are not predicting a lights-out Internet crash. An individual user, they say, would experience Internet clogging in the form of sluggish download speeds and frustration with data-heavy services that become much less useful or enjoyable.
“The Internet doesn’t collapse, but there would be a growing class of stuff you just can’t do online,” said Johna Till Johnson, president of Nemertes Research, which predicted the bandwidth squeeze by 2011, anticipating that demand will grow by 100 percent or more a year.
Others are less worried — at least in the short term. Andrew M. Odlyzko, a professor at the University of Minnesota, estimates that digital traffic on the global network is growing about 50 percent a year, in line with a recent analysis by Cisco Systems , the big network equipment maker.
That sounds like a daunting rate of growth. Yet the technology for handling Internet traffic is advancing at an impressive pace as well. The router computers for relaying data get faster, fiber optic transmission gets better and software for juggling data packets gets smarter.
“The 50 percent growth is high. It’s huge, but it basically corresponds to the improvements that technology is giving us,” said Professor Odlyzko, a former AT&T Labs researcher. Demand is not likely to overwhelm the Internet, he said.
The question of the problem’s severity is more than a technical one, since it will affect the shape and cost of the nation’s policy on broadband infrastructure, a matter that is expected to attract political attention after a new administration takes over in Washington.
While experts debate the immediacy of the challenge, they agree that it points to a larger issue. In the Internet era, they say, high-speed networks are increasingly the economic and scientific petri dishes of innovation, spawning new businesses, markets and jobs. If American investment lags behind, they warn, the nation risks losing competitiveness to countries that are making the move to higher-speed Internet access a priority.
“The long-term issue is where innovation happens,” Professor Odlyzko said. “Where will the next Google, YouTube, eBay or Amazon come from?”
The Internet, though a global network, is in many ways surprisingly local. It is a vast amalgam of smaller networks, all linked together. The worries about digital traffic congestion are not really about the Internet’s main trunk lines, the equivalent of network superhighways. Instead, the problem is close to home — the capacity of neighborhood switches, routers and pipes into a house. The cost of stringing high-speed optical fiber to a home, analysts estimate, can be $1,000 or more.
That is why Internet access speeds vary so much country by country. They depend on local patterns of corporate investment and government subsidy. Frederick J. Baker, a research fellow at Cisco, was attending a professional conference last month in Taiwan where Internet access is more than twice as fast and costs far less than his premium “high speed” service in California.
“When I mention my own service, people here shake their heads in disbelief,” said Mr. Baker, who is a board member of the Internet Society, a nonprofit organization that helps guide Internet standards and policy.
In the United States, the investment required to cope with rising Internet traffic will need to be made at several levels, not just cable and telecommunications carriers. Tim Pozar, an engineer and a co-owner of the Internet services company UnitedLayer in San Francisco, said a number of forces were combining: the surge in bandwidth-hungry video applications on Web sites, the need to handle traffic from more Internet-enabled devices like cellphones, and shortages of electrical power for data centers in places like San Francisco.
'Running Out of Horsepower'
“We’re running out of horsepower to accommodate the demand,” said Mr. Pozar, whose company’s data centers support Web sites for customers ranging from museums to social networks. “And upgrades needed in data centers are going to be a lot more expensive than in the past, now that all the excess capacity left over after the dot-com bubble burst has been gobbled up.” The pace of future demand is the big uncertainty surrounding the Internet traffic challenge, and how fast people will adopt emerging technologies is notoriously difficult to foresee.
In the aftermath of the bursting of the technology bubble in 2000, there was a glut of capacity — so-called dark fiber, strung around the world and then left dormant. Now demand is catching up with that supply. In its prediction of more than 100 percent annual growth, Nemertes, a telecommunications research firm, assumes brisk use of new innovations like high-end videoconferencing, known as telepresence, which corporations are beginning to embrace as an alternative to costly, time-consuming travel.
If this technology becomes a consumer product in the next few years, as some analysts predict, Internet traffic could spike even more sharply.
Slick video chats are something that William Bentley, a 13-year-old New Yorker, would like to see. He is fairly representative of the next generation of digital consumer: He has made and posted his own YouTube videos, subscribes to YouTube channels, enjoys multiplayer games like World of Warcraft and Unreal Tournament, and downloads music and videos.
Asked what he would want next from the Internet, he replied, “It would be nice to have everybody always right there — just click and you could see them clearly and talk to them.”
That sort of service is certainly going to require more bandwidth and more investment, with higher costs across the spectrum of the Internet ecosystem that includes cable and telecommunications carriers, Internet companies, media Web sites and even consumers. AT&T, for one, said last week that it would spend $1 billion this year — double its 2006 expenditures — to expand its overseas infrastructure.
But even if investment lags behind, there will be no Internet blackout. Indeed, the Internet has survived predictions of collapse in the past, most notably by Robert M. Metcalfe, a networking pioneer and entrepreneur, who in a 1995 magazine column warned of a “catastrophic collapse” of the Internet in 1996. There were service problems, but nothing like Mr. Metcalfe predicted, and on stage at a conference in 1997 he ate his words.
“The Internet has proven to be wonderfully resilient,” said Mr. Metcalfe, who is now a venture capitalist. “But the Internet is vulnerable today. It’s not that it will collapse, but that opportunities will be lost.”