Over the last few months, events like the broadcast of a baby’s birth and the publication on the web of Special Prosecutor Kenneth Starr’s report have drawn attention to the way the internet handles (or doesn’t handle) sudden and unexpected demand for specific content. This in turn has raised the profile of the nascent web caching industry, whose players – Inktomi, Network Appliance, Cacheflow, Mirror Image and SkyCache, to name a few – store frequently accessed content locally to spare backbone bandwidth and improve user response times. Eolian Inc is a relatively young entrant to this race. In some ways, chief technology officer Dennis Leggett says, that has already worked to the company’s advantage. When deciding where to go with all this, it was clear that core caching was rapidly becoming a commodity, he says. With so many providers in the market, Eolian chose to focus on adding services to basic caching. So instead of using proprietary kernel, we based ours on Linux and Squid, Leggett explains. Using off the shelf software let Eolian spend more development time tweaking for performance and adding unusual features. One such feature is cache pre-loading and refreshment. Another is a distributed environment, designed to overcome the technical limitations which often prevent small companies from investing in a cache. Web caching analyst Peter Christy of Collaborative Research explains: The fundamentals of caches are simple to describe. The more coherent traffic you have, the better. You get most value out of a cache when a user and wants the same thing someone’s already downloaded, while it’s still fresh. That gives giant ISPs like AOL a huge advantage. Caches fundamentally get better as the load increases, Christy says. To give small ISPs and corporations the benefits of a large scale cache, Eolian has built a subscription service called EagleWatch. EagleWatch aggregates statistics from its subscribers, who are running Eolian’s InfoStorm caches. Working together, Eolian’s customer community works like one giant ISP, pooling individual caches for the benefit of all. It balances out the scales, Leggett says. The geographic reach of the network also means EagleWatch can pre-load popular content from the East Coast to its westernmost servers, in Guam, while users sleep. Finally, so as to be a good network citizen, Eolian distributes the work of checking for fresh content. It assigns Yahoo to one server, Netscape to another and so on. That means only one InfoStorm cache checks each popular site, sharing its results with the rest. Less traffic on the backbone means less congestion. Collaborative’s Christy believes that Eolian is just one of a number of new cache companies – the others are Mirror Image, SkyCache, SandPiper and Ibeam – whose offerings are: intriguing propositions from an intellectual point of view, but unproven until proven. The sense I got from talking to the Inktomi guys is that their general ideas are valid, he says. Some of these schemes are going to work out. The trouble is, no one yet knows which ones they are. One thing is certain. The architecture of today’s internet, with hugely popular sites like Yahoo! mirrored in only a handful of places, will not scale well however much bandwidth is thrown at it. We have a major plumbing problem, says longtime industry pundit Robert X. Cringely, writing for PBS. Bigger pipes alone is not the answer because traffic growth will always fill those bigger pipes. Collaborative’s Christy agrees: People are going to have to have a replica out where it’s needed. The alternative is to watch the internet grind to a halt.
Content from our partners
