Google can hardly be called lazy when it comes to introducing new services or adding features to existing services, but the core of it’s search engine itself hasn’t been through major upgrades. A while back Google previewed a whole new engine for its search, which it dubbed Caffeine, and subsequently shut it down, declaring it a success and deeming it ready for the mass audience.
In a recent blog post they have declared that their index is now complete and detailed how the new system works. In the old indexing system, the index was composed of multiple layers, with each being refreshed at a different rates, and the main layer being updated every couple of weeks. With each refresh, the entire web would be analyzed, adding to the delay between a page being indexed, and a page being found.
In the new Caffeine-ated index is updated in small parts as new pages appear of old pages add new information, keeping the index up to date always. Google claims this Caffeine index is 50% fresher — and adds “hundreds of thousands of gigabytes per day” to its “nearly 100 million gigabytes of storage in one database” impressive indeed.
Most of all Caffeine is supposed to scale much better as future demands change, and Google will continue making improvements which we will see “in the months to come.”