I've seen X-Cart pre 4.1.x take down Quad CPU servers with massive amount of data crunching power with a low number of concurrent users. This is the main reason 4.1.x was written the way it was, to consume as little server resources as possible and be more efficient. A Fortune 5 isn't going to plug-n-pray a commercial Cart but rather write their own.
Google does do something very similar, the html catalog is a semi-spider, so in essence X-Cart is doing the same. Collect data, store it on a file system. Granted Google is in a much much LARGER scale, and they surely are not doing it in a interpreted language like PHP or even low level C. I would almost believe they are using Assembly mixed with C or Python to squeeze as much as they can out of low end machines in their data centers that do nothing BUT parse data all day long.
Proof is in the pudding:
http://www.uwtv.org/programs/displayevent.asp?rid=2459
Anyhow not to get off topic, just my 2cents.