I just read Paul Graham’s excellent post on what happened to Yahoo and this one paragraph jumped out at me:

I didn’t realize the answer till later, after I went to work at Yahoo. It was neither of my guesses. The reason Yahoo didn’t care about a technique that extracted the full value of traffic was that advertisers were already overpaying for it. If they merely extracted the actual value, they’d have made less.

[From What Happened to Yahoo]

There are two very important lessons here that companies of all sizes should take away and the first is that much of the web economy is built on arbitrage, the selling of something for more than you buy it for. When you are in an arbitrage business and the market forces do not demand efficiency and economy then it’s unlikely that efficiency will become a priority until a competitive force decimates you.

This is probably one of the most difficult challenges that technology companies face, high margins create a cultural dynamic that is very difficult to overcome and it ignores development of new businesses that are built on delivering efficiency to the marketplace, which then makes the company ill-prepared to deal with critical shifts in the market.

The second lesson is that even exceptionally smart people have limited ability to forecast what will be important shifts in the market beyond the immediate future; history proves conclusively that the conventional wisdom at any time on any subject is often wrong. Yahoo has become less relevant because they could not see the shift in search, which is ironic given that their business was founded on the notion of search.

[Editor’s Note: Jeff cross-posted this at his personal blog. –mrh]