AWS and hyperspecialization: the more the better
Infrastructure is the secret to Amazon’s cloud dominance, so naturally the company has kept quiet about it. But last month at re:Invent 2013, vice president James Hamilton, an early advocate of commodity hardware who helped architect Microsoft’s data centers, opened up about the technologies that enable AWS to disrupt the economics of IT at petabyte scale. In his latest article, Wikibon Principal Research Contributor Stu Miniman separates the signal from the noise for the enterprise CIO.
Facebook, a hyperscale pioneer, has standardized its massive environment on a set of five internally-developed server configurations optimized for maximum efficiency. Amazon takes a different approach, leveraging an array of hyperspecalized components each designed with a specific workload in mind. AWS S3 stores trillions of objects generating more than 1.5 million requests per second, with DynamoDB averaging a mere 3 milliseconds of latency, Miniman details. At this scale, the added overhead of a more heterogeneous (but nonetheless tightly integrated) stack is more than offset by the cost benefits, he explains, especially since the company’s applications are developed from the ground up to run at scale.
Hamilton revealed in his speech that Amazon’s rack configurations weigh over one ton, significantly more than the densest commercially available Hadoop solutions. And on the storage side, the cloud giant leverages a distributed DAS architecture that includes both traditional hard drives and flash.
Amazon ties it all together with a white box network based on custom routers and protocols that, as Miniman observes, enable admins to rapidly troubleshoot an issue instead of having to wait months for a vendor to release a patch. The end result is unparalleled efficiency, which the company sustains with what he calls a low margin cycle of innovation that drives efficiencies and created value for customers. Traditional enterprises should take notice.
“CIOs need to pay attention to the hyperscale players which herald the direction of technology. The forecast of public versus private cloud usage over the next few years is hotly debated, but it is without doubt that infrastructure designs and operational models are seeing seismic shifts and Amazon is a key disruptor,” Miniman concludes.
Since you’re here …
… We’d like to tell you about our mission and how you can help us fulfill it. SiliconANGLE Media Inc.’s business model is based on the intrinsic value of the content, not advertising. Unlike many online publications, we don’t have a paywall or run banner advertising, because we want to keep our journalism open, without influence or the need to chase traffic.The journalism, reporting and commentary on SiliconANGLE — along with live, unscripted video from our Silicon Valley studio and globe-trotting video teams at theCUBE — take a lot of hard work, time and money. Keeping the quality high requires the support of sponsors who are aligned with our vision of ad-free journalism content.
If you like the reporting, video interviews and other ad-free content here, please take a moment to check out a sample of the video content supported by our sponsors, tweet your support, and keep coming back to SiliconANGLE.