Wikibon CEO David Vellante and CTO David Floyer have published a major “Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth” examining the major trends in IT technology in light of the overall issues IT faces in the new decade.

One of those issues is the explosion in the percent of the IT budget spent on staffing.  The report cites IDC figures that show labor accounted for less than 35% of overall IT costs in 1996. By 2013 they will be nearly 70% of the entire IT budget.

The second is the increasing competition from cloud service providers whose services are simple, relatively inexpensive, and straightforward. By comparison, decades of layering of large numbers of applications and technologies has created extremely complex, inflexible internal IT environments that are difficult and expensive just to operate, much less move forward to meet the new business challenges of the 21st Century.

Contributing to this is the proliferation of servers designed to support one application that, as a result, are extremely underutilized, making them costly in terms of direct costs, power, space, and staffing.

Underlying all of this is a major difference in approach between traditional IT, as exemplified by many internal IT organizations, and the new IT model provided by the large cloud services such as Facebook. The former is based on provisioning custom hardware infrastructures for each application, leading to massive complexity in large IT shops. The latter uses standardized IT building blocks that break down traditional IT silos of network, server, storage, and application to create infrastructures that are easily scaled and simple to manage. The result is a much more flexible and inexpensive infrastructure that frees IT to focus most of its resources on responding to new business challenges rather than just to keep everything running.

The report argues that to compete IT needs to adopt a similar approach. A few critical systems that put very large demands on the underlying infrastructure will still require custom hardware solutions. For the rest, IT should use a combination of standardized building-blocks bought from a single vendor with fully integrated server, storage, and network management and virtualization across the entire infrastructure to provide a common platform with high utilization.

This approach will decrease the cost of keeping the lights on dramatically, allowing IT to refocus a large amount of its resources, including staff, to meeting changing business needs and exploiting new business opportunities. The big danger, the report says, is the potential for vendor lock-in that vendors might exploit in the future to impose mainframe-type pricing. The report suggests that enterprises avoid this in part by dual-sourcing those hardware building-blocks and designing processes to avoid creating data silos, so that data can be migrated easily to new infrastructure from different vendors.

The report also examines the implications of new storage technologies, and in particular flash storage, on the infrastructure stack and the breakdown of the boundary between the server and storage device. It examines five major storage efficiency technologies and overall storage directions.

As with all Wikibon research, this report is provided free. Readers are encouraged to make corrections and add comments.