A new must-read white paper on on Wikibon.org, Storage Directions in an Era of Big Data,  defines the major forces and trends in the storage industry and the major trends that will define IT development for the coming decade. It also has established its author, David Floyer, former IDC analyst and co-founder and CTO of Wikibon.org, as the leading expert and thinker today in storage, just as that technology goes through a revolutionary change and becomes a driving force in IT, business, government, and society in general. 

This major paper examines all the technological developments, industry trends, and major outside influences from the growth of big data to the appearance of the first entirely new major storage technology in two decades, to the convergence of network protocols and of hardware. The product of more than 30 years of experience and research and available publicly for free, it defines each major trend in the industry and its impact on storage development both in terms of technology and architecture. It defines the storage industry and maps where it is going and why at a major turning point in its development.

It is impossible to overestimate the importance of this paper. It is he kind of seminal study that will be cited by everyone who writes on the storage industry or IT in general for a decade or more. Although it focuses on storage, its implications reach beyond that to IT in general and to business and social change as data, and therefore storage and data analysis, are becoming central not just internally to IT but to businesses and government where data analysis is driving major change, and to society in general.

Floyer starts by defining what he calls “Infrastructure 2.0”, the radical change from client/server and centralized processing to totally virtualized public and private environments, big data combining structured, semi-structured, and unstructured data, Hadoop database technologies, and a worldwide operating environment. He then looks at cloud storage services and their implications and then gets down to specific important technological changes including multi-core processors, Flash (NAND) storage, hypervisors and virtualization, server racks and blades, disk drive technologies, and storage efficiency technologies. He then discusses big data and network directions. He examines the network implications of each of these for the development of storage over the last several years. He concludes with a set of overall storage directions and conclusions where he presents his vision for the future of storage and to a great extent IT over the next decade.

In the process he has defined the basis for discussion of the future of technologies that are to a great extent defining our futures. Much of the most important research happening in this decade – whether it is making cellular carriers more responsive to the needs of their customers, mapping the genomes of millions of people to discover the genetic underpinnings of disease, identifying and tracking terrorists hidden in large general populations, or defining the precise impacts of global warming to help us prepare better – is driven by the analysis of huge amounts of data. This gives Floyer’s paper an impact that extends far beyond what is normal in the IT industry. It is a must read for anyone who wants to understand what is happening today.

David Floyer may be reached via the public web site www.wikibon.com. He appears regularly on SiliconAngle.tv.