Intel’s new XEON processor family provides next-generation technology with chip-level support for erasure coding, a more efficient way to provide backup, says Chinmay S. Patel, ecosystems manager in the Storage Group at Intel. It provides 95% storage efficiency by deduping as data comes in and by eliminating the need for the multiple data copies typical of RAID implementations, he said in an interview webcast on the Cube from NAB Monday (full video to come). “XEON lets you do that in-line rather than later, providing much higher storage efficiency and controlling costs. EMC Isilon, Cleversafe, and other storage vendors are implementing these advanced methodologies using XEON,” he said, and working with those partners on the new data architecture that is evolving is an exciting assignment.

The new data architecture Intel is seeing develop in big data implementations is for all incoming data to go into a single primary object data store. Then individual applications withdraw the data they need and move that data to the appropriate storage tier.

One problem that this creates, particularly as disk capacity keeps growing, is that rebuilding a data store after a problem can take 48 hours or more. Erasure coding, he said, cuts that time while simultaneously making much more efficient use of disk space. This is part of a fundamental shift in the technology that manages content and changes how applications are deployed and managed.

This added efficiency allows companies to move their data archives off tape to near-line disk storage, making the archive much more available when it is needed. It will yield a 40% to 50% savings in storage efficiency. Wikibon founder David Vellante said that one user presented a case in which erasure coding provided a company with a 75% jump in storage efficiency.

This does not mean RAID is dead, Patel says. “A lot of investment has gone into RAID, and it will fade slowly. But erasure coding is what you will use in your cloud infrastructures.”