The mainframe is an often neglected internal source of invaluable data for enterprises developing Big Data strategies, writes Wikibon Principal Research Contributor and Big Data analyst Jeff Kelly in “The Mainframe and Big Data Analytics”.Despite the talk of the mainframe’s demise, many core transactional applications including reservation and online sales systems, payroll processing and inventory control, still run on Big Iron in corporations. Unisys and IBM continue to develop their mainframe systems, and IBM’s System z had its most successful quarter in terms of MIPS sold just two years ago.

Mainframes are reliable and highly automated, often running for years with virtually no human intervention. Mainframe analyst Josh Krischer tells a story about an Eastern European airline that ran its core systems on an IBM mainframe for five years without ever touching the machine after its mainframe IT guy retired.

However, that data often stays locked in the mainframe because sorting and transforming it to perform complex data analytics has been expensive and robs the core applications of CPU cycles, Kelly writes. “Considering the type of data stored in mainframes — namely customer financial transaction data – enterprises are leaving an awful lot of value on the table.”

The good news is that Big Data technologies are making it easier and less costly to export that data. One option is to use JCL batch workloads to move the data to Hadoop, where it can be processed, combined with other appropriate data and, for instance, moved to a NoSQL database to support forward-looking analysis to support business decisions. The challenge is the lack of native connectivity between mainframes and Hadoop. However, Kelly writes, Syncsort and other vendors have develop data ingestion techniques to significantly reduce the problem.

“Whatever options are chosen,” Kelly writes, “enterprises with heavy mainframe use cannot afford to overlook this important source of value.”

Like all Wikibon research, Jeff’s full Professional Alert is available without charge on the Wikibon Web site. IT professionals are invited to register for free membership in the Wikibon community. This allows them to participate in Wikibon research and publish comments, questions, and their own research on the Wikibon site.

image courtesy IBM