The recent crisis surrounding Samsung Electronics Co. Ltd.’s overheating smartphones has dramatized a problem that affects makers of durable products across industries: Once a product leaves the factory, it’s devilishly difficult ever to find it again.

Product lifecycle management has made it possible for manufacturers to standardize information-sharing through the design, development and production process. However, for most companies PLM ends at the shipping dock. Once the product hits the supply chain, it disappears into a cloud of disconnected logistics and customer service systems. As Samsung found, this can make it almost impossible to track problems in the field back to their source,

Siemens Product Lifecycle Management Software Inc. has stepped into this void with Omneo, a software-as-a-service application that gives manufacturers visibility into their supply chains to pinpoint problems in the field, avoid costly recalls and improve product quality. The service uses an enterprise data hub based on the Hadoop big data processing framework and powered by Cloudera Inc.’s Manager and Search, along with open-source components HBase, Hue, Oozie and Pig.

Beyond relational

Relational database engines weren’t well-suited to Omneo’s unique need to build an extensible master record of product information. Relational database management engines were too rigid to handle the wide range of formats, table structures, field names and product identification numbers that needed to be harmonized. Relational stores also quickly ran out of room, which required archiving and which limited the ability of Omneo customers to conduct searches on historical data.

“We had somewhat limited budget and nearly infinite need for scalability,” said Kathleen DeValk, chief architect at Siemens. Commercial software products required a forklift upgrade when capacity limits were reached. “We wanted more of a horizontal scaling approach than vertical scaling,” DeValk said.

The team looked at a wide variety of possibilities, including IBM’s Netezza, columnar stores and NoSQL HBase alternatives like Cassandra and MongoDB, but their checklist kept bringing them back to Hadoop. “We didn’t need a giant appliance. We just needed to put a couple of nodes together and grow it over time,” DeValk said.

No one on the Omneo team had Hadoop experience, so Cloudera’s Express starter package presented an attractive option. “I deployed a virtual machine on my laptop and had it running in a day,” DeValk said. The team conducted some benchmark tests comparing Cloudera’s total cost of ownership to Netezza’s and concluded that the Hadoop option was about 90 percent lower on a cost-per terabyte basis.

Data omnivore

Omneo is a data omnivore. It pulls in data from a customer’s manufacturing records and combines it with information from suppliers, field service partners and even after-market repair and remanufacturing services to create a single lifecycle record.

Siemens uses software from Pentaho Corp. to integrate data from multiple sources using a simple drag-and-drop interface. Customized MapReduce algorithms distribute the workload across a Hadoop cluster, and HBase stores records for real-time access.

HBase may not enjoy the market buzz of some of its commercial competitors, but for Siemens the fit was right. “HBase was the best-performing option when running at scale,” DeValk said. Its compatibility with Hadoop, along with its availability and flexibility to handle a wide range of data formats, made it the best core database engine Siemens found for the new service.

Not all processing is performed in HBase. For high-performance SQL queries the team uses the Apache Impala distributed query engine and Apache Parquet columnar storage format. It’s also now beginning to test stream processing using Apache Spark and Storm, and is experimenting with Amazon Web Services Inc.’s Kinesis as increasing amounts of its workload move to the cloud.

Find it fast yourself

The result is a self-service engine that makes it possible for customers to do their own supply chain and historical data tracking. If a product is showing a high level of field failures, customers can trace it back through the lifecycle to find out when the product was built, what suppliers were used, where supplies came from and who has sold and supported it in the field.

Users typically run a search and then dig into the results using embedded Omneo analytics. “We try to do most of the heavy lifting by getting everything into the system up front and then contextualizing,” DeValk said. “Once data is mapped to our system, the process becomes scheduled and automated. People immediately start seeing benefit.”

That’s one reason Omneo is seeing more than a 50 percent annual growth in new business from existing customers as clients report an average of $15 million to $25 million in yearly savings. As a result, “We have a lot of customers knocking down our door to get their hands on the solution,” DeValk said.

Photo by MustangJoe via Pixabay