Minimize Manufacturing Data Management Costs

Concerned about manufacturing data management costs? This white paper describes how Intel IT developed a custom benchmark to evaluate MPP databases and identify a cost-optimized solution that meets our technical requirements.

As Intel manufactures hundreds of millions of complex products every year, Intel IT collects and stores terabytes of manufacturing data to support continual engineering data analysis. As the volume, velocity and complexity of the data increases, it is imperative that we maintain this decision support system at the lowest possible cost. Additionally, we need to be able to assess the cost for future scaling needs. Therefore, we decided to evaluate the scalability, performance and cost of several Intel® architecture-based massively parallel processing (MPP) relational database management systems (RDBMS). We found that industry standard benchmarks did not closely resemble our manufacturing data and did not measure the metrics that were important to us. Therefore, we created a custom MPP RDBMS benchmark that helped us choose a cost-optimized solution.

The custom benchmark is based on a representative subset of data, obfuscated to protect intellectual property, and bloated it to the desired dataset size. The custom benchmark emulates the ingestion process as well as the mixed workload of the consumption through SQL.

A unique aspect of our benchmark is our use of k-means clustering and Classification and Regression Trees (CART) to perform workload characterization. This helped us build a benchmark that is as real-world as possible.

We used the custom benchmark to complete a comprehensive technical proof of concept (PoC) with several industry-leading MPP RDBMS vendors whose products run on Intel® architecture. We are confident that this benchmark enabled us to choose the best Intel® Xeon® processor-based MPP RDBMS solution while keeping manufacturing data management costs under control.

Also, based on the evaluation results, the vendors we worked with have improved their products, strengthening the entire industry ecosystem. And, with the release of the 4th Gen Intel® Xeon® Scalable processors and associated accelerators, we’re expecting that RDBMS vendors will make their products even more cost competitive. By sharing our benchmark methodology, we hope to help other companies to understand their data better and select a data management system that meets their needs.