Manufacturing & Supply Chain

Big data, analytics and AI are widely used in industries like financial services and e-commerce, but are less likely to be found in manufacturing companies. With some exceptions like predictive maintenance, few manufacturing firms have marshaled the amounts of data and analytical talent to aggressively apply analytics and AI to key processes.

Seagate Technology, an over $10B manufacturer of data storage and management solutions, is a prominent counter-example to this trend. It has massive amounts of sensor data in its factories and has been using it extensively over the last five years to ensure and improve the quality and efficiency of its manufacturing processes.

One of the primary foci for Seagate’s manufacturing analytics has been the automation of visual inspection of silicon wafers, from which disk drive heads are made, and of the tools to manufacture them. Multiple microscope images are taken from various toolsets throughout wafer fabrication, and these images play a key role in detecting faults within the wafer and monitoring the health of the toolsets. A factory controls team, headed by Sthitie Bom, Senior Director of Global Wafer Systems, has utilized the data provided by these images to create an automated fault detection and classification system with the ability to detect and classify wafer defects directly from the image. Other image classification models detect out-of-focus microscopes in tools.

Based on deep learning algorithms, these ADC (Auto Defect Classification) models were first deployed in late 2017, and since then the scale and power of image detection has grown extensively across the wafer factories in the U.S. and Northern Ireland, realizing multi-million dollar savings in inspection labor and scrap prevention. While the company has been able to reduce the number of manual inspections using these systems, the goal has been not just to free up inspection labor for other types of work, but also to make manufacturing processes more efficient. Visual inspection accuracy was at 50% several years ago, but now exceeds 90%.

Another area of success has been with the integration of sensor data from across the factory. Bom and the global wafer systems group normally have a strong preference for open source solutions. But it partnered with Savigent Software, an intelligent operations platform company, to integrate data from its many sensors across multiple machines in each factory and make the data available for analysis. Bom’s group was also the first within Seagate to use Hadoop to store data for use by analytics and AI applications.

Challenges with Model Explainability

While Seagate has made substantial progress in several different areas of manufacturing analytics and AI, there is still one key challenge. Given the large number of sensors and metrology tools in a factory, there can be some unavoidable false positives in the alerts for engineers. Traces of the transient faults in sensor readings are often never seen. Since most of the image recognition models that generate alerts are deep learning models, they are far too complex to be interpreted by the engineers. The only recourse for the engineer is a time-consuming series of investigations of the sensors and the fault readings. The analytics group currently provides an alert score to the engineers that predicts the likelihood of a valid sensor fault, but it doesn’t provide guidance as to why the score was assigned.

Bom’s group is working diligently to provide more model explainability, a problem that is also present in many other deep learning environments. Her group is focusing on two kinds of efforts in terms of explaining AI models. One is at a data science level. Where applicable, the group uses Shapley values, an approach to assessing the influence of particular features in models that is derived from game theory. The approach works well in relatively simple models, but currently requires too much expensive computation to be practical for deep learning models.

The second approach—a more real-world, practical treatment of explainability—involves establishment of a governance framework that extends standard software engineering practices to include controls on model behavior, using three different performance indicators. They include:

runtime errors monitoring—standard logging systems that monitor, diagnose, and explain and alert run time errors in a production-deployed model;
model performance monitoring—human-in-the-loop interfaces to capture domain expert validation of the algorithm’s decisions, along with supporting evidence on the plausible datasets that may explain the model decision;
consequential metric monitoring—these metrics measure anything that could be undesirably impacted as a result of adopting the machine learning decision, e.g. defective component hold rates.
These three governance pillars exist to understand and explain the consequences of the deployed models making incorrect decisions, knowing when the model performance degrades and the identification of remedial actions and the necessary human-in-the-loop validation.

In addition to the data science and governance approaches to explainability, Bom’s teams have joined forces with Seagate’s innovation center—Lyve Labs. Lyve Labs Israel is Seagate’s Innovation Center, opened in February 2020, whose mission is to form partnerships with the startup ecosystem to enable innovations with external technologies. They are also working closely with academic institutions in the U.S. and the U.K. to develop models that establish causal pathways among various events so that root cause analysis can be expedited, if not fully automated.

Leading AI Collaboration Among Minnesota High-Tech Manufacturers

Bom is aware that the AI and analytics challenges her group encounters are not, for the most part, unique to Seagate. She is collaborating with other high-tech manufacturers in the Minnesota area (where Seagate has extensive manufacturing operations) to determine how they deal with the issues. In October 2019 Seagate co-sponsored a two-day benchmarking session with Savigent and these companies to discuss the impact of AI on manufacturing operations.

Bom said that she learned two main things from benchmarking with the other companies. The first is not surprising given that Seagate often states that “data is in our DNA.” Last year, the company put out the Rethink Data report, based on an IDC survey showing that enterprises miss out on 68% of data available to them. Seagate, she believes, has a “very strong data infrastructure.” That provides the company with a much shorter runway for model development, because it has already invested heavily in structuring, cleaning, and democratizing data. Bom also felt that, compared to the other companies attending the session, Seagate appeared to have a more diverse AI portfolio. It had deployments in the visual inspection space, the time series space, and the soft sensing (when data from several sensors are measured and analyzed together) space.

The deployment successes of the group have been recognized by other organizations outside of Seagate. The deep learning-based visual inspection integration into the wafer factories won a Tekne Award in 2018 in the Emerging Technologies category, an award presented by the Minnesota Technology Association to celebrate a cutting edge product or service that is pushing the boundaries of technology. Seagate also brought home the Irish Manufacturing Research Award in 2019 for its implementation of AI-in-the-cloud dynamic fault detection in the wafer factories. There is much interest and investment across industries in the AI space with many successful proofs of concept; however, deploying a solution into production and then scaling it presents many challenges, especially in manufacturing. This makes the deployment successes of AI at Seagate Wafer particularly remarkable. Matt Johnson, VP for Process Engineering and Systems, summarizes it by saying, “The incorporation of AI/ML into the critical wafer monitoring systems has been instrumental in detecting issues faster, reducing the human resources required as part of ongoing monitoring of these processes, and improved the quality of the wafers that are shipped to our downstream internal customers.”

Sthitie Bom attributes the team’s success to its ethos of value creation, continuous innovation and strong vision. The group is single minded about addressing factory AI capabilities and is pursuing them across multiple fronts.

Bom has led the development of an ongoing consortium within the Minnesota Technology Association from the companies participating in the 2019 benchmarking meeting. She is pursuing a doctoral degree in cognitive science at the University of Minnesota, as well as serving on the advisory board to the statistics department at the university. Bom is a shining example of how a focused and value-driven leader can make a transformative difference in exploring and exploiting the power of AI in high-tech manufacturing.

Reference: Forbes.com