With the arrival of the giant wave of the Internet, big data, cloud computing, Internet of Things and Artificial Intelligence (hereinafter referred to as AI) and other information technologies are being integrated with traditional manufacturing industries. The concept of “Smart Manufacturing” that arose from this is bringing profound changes to the global manufacturing industry. China’s manufacturing giants have also leveraged these technologies to initiate intelligent digital manufacturing and implement strategic transformation. One of the most innovative examples comes from the Midea Group (hereinafter referred to as “Midea”), known globally for its consumer electronics and HVAC products. In January 2017, it acquired the world’s leading robot giant KUKA AG, filling an important role in its smart manufacturing industry chain.
“Robots help us resolve the issue of ‘hands’ and ‘feet,’ while machine vision solves the problem of the ‘eyes’ and ‘brains.’ Excellent algorithm and powerful computing power form important pre-requisites for our industrial visual inspection cloud platform. The introduction of Analytics Zoo Big Data Analytics and the AI platform has helped us solve this issue by providing end-to-end algorithm training and cloud computing deployment capabilities for cloud platforms. Intel® Xeon® Scalable processors helped achieve faster and better computing. The combination of Intel’s software, hardware products, and technology has helped us in taking a solid step forward in smart manufacturing.” —Zheng Hu director Midea Vision Institute
Applying machine vision in industrial inspection is one of the important directions of smart manufacturing, but there are many issues for traditional machine vision solutions: on the one hand, complex production environments bring a large number of non-standardized feature recognition requirements, resulting in long development cycles and high cost of customized solutions. On the other hand, the diverse inspection content makes parameter calibration cumbersome and difficult for workers to use. In addition, traditional solutions often require coordination from mechanical components for positioning, thus occupying a large space for the production line and consequently, impacting the process flow.
Massive data resources from the production frontline equipped Midea with the basis to leverage AI technology to solve these problems by constructing an AI-based industrial visual inspection cloud platform, integrating data acquisition, model training, and algorithm deployment. It features workpiece calibration, image positioning and calibration, and Midea also hopes to shorten development cycles and save costs by deploying optimized deep learning training and predictive models. Ultimately, this development will improve the ease of use and versatility of a device.
In response to Midea’s need, Intel, which has long been in the field of AI and machine vision, provided Analytics Zoo, a unified analytics, and AI platform bringing together Apache Spark*, TensorFlow*, Keras*, and BigDL* to help Midea’s industrial visual inspection cloud platform to build a comprehensive deep learning process in a quick and agile manner. This comprehensive deep learning process ranges from frontend data preprocessing, to model training, inference, data prediction, and feature extraction. A variety of Intel® processors, especially Intel® Xeon® Scalable processors, provide the cloud platform with the computing power it needs. At present, this visual inspection cloud platform has been deployed in many production bases in Midea. The feedback from plant managers has been enthusiastic because the solution greatly improves fault detection rates, which significantly improves the product quality. In addition, this solution helps Midea reduce equipment cost and extend the equipment life cycle. Overall, the visual inspection cloud platform has won praise from frontline workers to top-level management.
Industrial inspection is an indispensable process in modern manufacturing. In the past, factories mainly detected defects in products using the human eye. This method is not only inefficient but also inaccurate. The introduction of machine vision introduces a hundred-fold increase in the accuracy and efficiency of product inspection. However, the construction of an effective automated machine vision system requires an investment of money and time for customized development and verification. At the same time, if the system is not versatile, it will be difficult to use on different production lines. Therefore, many traditional manufacturing companies are adopting a wait-and-see attitude towards the application of machine vision systems.
From the perspective of the Midea KUKA Machine Vision team, this problem is largely due to non-standard visual application scenarios. The differences in the detection environment and the respective needs, such as different angles of capture for images and differences in ambient brightness and surface curvature result in the need for customized solutions (including cameras, light sources, algorithms) for each project and a large number of tests to verify the feasibility of any solution.
The combination of AI and big data technology is the way to deal with these problems. The Midea KUKA Machine Vision team has worked with Intel to build a new, AI-based industrial vision inspection cloud platform, as shown in Figure 1, which connects all visual inspection terminal devices to the cloud through the network. The collection of image big data and the training of this data in the deep learning framework to obtain generalized feature parameters and models has allowed them to achieve agile, high-performance defect detection capabilities.
Figure 1. Schematic diagram of Midea’s industrial vision detection cloud platform architecture
Industrial vision inspection system combines AI with Cloud Services
Midea’s AI-based industrial vision inspection cloud platform is composed of two main parts: 1) the image acquisition frontend, consisted of components such as an industrial robot, industrial camera, and industrial computer, which is deployed on the factory production line; and 2) the backend system, which is supported by the deployment of the cloud-based Intel® architecture server cluster.
At the front end, as shown in Figure 2, the robot performing image acquisition is equipped with two industrial cameras, one for long-range shooting for detecting presence and location, and the other for close-up shooting for OCR recognition. Using microwave oven inspection as an example, when the system starts to work, through the linkage between the robot and the rotary table, the long-range camera is used to capture a global image of the surface to be inspected in order to calculate and detect the areas where OCR recognition is needed. The close-range camera is then activated to capture images of the identified local areas. Because industrial cameras on sites are often from different brands with most being non-intelligent traditional imaging devices, images and data are first transmitted to Intel® Core™ processor-based IPC for unified batch processing. Batch processing here includes size adjustment, denoising, and rotation of images, synchronization of clock data as well as consistency check of labels. After the batch processing by IPC, if the network environment is good, the images and data will be transmitted to the cloud for subsequent operations like training and inference; however, if the network environment is bad, edge servers based on Intel Xeon Scalable processors can be deployed on sites to use trained models to perform tasks like inference or model iteration and optimization.
On the end-to-end architecture, the system first uses the SSD (Single Shot Multibox Detector) model provided by Analytics Zoo to identify the pre-processed images and extract the objects that need to be detected, such as screws, nameplate labels, or models. Then, the AI capabilities provided by Analytics Zoo will help the cloud platform perform a series of AI processes such as massive data management, distributed model training, model redefinition, and model inference (in the cases where the network environment is good). Through the deep learning development framework such as TensoFlow* and BigDL* integrated in Analytics Zoo, the system can improve the recognition rate of the detected objects through continuous iterative distributed training.
Introducing the deep learning method into industrial inspection allows the industrial visual inspection cloud platform to perform quickly and in an agile manner, automatically identifying any defects in the product being tested, such as a missing screw, a missing nameplate, or a logo silk screen defect. And, importantly, the cloud platform can adapt well to non-standard change factors. Even with content and environment change, the cloud platform can adapt quickly, eliminating the need for lengthy new feature recognition and verification time. All this means that this solution can improve the robustness of detection, achieving a recognition rate of up to 99.8%1 and overcoming the issues with traditional visual inspection relying too much on image quality.
Since deployment in Midea’s production lines, the industrial vision inspection cloud platform has been highly appreciated by frontline producers. The Midea microwave oven factory equipment manager said: “First of all, the deployment of this new solution doesn’t require mechanical positioning and the space required is very small, so it doesn’t affect the production line. Secondly, it’s flexible and compatible so it can be used even when the production line is changed or upgraded in the future meaning this solution has a long lifecycle.”
Data from the Midea microwave oven visual inspection project shows that the new AI-based industrial vision inspection cloud platform solution has reduced the project deployment cycle by 57%, material costs by 30% and labor costs by 70%.2 For traditional manufacturing this is a transformative innovation. Based on its success, Midea plans to replicate the new solution in nine additional production lines within two months and to further replicate it in all microwave oven production lines in the near future.
Equipping the system with “Big Data + AI” end-to-end solution for the system
As the product of Intel’s latest technology in the field of big data and AI, Analytics Zoo, provides an end-to-end AI solution for Midea’s industrial vision inspection cloud platform. Analytics Zoo can run directly on a large data cluster of Intel architecture servers and can be further optimized with Intel Xeon Scalable processors to fully unleash its performance potential. At the same time, Intel® Math Kernel Library (Intel® MKL) and multi-threading technology integrated by Analytics Zoo also assist the Midea industrial visual inspection cloud platform in significantly improving feature training, image prediction, and data batch processing, as well as other aspects of work efficiency.
With the introduction of Analytics Zoo, Midea’s industrial vision inspection cloud platform easily enables efficient end-to-end AI development and deployment. As shown in Figure 3, during the data acquisition and pre-processing optimization phase, Analytics Zoo helps the cloud platform perform efficient distributed data pre-processing and code optimization, enabling it to read and pre-process images within 50 milliseconds.3 In the massive data management phase, it helps the cloud platform to perform data storage, classification, and update efficiently. In the distributed model training phase, Analytics Zoo assists the cloud platform to build the detection model while the TensorFlow optimizer integrated within it can quickly launch the distributed training process. In the model redefinition phase, Analytics Zoo performs parameter adjustments and accelerates the speed of model inference. The relevant test data shows that the model inference time of the cloud platform based on Intel® architecture platform has been shortened from 2 seconds to 124 milliseconds.4 Finally, Analytics Zoo also provides support for the command line mode and web service mode, which enables the cloud platform to interface smoothly with other Midea applications.
Figure 3. Analytics Zoo-based end-to-end AI solutions
Computing power is another key element of AI and the performance of Intel Xeon Scalable processors is key to providing a strong foundation for the industrial vision detection cloud platform. Through collaboration between Midea and Intel, Intel® Xeon® Scalable processors deployed in the cloud platform are fully optimized for performance using technologies such as Intel® Advanced Vector Extensions 512 (Intel® AVX-512). With its excellent parallel computing ability, Intel Xeon Scalable processors can meet the stringent demands of the cloud platform for computational power in model training and model inference.
Diversified deployment plans and greater industry influence
Through the continuous technical cooperation between Midea and Intel, the two teams also proposed a variety of edge computing-based deployment modes based on the enterprise’s actual production conditions. One solution is to deploy an Intel-based edge computing server on the frontend to perform algorithm processing as close as possible to the platform in order to improve the response speed. Another solution is to embed a video processing unit (VPU) such as Intel® Movidius™ Myriad™ X VPU in frontend cameras or industrial cameras. By doing so, large amounts of algorithm processing tasks are performed directly on the frontline of image acquisition to obtain faster and more sensitive processing capabilities, reduce system load and further reduce system operating costs.
At the same time, the optimization of AI processing flow based on Analytics Zoo is also proceeding progressively in the cloud platform. With the introduction of new algorithms and new technologies the two parties are planning to expand the application range of the Midea industrial visual inspection cloud platform. For example, the next step will be to extend the detection process to the more difficult scratch detection and weld defect detection, to further improve factory yield.
Looking to the future, the Midea KUKA Machine Vision team expects to share this innovative machine vision solution with industry peers to help drive China’s smart manufacturing development. To this end, Midea plans to provide industrial visual inspection technology based on a distributed deep learning framework for small- and medium-sized manufacturing companies through cloud-service or edge computing, so that these companies can replicate the capabilities on their own production lines As the concept of Anything as a Service (XaaS) becomes increasingly popular and through the efforts of Midea and, Intel as well as the entire manufacturing industry, in the near future we may see the joint launch of Industrial Inspection as a Service (IIaaS).