Overview
Deploy an industrial AI application for product quality checks using the Intel® Smart Edge Open platform. Includes support for two types of defect detection: missing components and short circuits due to solder bridge formed during the assembly process.
To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.
Once you have installed the Intel® Smart Edge Open Developer Experience Kit, select Configure & Download to download the reference implementation and the software listed below.
- Time to Complete: 1-1.5 hours
- Programming Language: Python*, C++
- Available Software:
- Intel® Smart Edge Open version 22.03 or above
- Intel® Open Edge Insights version 3.0.0
Target System Requirements
Intel® Smart Edge Open Nodes
- One of the following processors
- Intel® Xeon® Scalable processor.
- Intel® Xeon® D processor.
- At least 64 GB RAM.
- At least 265 GB hard drive.
- An Internet connection.
- Ubuntu* 20.04 LTS Server.
- IP camera or pre-recorded video(s)
How It Works
The Printed Circuit Board (PCB) defects detection is a reference implementation enabled for detection of defects in the components assembly in a factory environment. The IP Camera will record the video of the assembled boards moving over a conveyor belt. These IP cameras will stream the video to Open Edge Insights (OEI) video processing applications namely Video Ingestion, Video Analytics, Time Series analysis, etc. using the RTSP streaming protocol. The captured video is analyzed and filters out selected images for further defect analysis.
The Intel® Smart Edge Open Developer Experience Kit platform infrastructure is used to deploy the application pods of OEI performing image processing functionality. The purpose of this reference implementation is to showcase how the Intel® Smart Edge Open Developer Experience Kit platform can be used for deploying Industrial AI applications for product quality check. There are two types of defect detection supported in this demo: missing component detection and short circuit due to solder bridge formed during the assembly process.
The video file used in this reference implementation is pcb_d2000.avi which is sent from the camera stream pod as RTSP stream. This PCB defects detection reference implementation includes five types of application pods in the Intel® Smart Edge Open Developer Experience Kit Edge Nodes.
Edge Insights for Industrial from Intel is a set of pre-validated ingredients for integrating video and time-series data analytics on edge compute nodes.
- Config Manager Agent
Config Manager Agent is an OEI service responsible to put the OEI services configs to the OEI config manager data store. It creates required provisioning folders with the right permissions needed for other OEI services via volume mounts. - Camera Stream
The Camera Stream pod simulates IP cameras in the real environment. It is used to send recorded video pcb_d2000.avi file as an RTSP stream like an IP camera. On LTE/5G Network real-time deployment, replace Camera Stream pod with a real IP camera. - Video Ingestion
The Video Ingestion pod is mainly responsible converting the video received from camera stream into frames. It filters the frames and ingests these video frames into the OEI stack for further processing and defect analysis. - Video Analytics
The Video Analytics pod receives frames from the Video Ingestion pod, performs classifier converting to Universal Disk Format (UDF), and does the required inferencing on the chosen/support hardware (CPU, VPU) using OpenVINO™ toolkit. In this PCB demo, udfs filter configured as pcb. pcb_filter which will detect a defect on the board that has any solder short or missing part. Video Analytics identifies the defects and marks the defect on the images. - Web Visualizer
The Web Visualizer pod mainly displays the frames on GUI display received from Video Analytics pod.
The block diagram below shows the video/image flow through the application.
Get Started
These instructions walk you through installing and uninstalling an example PCB Defect Detection reference application using commands. After installation, you will be able to see the application is running on the cluster. The uninstall commands can be used to uninstall the application.
Installation Prerequisites
To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.
Ensure that the following conditions are met properly to ensure a smooth installation process for a reference implementation done through Edge Software Provisioner (ESP) Intel® Smart Edge Open Developer Experience Kit package.
Hardware Requirements
Make sure you have the hardware as specified in the Target System Requirements section for fresh ESP Intel® Smart Edge Open Developer Experience Kit installation.
Ensure that the steps below are completed in the ESP Intel® Smart Edge Open Developer Experience Kit installation:
- Proxy Settings
If you are behind a proxy network, ensure that proxy addresses are configured in the system:export http_proxy=<proxy-address>:<proxy-port> export https_proxy=<proxy-address>:<proxy-port>
- Ensure the /etc/wgetrc file is configured with required proxy settings as follows:
https_proxy=<proxy-address>:<proxy-port> http_proxy=<proxy-address>:<proxy-port> ftp_proxy =<proxy-address>:<proxy-port> use_proxy = on -
Date and Time
Make sure that the date and time are in sync with current local time. -
Create Non-Root User
Make sure that non-root user with name and password smartedge-open and with sudoers permission is created. If not available, follow the instructions below.-
To create a user smartedge-open, execute the command:
useradd -s /bin/bash -d /home/smartedge-open/ -m -G sudo smartedge-open passwd smartedge-open -
A password for the given user is required:
passwd "smartedge-open" -
As some tasks require root privileges, the non-root user needs to have a possibility to become a root:
echo "smartedge-open ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/smartedge-open
-
-
Exchanging SSH keys between hosts
Exchanging SSH keys between hosts permits a password-less SSH connection from the host running Ansible to the hosts being set up. If password-less SSH is not permitted, follow the instructions below to exchange SSH keys between hosts. Generate and install the public key on your host by logging in as root and executing the below commands:
In the first command, the Edge Controller host must have a generated SSH key. The SSH key can be generated by executing ssh-keygen and obtaining the key from the output of the command.
In the second command, the generated key must be copied to every Edge Node host, using the ssh-copy-id command.
ssh-keygen ssh-copy-id smartedge-open@<smartedge-open-node-ip-address>
Install Python dependent libraries
Use the following commands:
pip3 install –-user Cython
sudo apt-get install python-dev
sudo apt-get install python3-dev
Install the Reference Implementation
Select Configure & Download to download the reference implementation and then follow the steps below to install it.
- Make sure that the Target System Requirements are met properly before proceeding further.
- For single-device mode, only one machine is needed. (Both controller and edge node will be on the same device.)
- For multi-device mode, make sure you have at least two machines (one for controller and other for Edge Node).
NOTE: Multi-device mode is not supported in the current release.
- Open a new terminal and login to server as a non-root user.
ssh smartedge-open@<server-ip-address> -
Confirm that the server is configured with ESP Intel® Smart Edge Open Developer Experience Kit as prerequisite.
-
Move the downloaded zip package to /home/<non-root-user> folder.
mv <path-of-downloaded-directory>/Wireless-network-ready-pcb-defect-detection.zip /home/<non-root-user> - Go to the /root directory of the non-root user using the following command and unzip the RI:
cd /home/<non-root-user> unzip Wireless-network-ready-pcb-defect-detection.zip - Go to Wireless-network-ready-pcb-defect-detection/ directory:
cd Wireless-network-ready-pcb-defect-detection - Change permission of the executable edgesoftware file:
chmod 755 edgesoftware - Run the command below to install the Reference Implementation:
./edgesoftware install - Installation of the package starts.
Figure 3: Installation Start Screen
- When the installation is complete, you see the message Installation of package complete and the installation status for each module.
NOTE: Installation failure logs will be available at: /var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_<version>/Wireless_NetworkReady_PCB_defect_detection/install.log
Where <version> is the package version downloaded.
Figure 4: Installation Successful
- If Intel® Smart Edge Open Developer Experience Kit is installed, running the following command should show output similar to the image below. All the pods should be either in the running or completed stage.
kubectl get pods -A
Figure 5: Pods Status
Note: The status of the Visualizer pod might be CrashLoopBackOff. This is an expected behavior and can be ignored.
- If PCB Defect Detection is installed, running the following command should show output as follows:
$ smartedge-open@host:/$ kubectl get pods -n smartedge-apps NAME READY STATUS RESTARTS AGE deployment-etcd-ui-b6f9868f5-ll9nc 1/1 Running 0 35m deployment-video-analytics-5749dbcbc9-554rf 1/1 Running 0 35m deployment-video-ingestion-7485bdff74-2jbbt 1/1 Running 0 35m deployment-webvisualizer-76496f95d6-rzpfp 1/1 Running 0 35m ia-configmgr-agent-6f7c488fd5-hk5sw 1/1 Running 0 35m
Figure 6: PCB Defect Detection Pods Status - Run the below command to make sure the network policy is created:
Note: In Intel® Smart Edge Open, the default network policy blocks ingress traffic to all pods, hence the pcb defect detection pod network policy is created to allow the ingress traffic.
$ smartedge-open@:/$ kubectl get networkpolicies -n smartedge-apps NAME POD-SELECTOR AGE deny-from-other-namespaces <none> 26d etcd-ui-network-policy app=etcd-ui 17m webvisualizer-network-policy app=webvisualizer 17m
Figure 7: PCB Defect Detection Network Policy
- Run the command below to check the Docker* images and their details:
$ smartedge-open@host:/$ docker images | grep ia_ openedgeinsights/ia_web_visualizer 3.0 dc770e4abb2d 5 weeks ago 2.77GB openedgeinsights/ia_video_analytics 3.0 ae2d02273229 5 weeks ago 3.05GB openedgeinsights/ia_video_ingestion 3.0 ac31dafabc77 5 weeks ago 3.3GB openedgeinsights/ia_etcd_ui 3.0 939cb1e5db69 5 weeks ago 1.11GB openedgeinsights/ia_configmgr_agent 3.0 278d619c406c 6 weeks ago 268MB - List the reference implementation deployed module using the following command:
./edgesoftware list
Figure 8: PCB Defect Detection Module List -
Uninstall the reference implementation module using the following command:
./edgesoftware uninstall <pcb-module-id>
Figure 9: Uninstallation Successful -
Run the command below to make sure the network policy is deleted after uninstallation:
smartedge-open@:/$ kubectl get networkpolicies. NAME POD-SELECTOR AGEs block-all-ingress <none> 32d
Node Feature Discovery (NFD)
PCB Defect Detection uses the Intel® Distribution of OpenVINO™ toolkit, which is optimized for Intel® processors that support special instructions like AVX512VNNI for optimized performance. The deployment of this application will require the node with this feature supported on the node along with Ubuntu 20.04 OS. This NFD feature ensures to deploy the application on the node supported with these features.
NFD is installed by Intel® Smart Edge Open Developer Experience Kit and running as two pods on Intel® Smart Edge Open, as shown below.
$ kubectl get pods -A | grep smartedge-system
smartedge-system nfd-release-node-feature-discovery-master-7b94765ccf-9ghjg 1/1 Running 5 (83d ago) 85d
smartedge-system nfd-release-node-feature-discovery-worker-dq4x6 1/1 Running 5 (83d ago) 85d
PCB Defect Detection pods scheduled and running successfully on Intel® Smart Edge Open node based on hardware capabilities of Intel® Xeon® Scalable server.
$ kubectl get pods -A | grep smartedge-apps
smartedge-apps deployment-etcd-ui-b6f9868f5-ll9nc 1/1 Running 0 54m
smartedge-apps deployment-video-analytics-5749dbcbc9-554rf 1/1 Running 0 54m
smartedge-apps deployment-video-ingestion-7485bdff74-2jbbt 1/1 Running 0 54m
smartedge-apps deployment-webvisualizer-76496f95d6-rzpfp 1/1 Running 0 54m
smartedge-apps ia-configmgr-agent-6f7c488fd5-hk5sw 1/1 Running 0 54m
The following output shows a description of one PCB Defect Detection pod, which shows that it is running successfully with the NFD feature.
$ kubectl describe pod deployment-video-analytics-5749dbcbc9-554rf -n smartedge-apps
...
...
...
Node-Selectors: feature.node.kubernetes.io/cpu-cpuid.AVX512VNNI=true
feature.node.kubernetes.io/system-os_release.ID=ubuntu
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 56m default-scheduler Successfully assigned smartedge-apps/deployment-video-analytics-5749dbcbc9-554rf to esi62
Normal Pulled 56m kubelet Container image "openedgeinsights/ia_video_analytics:3.0" already present on machine
Normal Created 56m kubelet Created container ia-video-analytics
Normal Started 56m kubelet Started container ia-video-analytics
Multi-Tenancy and Multi-Verticality Feature
Besides covering multiple IoT verticals, the RIs with Intel® Smart Edge Open increase time-to-market for Independent Software Vendors (ISVs) and provide a reference for edge solution builders looking to deploy multi-tenant and multi-vertical services on the edge platform. Intel® Smart Edge Open via the Intel® Smart Edge Developer Experience Kit offers support for multiple software tools/frameworks (including OEI, OpenVINO™, WebRTC) and diverse, multi-segment Reference Implementations to run simultaneously on the same edge platform) showcasing the foundational principals of interoperability across Intel offerings and assets.
Multiple Reference Implementations can be deployed on a single instance of Intel® Smart Edge Open Developer Experience Kit using this feature. The Multi-Tenancy and Multi-Verticality feature is supported and tested with the following set of Reference Implementations with Developer Experience Kit v22.03:
- Wireless Network Ready PCB Defect Detection (v5.0.0)
- Telehealth Remote Monitoring (v3.0.0)
NOTE: The reference implementations listed above were validated to support this feature; however, the feature is not limited to these RIs only.
Public Helm Registry for Helm Charts
Installation of PCB Defect Detection Reference Implementation on Intel® Smart Edge Open is accomplished using Helm charts. Earlier Helm charts used to be a part of Reference Implementation installation package. Now a global Helm repo is issued, so that Reference Implementation Helm charts can be accessible from private and public network. This will speed up and ease the process of introducing updates and their integration with the Reference Implementation.
Visualize the Output
To visualize the results, open Google* Chrome and navigate to the links below for respective results:
Web Visualizer
- http://<Controller-System-IP-Address>:30009 -DEV Mode
- http://<Controller-System-IP-Address>:30007 -PROD Mode
Etcd UI
- http://<Controller-System-IP-Address>:30010
To access the visualizer, log in with the username = admin and password = admin@123
PCB Defect Detection Results
You can view reference input and output videos of the PCB defect detection demo in GitHub.
Refer to the PCB defects detection results section to cross-verify the results on your setup.
Following are the reference input and output videos of PCB defect detection demo.
-
Input PCB rtsp stream: The input video file has three PCBs rotating through the screen are sent from the camera stream pod, two of these boards are defective out of three.
-
Output defect detection on PCB: The Visualizer pod showing the output of defect detection on PCB image, i.e. with red square box defect detection marked as PCB board missing component and solder bridge causing a short circuit.
Summary and Next Steps
The PCB demo sample application when deployed on the Intel® Smart Edge Open Developer Experience Kit creates an impactful edge computing use case that utilizes the capability of Intel® Smart Edge Open Developer Experience Kit and Open Edge Insights (OEI).
Learn More
To continue learning, see the following guides and software resources:
Troubleshooting
If any issue is faced during installation, ensure that the Installation Prerequisites are configured properly.
Pods Status Check
Verify that the pods are Ready as well as in Running state using the following command:
kubectl get pods -A
Pod Status shows "ImagePullBackOff" for a long time
If the pods are in ImagePullBackOff state for a long time as shown below, run the following commands.
Describe the pods using:
kubectl describe pod -n smartedge-apps <pod name>
Fetch the image name with the tag and do a manual Docker pull using the commands:
docker login
docker pull <image-name:tag>
Pod Status shows "ContainerCreating" for a long time
If pod status shows ContainerCreating or Error or CrashLoopBackOff for 5 minutes or more, run the following commands:
./edgesoftware uninstall -a
./edgesoftware install
Pod Status shows "pending" for a long time
If only video-analytics pod status is pending for 5 minutes or more, run the following command:
kubectl describe pods -n smartedge-apps
In video-analytics pod, if you see the following event due to FailedScheduling, it may be possible that AVX-512VNNI instruction set is not available in system CPU as mentioned in the NFD feature section.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 43s (x9 over 7m58s) default-scheduler 0/1 nodes are available: 1 node(s) didn't match Pod's node affinity/selector.
If all four pods etcd-ui, video-analytics, video-ingestion and web-visualizer are in pending state for more than 5 minutes, check if the Operating system is Ubuntu 20.04 LTS or not.
Uninstallation Failure
If uninstall command (./edgesoftware uninstall < module id >) fails, manually uninstall PCB Defect Detection deployment and network policy, which are specific to pcb defect detection pod:
helm uninstall eii-deploy -n smartedge-apps; helm uninstall eii-provision -n smartedge-apps
kubectl delete networkpolicy etcd-ui-network-policy webvisualizer-network-policy -n smartedge-apps
Installation and Debug Log Info File Path
Installation log info of Intel® Smart Edge Open Developer Experience Kit will be available at:
/var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_3.0.0/Smart_Edge_Open_Developer_Experience_Kits/install.log
Installation log info of Wireless Network Ready PCB Defect Detection module will be available at:
/var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_<version>/WirelessNetwork_Ready_PCB_defect_detection/install.log
Where <version> is the package version downloaded.
Execute the command below to consolidate a list of the log files in tar.gz compressed format, e.g., Wireless_NetworkReady_PCB_defect_detection.tar.gz.
tar -czvf Wireless_NetworkReady_PCB_defect_detection.tar.gz /var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_5.0.0/Wireless_NetworkReady_PCB_defect_detection/ /var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_3.0.0/Smart_Edge_Open_Developer_Experience_Kits/
Support Forum
If you're unable to resolve your issues, contact the Support Forum.
-
Make sure you have an active internet connection during the full installation. If you lose Internet connectivity at any time, the installation might fail.
-
Make sure you are using a fresh installation. Earlier software, especially Docker* and Docker Compose*, can cause issues.
-
Use the kubectl commands like describe, log, get etc. to analyze the problems.
-
Ensure the proxy settings are properly set.
-
In case of Docker pull rate limit issue, we recommend you use the Premium/PRO Docker teams to get unlimited Docker pull support.