How the Textile Defect Classifier Works

The tutorial shows an example of how the Intel® Distribution of OpenVINO™ toolkit (OpenVINO™) optimized classification networks can be leveraged in industrial quality inspection applications and deployed using Edge Insights for Industrial. This tutorial includes:

• Textile inspection video: Sample video that is used to simulate a textile inspection camera stream. The video includes 5 types of common textile defects, some of which are not easily noticeable to the human eye.

• Textile Defect Classifier Custom UDF: Ingest the frames from the sample test video and performs inference on each frame to classify it as defective or not. The rate at which each frame is classified is also recorded and displayed on the output Visualizer window of the Edge Insights for Industrial software stack. The UDF utilizes the Inference Engine module from the Intel® Distribution of OpenVINO™ toolkit to optimize the inference on Intel hardware.

• Configuration files: Visualizer and WebVisualizer configuration files to add the textile defect classifier UDF to the Edge Insights for Industrial pipeline.

• Intel hardware-optimized models: Deep learning classification models to detect textile defects optimized using the Intel® Distribution of OpenVINO™ toolkit; IR files generated by the Model Optimizer.

All communication between the image acquisition from sample video and analytics to detect defects (custom UDF) to visualizing the inspection results (Visualizer) occur over the Edge Insights Internal Message Bus, as shown in the following figure.

The sample textile data included as part of this application has 5 types of common textile defects, as shown in the following figure.

Get Started

The Textile Defect Classifier reference implementation is a plug-and-play application developed for the Edge Insights for Industrial package.

Prerequisites - Edge Insights for Industrial Installation Options

Download and install the Edge Insights for Industrial package after selecting the Textile Defect Classifier Module in the custom download section of the video or video-timeseries use case. Both the use cases will download the video pipeline related services (custom UDF, Visualizer, and WebVisualizer) required to satisfy the prerequisites for this application.

Step 1: Add Textile Defect Classifier UDF components

1. Change working directory to the Textile Defect Classifier package and create an environment variable $EII_ HOME to point to the IEdgeInsights install path for ease of reference:

   cd $WORKDIR/edge_insights_industrial/Edge_Insights_for_Industrial_<version>/textile_defect_classifier/textile_defect_classifier/
   
   
   EII_HOME=$WORKDIR/edge_insights_industrial/Edge_Insights_for_Industrial_<version>/IEdgeInsights

   NOTE:

   $WORKDIR in this tutorial refers to the host system path where the Edge Insights for Industrial package was downloaded and unzipped.

   <version> indicates the downloaded version of Edge Insights for Industrial.

2. Copy the Textile Defect Classifier UDF file to the Edge Insights for Industrial UDFs directory:

   cp -r PyTextileClassificationIngestion/ $EII_HOME/CustomUdfs/.

3. Modify the Visualizer and WebVisualizer configuration files to add Textile Defect Classifier to the IEdgeInsights pipeline.

   1. Make a copy of the original Visualizer and WebVisualizer config.json files:

      mv $EII_HOME/Visualizer/config.json $EII_HOME/Visualizer/config.json_original
      
      
      mv $EII_HOME/WebVisualizer/config.json $EII_HOME/WebVisualizer/config.json_original

   2. Copy the new Visualizer and WebVisualizer config.json files to execute the textile classification application.

      Optionally, edit the PyTextileClassificationIngestion/config.json to use the inference hardware of choice [CPU/GPU/Myriad/HDDL]. Default is CPU.

      Note that the deep learning model used in the reference implementation is a deep model, and hence will take a few minutes for the initial loading into an accelerator (GPU/Myriad/HDDL) memory.

      cp vis_config.json $EII_HOME/Visualizer/config.json
      cp webvis_config.json $EII_HOME/WebVisualizer/config.json

   3. Copy the custom UDF yml file to include the custom textile UDF container:

      cp video-streaming-textileUDF.yml $EII_HOME/build/usecases/.

4. Follow the Edge Insights for Industrial instructions to provision and launch the containers:

   # Configure the IEdgeInsights pipeline to use UDF, Visualizer & WebVisualizer containers
   cd $EII_HOME/build/
   sudo python3 builder.py -f usecases/video-streaming-textileUDF.yml
   
   
   # Provision
   cd $EII_HOME/build/provision
   sudo -E ./provision.sh ../docker-compose.yml
   
   
   # Build
   cd $EII_HOME/build/
   sudo sg docker -c 'docker-compose -f docker-compose-build.yml build python_textile_classification'
   sudo sg docker -c 'docker-compose up -d'
   
   
   # Enable visualizer display
   xhost +

The result looks similar to:

Step 2: Check the Reference Implementation

Edge Insights for Industrial builds the custom UDF, Visualizer, and WebVisualizer containers. The UDF container streams the sample textile video and performs classification on each frame to detect occurrence of defects. Once the frame has been classified, the image is displayed on the Visualizer output window with the classification result and inference time.

Execute the following command to ensure all the containers are running without errors:

sudo docker ps

Check for Success

If it was successful, the results will be similar to:

Visualizer Output

The sample textile video has 5 types of defects that can be monitored using the visualizer output.

Step 3: Stop the Vision Pipeline

To stop Edge Insights for Industrial software, execute the following command:

cd $EII_HOME/build
sudo sg docker -c 'docker-compose down'

ATTENTION: To revert to the default PCB Demo application, move the Visualizer and WebVisualizer config.json files to the original copies [Step 1.3.a] and rebuild using the instructions in Step 1.4.

Troubleshooting

Troubleshooting a custom UDF application

The Textile Defect Classifier UDF file includes a tester code snippet that allows the file to be tested outside of the Edge Insights for Industrial environment on individual images to aid in debugging. Executing the application outside of the Edge Insights for Industrial deployment environment, requires OpenVINO™ to be installed on the development system.

Please follow instructions on the OpenVINO official page for installation steps for different inference hardware (CPU/GPU/Myriad).

To execute the application on the host development system, follow the instructions:

# Install Python dependencies
pip3 install opencv-python==4.4.0.42
pip3 install tensorflow==1.15.0


# Set OpenVINO environment variables
source /opt/intel/openvino_2021/bin/setupvars.sh -pyver 3.6

Run inference on a single test image, where <version> indicates the downloaded version of Edge Insights for Industrial:

cd $WORKDIR/edge_insights_industrial/Edge_Insights_for_Industrial_<version>/textile_defect_classifier/textile_defect_classifier/


# Run inference on single test image
cd PyTextileClassificationIngestion/
python3 textile/textile_classifier.py \
                   --model ./textile/ref/model.xml \
                   --label ./textile/ref/labels.txt \
                   --image ./sample_data/frame_a.png

Expected Output

Run inference on a directory of test images to calculate accuracy information of the model, where <version> indicates the downloaded version of Edge Insights for Industrial:

cd $WORKDIR/edge_insights_industrial/Edge_Insights_for_Industrial_<version>/textile_defect_classifier/textile_defect_classifier/


# Run inference on a directory of test images
cd PyTextileClassificationIngestion/
python3 textile/textile_classifier.py \
                   --model ./textile/ref/model.xml \
                   --label ./textile/ref/labels.txt \
                   --dir ./sample_data/

Expected Output

Summary and Next Steps

In this tutorial, you successfully ran the Textile Defect Classifier application and displayed the result using the Edge Insights for Industrial Visualizer output.

To get access to the deep learning training algorithm that was used to generate the Textile Defect classification model, please reach out to your Intel account manager.

As a next step, see the Industrial Text Line Recognition tutorial.