Revision History

New Features

Collaborative Visual SLAM

• Added support for the ORB feature extractor with C for Metal using the Intel® oneAPI Base Toolkit level zero interface

• Added support for stereo input in the EuRoC dataset

• Added support for both monocular and stereo input in the KITTI dataset

• Added re-localization mode

• Integrated FastMapping features: basic, loop closure, and map merge

• Updated the rviz2 configuration for FastMapping

• Reduced the tracker=to-server communication bandwidth in both mapping and localization mode

VDA Navigator

• This is the first release of the VDA navigator, a basic Navigation 2 based waypoint follower. This release includes support for updateRoute and cancelRoute functionalities.

VDA5050-ros2-bridge

• Added support for Order and Update Order messages

• Initial release of the AGV state machine feature

• Added a custom instant action for pausing and resuming the robot’s wandering

• Enabled the forwarding of the AGV state to the ThingsBoard* server

• Added custom Host and Port MQTT connections based on environment variables

Wandering AI Algorithm

• Added a pause-resume option for the robots

EI for AMR Containers (Robot and Edge Server)

• Added PCL and FLANN libraries to the Navigation 2 and rtabmap Docker* images

• Enabled the option to install either OpenVINO™ developer package or runtime package, reducing the size of the Docker* containers that include OpenVINO™

• Added support for offloading tasks to the GPU over a Intel® oneAPI Base Toolkit level zero interface when using C for Metal

• Included the ORB feature extractor in the Intel® oneAPI Base Toolkit Docker* image

• Created Docker* images for WWAN 5G modem and the VDA navigator

• Integrated Kudan visual SLAM with the ORB feature extractor into a Docker* image

• Added wandering, navigator, and common-dependencies playbooks

• Created a localization mode launch configuration for AAEON’s UP Xtreme i11 Robotic Kit

• Added secure file server support to download an additional file to EI for AMR during the onboarding process

• Included all EI for AMR Docker* images into a single Kubernetes* pod

• Added support for running a ROS 2 node on a Kubernetes* cluster

New Optimizations

Optimized ORB Feature Extractor for an Intel® GPU

• The ORB feature extractor, on version 1.0, is optimized for 12th Generation Intel® GPUs.

• The ORB feature extractor improves the frame processing speed compared to a CPU implementation and reduces CPU utilization.

• It includes CV kernels that are optimized for a GPU using C for Metal language to run on Intel® oneAPI Base Toolkit level-zero runtime: resize, gaussian, fast, compute descriptor, and compute orientation.

• The per frame latency can be adjusted via environment variable GPU_CHECK_SLEEP_TIME (the default is 50 µs).

PCL Optimization

• Added the march=native flag in Intel® oneAPI Base Toolkit CMakeLists.txt to fix an MLS segmentation fault

• Added octree approximate nearest search

• Changed namespace pcl::device for Intel® oneAPI Base Toolkit functions (For Intel® oneAPI Base Toolkit functions of octree and convex hull, use pcl::oneapi::device namespace instead of pcl::device.)

• Added octree approximate nearest search with the following modifications:

  • Included code converted with Intel® DPC++ Compatibility Tool

  • Removed improper Intel® DPC++ Compatibility Tool warnings

  • Used LOG_WARP_SIZE of four instead of five for better performance

  • Used sycl::range<1> instead of sycl::range<3> when submitting the kernel

• Added radius search

• Added the wait_for sycl::event API to reduce CPU utilization

• Replaced q.wait() with wait_for event for octree and convex hull

• Added octree knn_search with the following modifications:

  • Included code converted with Intel® DPC++ Compatibility Tool

  • Changed the sub-group size from 32 to 16 for better performance

  • Used wait_for(event) instead of queue wait

  • Supports k > 1

• Fixed Eigen v. 3.3 compilation issues

• Enabled support for end users to build Intel® oneAPI Base Toolkit independently

• Implemented point cloud passthrough and voxel grid filters

• Added script to deploy PCL binary as a standalone, outside of EI for AMR Docker* images

FLANN Optimization

• Added the wait_for sycl::event API to reduce CPU utilization

New Tools

• Added new tutorials for PCL optimization

Features for Evaluation

EI for AMR can be downloaded on ADL-P-based systems for evaluation purposes.

Requirements

These requirements are unique for this evaluation and are not reflected in the other EI for AMR documentation.

• Base OS: Ubuntu* 22.04 LTS

• EI for AMR 2022.3 robot kit SDKs: Product Download

• Documentation: Get Started Guide for Robots

Workarounds for Known Issues

• Intel® RealSense™ D435i Camera Low FPS Rate

  • Summary There can be a low FPS rate when a Intel® RealSense™ D435i camera is connected to a USB 3.2 port.

  • Impact A low FPS rate issue can be observed when a Intel® RealSense™ D435i camera is connected to a USB 3.2 port.

  • Workaround There is no workaround available.

• Intel® ESDQ Tests Do Not Support Ubuntu* 22.04 LTS

  • Summary The current version, version 9.0, of the Intel® ESDQ tests do not support Ubuntu* 22.04 LTS.

  • Impact The Intel® ESDQ tests cannot be executed on Ubuntu* 22.04 LTS.

  • Workaround There is no workaround available.

• VTune™ Profiler Fails to Collect GPU Hardware Metrics

  • Summary Errors arise when collecting GPU hardware metrics with the VTune™ Profiler.

  • Impact This issue is observed every time that the VTune™ Profiler is used to collect GPU hardware metrics for the execution of a given application.

  • Workaround

    1. Install the latest VTune™ Profiler version 2022.4.1 (build 624516).

    2. Download and compile the Intel® Metrics Discovery Application Programming Interface library.

    3. Copy the previously compiled libigdmd.so library into the newly installed VTune™ lib64/ folder, and rename it libmd.so.

• Docker* Docker_Community_Edition_CE Module Not Installed

  • Summary The Docker* installation stops at 58% and does not proceed.

  • Impact This issue is observed every time edgesoftware tries to install Docker*. The Docker* installation cannot be completed without human interference.

  • Workaround

    1. Start the edgesoftware install command, and wait until the Docker* installation proceeds to 58% complete.

    2. When the Docker* installation hangs at 58% complete and at two other times, press Enter each time to proceed.

       NOTE:

       In total, you have to press Enter three times for the Docker* installation to complete.

• Intel® RealSense™ SDK’s librealsense2 Package Compatibility Issue with the Device Onboarding End-to-End Use Case

  • Summary In Step 5 of Configure the Robot and the FDO Server for the Onboarding Flow, in the code snippet on line #11, the source repository is added as focal which is not compatible with jammy.

  • Impact This command does not work in Ubuntu 22.04, because focal is used instead of jammy. You may see this error:

    Package librealsense2-udev-rules is not available, but is referred to by another package.
    This may mean that the package is missing, has been obsoleted, or
    is only available from another sourceE: Unable to locate package librealsense2
    E: Unable to locate package librealsense2-utils
    E: Unable to locate package librealsense2-dev
    E: Unable to locate package librealsense2-gl
    E: Unable to locate package librealsense2-net
    E: Unable to locate package librealsense2-dbg
    E: Version '2.50.*' for 'librealsense2-udev-rules' was not found

  • Workaround

    1. Replace the command on line #11 with this:

       && sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u

    2. Replace line #14 through line #20 with this:

       librealsense2=2.*                                                     \
       librealsense2-utils=2.*                                               \
       librealsense2-dev=2.*                                                 \
       librealsense2-gl=2.*                                                  \
       librealsense2-net=2.*                                                 \
       librealsense2-dbg=2.*                                                 \
       librealsense2-udev-rules=2.*                                          \

• cgroup Version Incompatibility in Device Onboarding End-to-End Use Case

  • Summary Ubuntu* 22.04 LTS defaults to cgroup v2, but the amr-fleet-management container requires cgroup v1, the default for Ubuntu* 20.04 LTS.

  • Impact After robot onboarding, the amr-fleet-management container fails at startup.

  • Workaround Perform these steps after Step 6 of Configure the Robot and the FDO Server for the Onboarding Flow.

    1. Robot Change kernel command line to set systemd to use cgroupsv1:

       sudo sed -ie '/GRUB_CMDLINE_LINUX=/ s/"$/ systemd.unified_cgroup_hierarchy=0"/' /etc/default/grub
       sudo update-grub

    2. Robot Reboot the system:

       sudo reboot

Resolved Issues

PCL Optimization

• Fixed Eigen v. 3.3 compilation issues

Collaborative Visual SLAM

• Fixed the issue with memory consumption in localization mode when re-localizing

Wandering AI Algorithm

• Fixed indices in the loop when a robot is looking for the next cell to explore (When looking for next free cell to explore, the width and height of the map are changing during the search.)

Remote Inference

• Updated the copyright header in the source files

Object Detection Node

• Updated the copyright header in the source files

EI for AMR Containers (Robot and Edge Server)

• Minor fixes and cleanups on Docker* containers

• Updated dependencies in the collaborative visual SLAM Docker* file

• Enabled Intel® Smart Edge Open device registration

Known Issues and Limitations

• It currently takes up to 100 minutes to install the EI for AMR bundle. The time varies depending on the speed of your internet connection and system’s capabilities.

• The POTA implementation in the onboarding flow requires manual input for Product Name and Manufacturer for each type of robot added to the flow.

• Kudan visual SLAM with an ORB feature extractor on the GPU stops mapping using the robot_moving_15fps ROS 2 bag.

• The Inertial Measurement Unit (IMU) cannot be started on Intel® RealSense™.

• Due to the camera’s discontinuation, the Intel® RealSense™ LIDAR L515 camera is no longer supported by EI for AMR.

Intel® Atom® CPU

• The server setup deploys collaborative visual SLAM on Cogniteam’s Pengo robot without checking the type of CPU it has. (The Intel® Atom® CPU in Cogniteam’s Pengo robot is not supported by default.)

• The installed TensorFlow* version in EI for AMR contains Intel® Advanced Vector Extensions (Intel® AVX) instructions. These Intel® AVX instructions are not supported by Intel® Atom® CPUs like the CPU in Elkhart Lake platform. Any action, including the OpenVINO™ sample application, fails on a platform with an Intel® Atom® CPU. To be able to run TensorFlow* on an Intel® Atom® CPU, it must be re-compiled without the Intel® AVX instructions using the steps from: How to Build and Install the Latest TensorFlow* Without CUDA GPU.

• Gazebo* simulation does not work on the Intel® Atom® 3000 processor family like the Apollo Lake-based UP2 (UP Squared). Intel recommends creating the Gazebo* simulation environment on more powerful systems that have, for example, 11th generation Intel® Core™ or Intel® Xeon® Scalable processors.

AAEON’s UP Xtreme i11 Robotic Kit

• It has been observed that, after 20 minutes of run time, an AAEON* robot gets too close to obstacles and, because Intel® RealSense™ depth does not give information under 20 cm, the robot collides with obstacles.

• The AAEON’s UP Xtreme i11 Robotic Kit loses its orientation and redraws the walls in the test area multiple times. Due to this improper mapping, the robot cannot correctly identify the position of the obstacles and might collide with them.

Wandering AI Application

• RTAB_MAP is not best suited for indoor navigation. Therefore, some obstacles may not be detected with the highest accuracy, due to reflections, etc.

ADBSCAN

• ADBSCAN is configured to work with low resolutions (360). Using higher resolutions like 1440 makes ADBSCAN report inconsistent findings on each run.

Collaborative Visual SLAM

• For long runs, where the edge server accumulates more than 80,000 key frames, the shutdown process takes more time to exit cleanly.

• For visual odometry fusion with monocular input, after the visual tracking is lost, the system relies on odometry input (if enabled) to sustain tracking and is never able to switch back to visual tracking.

• The visual-inertial fusion is not supported in localization mode.

• Fusion of visual, inertial, and odometry data at the same time is not supported.

• Map merge does not happen if robots are moving in opposite directions through a common area.

• When both the server and tracker are launched, the robot looses tracking while rotating in place.

• The collaborative visual SLAM tracker crashes when running in localization mode with 30 FPS.

VDA5050-ros2-bridge

• VDA5050-ros2-bridge receives fewer MQTT generated messages than the defined number of messages.

Intel® ESDQ for EI for AMR

• The Intel® RealSense™ camera test fails if there is no Intel® RealSense™ camera attached to the target System, reporting this error message: [ERROR]: No RealSense devices were found.

• Object detection on Intel® Movidius™ Myriad™ X VPU fails if there is no Intel® Movidius™ Myriad™ X connected to the target system.

• If the internet connection is not stable or GitHub is blocked by a firewall, some Intel® ESDQ tests fail.

VTune™ Profiler in a Docker* Container Sample Application

• This sample application fails because of a compilation issue due to a missing header file in the source code of the application that is going to be analyzed with VTune™. Using a version of the sample application that matches the libraries available fixes the issue. Before Step 5 of the VTune™ Profiler in a Docker* Container, run the following command once:

  sed -i -e '26 a\' -e '       cd oneAPI-samples; git checkout tags/2021.4.0; cd ..'  01_docker_sdk_env/docker_compose/05_tutorials/vtune.tutorial.yml

Kudan Visual SLAM License File

The license file of the Kudan Visual SLAM component that is provided with the 2022.3 Release of the EI for AMR software expires at the end of January 2023. For this reason, the 2022.3.1 Release of the EI for AMR software includes the amr-kudan-slam:2022.3.1 Docker* image, whose license file has been extended to May 30, 2023.

Workarounds for Known Issues:

• The Docker* engine cannot find the amr-kudan-slam:2022.3.1 Docker* image.

  • Impact Because all other Docker* images are still tagged 2022.3, the script 01_docker_sdk_env/docker_compose/common/docker_compose.source sets the environment variable DOCKER_TAG to 2022.3. In consequence, the Docker* engine tries to load the amr-kudan-slam:2022.3 Docker* image, which does not exist.

  • Workaround Set the environment variable DOCKER_TAG manually before you run the Kudan Visual SLAM tutorials:

    export DOCKER_TAG=2022.3.1

• The Kudan license file outside the amr-kudan-slam Docker* image expires at the end of January 2023.

  • Impact The EI for AMR software includes a copy of the Kudan license file outside the amr-kudan-slam Docker* image for building your own Kudan Visual SLAM Docker* image. The 2022.3.1 Release of the EI for AMR software does not include the updated version of this license file. This file resides outside the Docker* image here:

    ls -la <edge_insights_for_amr_path>/Edge_Insights_for_Autonomous_Mobile_Robots_*/AMR_containers/01_docker_sdk_env/artifacts/01_amr/amr_kudan_slam/install/kdvisual_ros2/share/kdvisual_ros2/config/

  • Workaround To build your own Kudan Visual SLAM Docker* image, retrieve the updated license file (intel_amr.kdlicense) from the amr-kudan-slam:2022.3.1 Docker* image:

    id=$(docker create amr-kudan-slam:2022.3.1)
    docker cp $id:/home/eiforamr/ros2_ws/install/kdvisual_ros2/share/kdvisual_ros2/config/intel_amr.kdlicense - > temp.tar
    docker rm -v $id
    tar xvf temp.tar && rm temp.tar