Configure the OpenVINO™ Model Server

On Machine A

1. Go to the AMR_containers folder:

   cd <edge_insights_for_amr_path>/Edge_Insights_for_Autonomous_Mobile_Robots_<version>/AMR_server_containers

2. Set up the necessary environment variables. Add the environment variables in /etc/environment:

   export DOCKER_BUILDKIT=1
   export COMPOSE_DOCKER_CLI_BUILD=1
   export DOCKER_HOSTNAME=$(hostname)
   export DOCKER_USER_ID=$(id -u)
   export DOCKER_GROUP_ID=$(id -g)
   export DOCKER_USER=$(whoami)
   export DISPLAY=localhost:0.0

3. Source the configuration file you have edited:

   source /etc/environment

4. Generate the keys used for server-client remote inference:

   cd 01_docker_sdk_env/artifacts/02_edge_server/edge_server_fleet_management/ovms/
   chmod +x generate_ovms_certs.sh
   sudo ./generate_ovms_certs.sh <host-name-of-server>

   NOTE:

   If Single-Node orchestration is deployed, the hostname of the Single-Node server shall be used to generate certificates. If Multi-Node orchestration is deployed, the hostname of the control plane shall be used to generate certificates.

5. Copy required client keys onto the client machine:

   # From server folder `keys/` transfer (scp) server.pem, client.pem and client.key to client:
   cd keys/
   scp -r client.key client.pem server.pem <client_user>@<client_hostname>:~/

On Machine B

1. Copy the keys to /etc/amr/ri-certs:

   mkdir -p /etc/amr/ri-certs
   sudo cp ~/{client.key,client.pem,server.pem} /etc/amr/ri-certs

Start the OpenVINO™ Model Server

On Machine A

If Intel® Smart Edge Open Multi-Node is deployed, there are two machines for orchestration. Machine A-1 represents the controller, and Machine A-2 represents the server node where the OpenVINO™ model server pod is deployed.

If Intel® Smart Edge Open Single-Node is deployed, Machine A-1 and A-2 are the same machine.

1. Run the following command to avoid an Intel® Smart Edge Open playbook known limitation:

   sed -i "s/edge-server-ubuntu2004-ovms-tls/edge-server-ovms-tls/g"  AMR_server_containers/01_docker_sdk_env/docker_orchestration/ansible-playbooks/02_edge_server/openvino_model_server/ovms_playbook_install.yaml

2. Go to <edge_insights_for_amr_path>/Edge_Insights_for_Autonomous_Mobile_Robots_<release>, and run on Machine A-1:

   ansible-playbook AMR_server_containers/01_docker_sdk_env/docker_orchestration/ansible-playbooks/02_edge_server/openvino_model_server/ovms_playbook_install.yaml

3. Verify that the services, pods, and deployment are running on Machine A-1:

   $ kubectl get all --output=wide --namespace ovms-tls
   
      NAME                                         READY   STATUS    RESTARTS      AGE   IP             NODE            NOMINATED NODE   READINESS GATES
      pod/ovms-deployment-75c7dffdc5-cgxp9         1/1     Running   1 (50m ago)   50m   10.245.202.9   glaic3roscube   <none>           <none>
   
      NAME                                         TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                         AGE   SELECTOR
      service/ovms-service                         NodePort   10.110.177.249   <none>        3335:32762/TCP,2225:32763/TCP   50m   app.kubernetes.io/instance=ovms-tls-abcxzy,app.kubernetes.io/name=ovms-tls
   
      NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                                      SELECTOR
      deployment.apps/ovms-deployment              1/1     1            1           50m   ovms-tls     10.237.23.153:30003/intel/ovms-tls:2022.2   app.kubernetes.io/instance=ovms-tls-abcxzy,app.kubernetes.io/name=ovms-tls
   
      NAME                                         DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES                                      SELECTOR
      replicaset.apps/ovms-deployment-75c7dffdc5   1         1         1       50m   ovms-tls     10.237.23.153:30003/intel/ovms-tls:2022.2   app.kubernetes.io/instance=ovms-tls-abcxzy,app.kubernetes.io/name=ovms-tls,pod-template-hash=75c7dffdc5

   NOTE:

   CLUSTER-IP is a virtual IP that is allocated by Kubernetes* to a service. It is the Kubernetes* internal IP. Two different pods can communicate using this IP.

4. Verify that the Docker* container is running on Machine A-2:

   docker ps | grep ovms-tls
   6b64514a4b9a   c9e7db04fe06                       "/usr/bin/dumb-init …"   10 minutes ago   Up 10 minutes             k8s_ovms-tls_ovms-deployment-5856948447-t78tz_ovms-tls_3cd84b35-c604-4948-9228-e381fd0714fa_1
   ceeaa7673bcd   k8s.gcr.io/pause:3.5               "/pause"                 10 minutes ago   Up 10 minutes             k8s_POD_ovms-deployment-5856948447-t78tz_ovms-tls_3cd84b35-c604-4948-9228-e381fd0714fa_1

If you encounter errors, see Troubleshooting for Robot Orchestration Tutorials.

Configure Object Detection with Remote Inference

On Machine B

1. Go to the AMR_containers folder:

   cd <edge_insights_for_amr_path>/Edge_Insights_for_Autonomous_Mobile_Robots_<version>/AMR_containers

2. Set up the environment:

   export DOCKER_BUILDKIT=1
   export COMPOSE_DOCKER_CLI_BUILD=1
   export DOCKER_HOSTNAME=$(hostname)
   export DOCKER_USER_ID=$(id -u)
   export DOCKER_GROUP_ID=$(id -g)
   export DOCKER_USER=$(whoami)

3. Make additional changes for remote inference:

   1. Put the <host-name-of-server> used in the key generation step above inside the config file:

      vim 05_tutorials/launch/remote_inference.launch.py

   2. Update remote_hostname:

      • For Intel® Smart Edge Open Multi-Node, use the control plane hostname.

      • For Intel® Smart Edge Open Single-Node, use ThingsBoard* server pod’s hostname.

      <remote_hostname>: 32762

Start Object Detection with Remote Inference

On Machine B

1. When running the client on a robot:

   CHOOSE_USER=root docker-compose -f 01_docker_sdk_env/docker_compose/05_tutorials/aaeon_wandering_remote_inference.yml up object-detection realsense aaeon-amr-interface ros-base-camera-tf collab-slam fastmapping nav2 wandering object-detection vda5050-ros2-bridge

   If a battery bridge is installed, run:

   CHOOSE_USER=root docker-compose -f 01_docker_sdk_env/docker_compose/05_tutorials/aaeon_wandering_remote_inference.yml up battery-bridge object-detection realsense aaeon-amr-interface ros-base-camera-tf collab-slam fastmapping nav2 wandering object-detection vda5050-ros2-bridge

2. When running the client on a laptop or PC with a Intel® RealSense™ camera connected:

   Before opening the Docker* images required to run object detection inference, make sure to set up your ROS_DOMAIN_ID in the common.yml file. See: Troubleshooting for Robot Tutorials, “Use ROS_DOMAIN_ID to Avoid Interference in ROS Messages”.

   CHOOSE_USER=root docker-compose -f 01_docker_sdk_env/docker_compose/05_tutorials/aaeon_wandering_remote_inference.yml up battery-bridge realsense ros-base-camera-tf object-detection vda5050-ros2-bridge

   Because this use case is not executed on a robot with a real battery, you can switch between local and remote inference by manually setting the battery percentage values. In order to be able to do this, you need to have installed the battery kernel module. See: Basic Fleet Management Use Case, “Install the battery-bridge kernel module” for details.

   After the battery-bridge kernel module is installed, you can adjust the battery percentage to different values.

   Open a new terminal window, and update the battery percentage to a value equal to or higher than 60%:

   echo 'capacity0 = 60' |  sudo tee /dev/battery_bridge

   After setting the battery percentage to 60, the inference in the object-detection terminal should continue to run locally.

   Set the battery percentage to a value below 60:

   echo 'capacity0 = 59' |  sudo tee /dev/battery_bridge

   After setting the battery percentage to 59, the object-detection should switch to run remote inference.

3. Example logs when Local Inference is performed:

   [object_detection_node-3]
   
   [object_detection_node-3] [ INFO ] <LocalInference> Done frame: 5516 . Processed in: 0.279625 ms
   
   [object_detection_node-3]
   
   [object_detection_node-3] [ INFO ] <LocalInference> Label tv
   
   [object_detection_node-3]
   
   [object_detection_node-3] [ INFO ] <LocalInference> Done frame: 5517 . Processed in: 0.240508 ms
   
   [object_detection_node-3]
   
   [object_detection_node-3] [ INFO ] <LocalInference> Label tv

4. Example logs when Remote Inference is performed:

   [object_detection_node-3] [INFO] [1643382428.696445729] [object_detection]: switchToRemoteInfCallback
   
   [object_detection_node-3] [INFO] [1643382428.720869717] [object_detection]: <RemoteInference> Sending Image
   
   [object_detection_node-3] [INFO] [1643382428.854300655] [object_detection]: <RemoteInference> Sending Image
   
   [remote_inference-4] [INFO] [1643382428.863697253] [remote_inference]: <RemoteInference> Receiving video frame
   
   [object_detection_node-3] [INFO] [1643382428.882912426] [object_detection]: <RemoteInference> Sending Image
   
   [remote_inference-4] [INFO] [1643382428.895332223] [remote_inference]: <RemoteInference> Processing and inference took 31.16
   
   [object_detection_node-3] [INFO] [1643382428.896419726] [object_detection]: <RemoteInference> Detected Objects Received
   
   [object_detection_node-3] [INFO] [1643382428.896478543] [object_detection]: <RemoteInference> Label : tv
   
   [remote_inference-4] [INFO] [1643382428.897817637] [remote_inference]: <RemoteInference> Receiving video frame
   
   [object_detection_node-3] [INFO] [1643382428.921090305] [object_detection]: <RemoteInference> Sending Image
   
   [remote_inference-4] [INFO] [1643382428.922211172] [remote_inference]: <RemoteInference> Processing and inference took 23.68

New Rules in the Basic Fleet Management Server

On Machine A

New rules in Root Chain are added to handle a battery level of less than 60% and equal to or greater than 60% scenarios.

Collaboration Diagram

When a robot’s battery level is less than 60%, basic fleet management tells the robots to do Remote Inference. When the battery level is back to equal or greater than 60%, basic fleet management tells the robots to do Local Inference. The following figure depicts the steps.