Dockerize your Deep Neural Networks (DNN) and use them in ROS1

• 🚀 Get rid of dependency (CUDA, PyTorch, cuDNN, ...) problems and install them in docker container
• 🚀 Simple ROS agnostic communication (IPC) between Docker container and ROS1 node
  • IPC based on Shared Memory (very low latency, 2ms)
  • IPC based on TCP socket (allows inference on remote machine with strong GPUs)
• 🚀 Automatic deployment: build, start and stop of Docker container

Install:

1. Install Docker
2. Install NVIDIA Container Toolkit
3. Install ROS1
4. Setup ROS workspace (if not already existing)
5. Navigate to src folder of your catkin workspace (or some sub-folder of your choice)
6. Get ROS GitHub Dependencies & this project:

# ROS message definitions for instance/semantic/panoptic segmentation & object detection
git clone https://github.com/UniBwTAS/object_instance_msgs.git

# RVIZ plugin to visualize above messages
git clone https://github.com/UniBwTAS/rviz_object_instance.git

# our ROS package
git clone https://github.com/UniBwTAS/containerize_inference.git

7. Get some other dependencies via rosdep:

sudo rosdep update
sudo rosdep install --from-paths . --ignore-src -r -y

8. Get dependencies for IPC:

sudo apt install python3-opencv
pip3 install posix-ipc

9. Build everything
10. Run node (with YOLOv8) + visualization:

roslaunch containerize_inference inference.launch

Note

The first time a new docker_image is used this takes some time as it has to be built first! In subsequent calls this will be much faster.

Available Docker Images (and Models) so far:

• mmdetection:
  • mask-rcnn_r50_fpn_1x_coco (Instance Segmentation)
  • mask2former_swin-t-p4-w7-224_8xb2-lsj-50e_coco-panoptic (Panoptic Segmentation, Tiny)
  • mask2former_swin-s-p4-w7-224_8xb2-lsj-50e_coco-panoptic (Panoptic Segmentation, Small)
  • mask2former_swin-l-p4-w12-384-in21k_16xb1-lsj-100e_coco-panoptic (Panoptic Segmentation, Large)
  • faster-rcnn_r50_fpn_1x_coco (Object Detection)
  • and more... (see GitHub project)
• mmdetection3d
  • (Not fully implemented yet)
• ultralytics (YOLOv8)
  • yolov8m.pt (Object Detection)
  • yolov8m-seg.pt (Instance Segmentation)
  • and more... (see GitHub project)

roslaunch containerize_inference inference.launch docker_image:=ultralytics neural_network_config:=yolov8m-seg.pt

Run inference on remote machine:

roslaunch containerize_inference inference.launch docker_host:="ssh://andi@192.168.1.4"

Note

Docker and NVIDIA Container Toolkit has to be installed on remote machine. Consider copying your public ssh key to the remote machine as an authorized key in order to avoid entering the password each time: ssh-copy-id andi@192.168.1.4