Introduction

These installation and setup instructions are aimed for yolov5 deployment on a PCIe connected Hailo device. If you want to deploy a different network, check the [other networks](Other Networks) section.

Preemptive Note

The Hailo pipeline is constantly updating and changing, so it might not be ideal to follow these instructions as they are not updated regularly. I would recommend following the hailo tutorials from the hailo model zoo and go through the docs on hailo.ai. guide

Requirements

For deployment

access to hailo.ai developer zone
Hailo PCIe Device
Ubuntu 18.04 or 20.4

For taining

GPU enabled device (recommended)
hailo.ai developer account
docker
nvidia docker

Installation Tutorial

This is split into the 'deploy device' (which has your Hailo PCIe Device connected) and the 'training device' which you'll use for network training and network to hailo quantization. Skip this if you already have a .har file you would like to deploy.

For your (GPU enabled) training Device

You'll need a way to train your network (yielding a .pb file from it) and the hailo software suite to generate the Hailo File from that.

always download the newest version of the hailo software suite from the hailo.ai software download zone which are getting monthly updates at the time of writing. This of course means that these notes are rapidly aging into inaccuracy.
clone the hailo model zoo, which has some convenient yolov5 Docker containers.
unzip the hailo software suite and run inside ./hailo_sw_suite_docker_run.sh. If this is your fist time running, the container will set up. Otherwise, you'll have to either --resume or --override

For your Device with a Hailo Chip attached

downloads the newest hailo rt from the hailo.ai software download zone
extract the file and run the installer. Once you're done, reboot.
Source the virtual environment under /path/to/hailo_rt/Installer/hailo_platform_venv/bin/activate
test virtual environment by running hailo

Setup Example for a custom Yolov5m

Here you'll train, quantize, compile (on a gpu if possible) and infer (on the hailo chip)

Train your own Yolov5m

For yolov5 we'll be using hailo Docker containers, which are based on the ultralytics yolov5 containers
hailo model zoo now has a guide how to train yolov5 for hailo follow that one. Some notes:
- You'll need to create your own dataset structure for the training process. This guide explains well on how to create that.
- There's a minimal example dataset in this repository under /dataset
- To mount this, use eg.: docker run -it --gpus all -ipc=host -v /path/to/dataset/:/dataset yolov5:v0
- For training, make sure you target the correct --model and use the correct --weights (which are now conveniently already in the hailo docker)
- once you've saved the best.pb onnx file, you can exit this docker container
once you are done with the steps 'training and exporting to ONNX', move on to the next step.

Create Hailo representation (hef)

in hailo_sw_suite_docker_run.sh add a volume so you can access your best.pb onnx file, for example in the DOCKER_ARGS you could add -v /home/user/files:/files
run ./hailo_sw_suite_docker_run.sh --resume to get into the docker container
cd hailo_model_zoo and continue the tutorial.

Further Notes and Examples:

Another example to compile the model is to run: python hailo_model_zoo/main.py compile yolov5m --ckpt /files/my_dataset/best.onnx --calib-path /files/watt_dataset/dataset/dataset/images/ --yaml ./hailo_model_zoo/cfg/networks/yolov5m.yaml
note that here you're quantizing and compiling to HEF file. If you just want to quantize into a har file, run python hailo_model_zoo/main.py quantize yolov5m --ckpt /files/my_dataset/best.onnx --calib-path /files/my_dataset/dataset/dataset/images/ --yaml ./hailo_model_zoo/cfg/networks/yolov5m.yaml, which yields the .har representation of the file
- You have to provide a list of images for quantization (--calib-path)
- also provide a quantization scheme (--yaml), where hailo_model_zoo provides a variety
There's no real reason to not use compile directly from what I know.
Note that the file locations in these examples are different to Hailos. These worked, while Hailo's didn't.
follow the getting started guide from hailo model zoo / GETTING STARTED, specifically the optimizing and compile steps if you want more information on quantization parameters and how to go on from there.

run inference

this git repository includes a inference.py file, which which loads a specified .hef file. Change that to your .hef file location.
If you have anything other than yolov5m, you will have to change the pre/post-processing steps. Otherwise it won't work. Check out the Hailo Handler Class and go from there.

Other Networks

For yolov5m, Hailo provides a configuration yaml which defines the quantization levels for the various networks. If you have a custom network you will have to define your custom yaml file. I liked using netron to visualize the network for that.
Anything the hailo_model_zoo provides a configuration file for (under hailo_odel_zoo/cfg/networks/) can be easily trained and deployed using the process described above.
Also check out the Hailo Tappas which supply a variety of pre-trained .hef files, which you can run without having to compile anything.

6.2 KiB Raw Blame History