Machine learning inference framework in Autoware.auto
Description
As autoware develop towards more sophisticated ODDs, functionalities involving neural networks will inevitably get involved. Here are many tasks in an autonomous driving stack that can be readily achieved using neural networks. e.g.:
- Traffic light detection
- 2D/3D obstacle detection
- Lane Detection
- Drivable area segmentation
This need has lead to the proposal and the development of a Unified machine learning inference framework in Autoware using TVM at its core. An update on the progress of this work can be found here.
TVM has many attractive properties for Autoware
- Supports a wide range of input formats
- Supports a wide range of target devices
- Delivers good performance without vendor lock-in
- Completely open source and uses an apache license
However, TVM still replies on backend APIs in order to reach the hardware. Here is a partial list of what backend it supports which are not proprietary or vendor-specific:
- OpenCL (GPU)
- OpenGL (GPU)
- Vulkan (GPU)
- NNPack (CPU)
- BLAS (CPU)
- OpenMP (CPU)
- LLVM (CPU)
Most of these have open-source implementations and supports off the shelf hardware. However, the degree of completeness of the integration with TVM varies between these backends.
As autoware we want to offer a solution that is:
- Fully open-source
- Work out of the box for the majority of developer hardware
- Deliver reasonable performance
- Offers route to production
Hence some work needs to be done to assess and pick the backend supported in Autoware for future development.
What need to be done
-
Identify neural networks that need to be used in Autoware in the foreseeable future. - see https://github.com/autowarefoundation/modelzoo. The first one will be apollo segmentation detect
-
Identify a common set of developer hardware that require support out of box - x86 cpu, aarch64 cpu, nvidia graphics card, amd graphics card.
-
Make sure there is a route of porting for people using specialised hardware - Via TVM backend API
-
Discuss and decide on what backend support we should provide out of the box - LibVulkan
-
Provide performance data on different options(solution already agreed on and implemented) -
Implement backend support and maintain the integration - Planned for May
-
Deploy ML workloads in Autoware - Planned for May, apollo segmentation detect.