2 December, 2020

TVM at Imagination

Imagination has developed a software development environment with the help of TVM infrastructure to enable deep learning inference on PowerVR GPU and Neural Network Accelerator (NNA)

Presenter: Ashutosh Parkhi, Senior Software Engineer at Imagination Technologies

Presentation: TVM at Imagination

Imagination has developed a software development environment with the help of TVM infrastructure to enable deep learning inference on PowerVR GPU and Neural Network Accelerator (NNA). Since the device drivers and compilers are proprietary to Imagination, TVM has helped us in connecting the deep learning frameworks to the Imagination software. Major challenges in meeting this objective were to support ever-increasing operator sets across a wide range of frameworks, optimal mapping of benchmarks onto underlying heterogeneous PowerVR platforms and optimized support for diverse data types. TVM infrastructure was leveraged to overcome them by developing equivalents of Tensorflow tools such as graph transforms, graph partitioning and quantization tools in NNVM and Relay. While the Caffe and TFLite front-ends were developed in-house, TVM saved us from the vagaries of TensorFlow and ONNX frameworks. We have also upstreamed internal changes done in the latter frameworks. Based on the different PowerVR device capabilities, a state-of-the-art automatic partitioning tool was developed to optimally split networks across different IPs. A robust infrastructure in TVM that offers a seamless connection between Python and C++ has enabled us to swiftly connect our software development tools to the code generators. Because of the lightweight TVM runtime, developing a pipeline scheduler came easy. Acceleration of throughput sensitive applications has shown the performance benefits of up to 30%. Needless to say that easy hooks provided via packed functions made writing a runtime communication interface across devices effortless. While many diverse tools were developed in-house, the active open source development with TVM’s foundation aligned to our taste has benefited us through several customer releases for the past two years.

Benefits of TVM

More Information