TVM at Imagination

Imagination Logo hori Blk 600x300

Join us at Apache TVM and Deep Learning Compilation Conference

Time: 3rd December 10:45 am PST

Presenter: Ashutosh Parkhi, Senior Software Engineer at Imagination Technologies

Presentation: TVM at Imagination

Imagination has developed a software development environment with the help of TVM infrastructure to enable deep learning inference on PowerVR GPU and Neural Network Accelerator (NNA). Since the device drivers and compilers are proprietary to Imagination, TVM has helped us in connecting the deep learning frameworks to the Imagination software. Major challenges in meeting this objective were to support ever-increasing operator sets across a wide range of frameworks, optimal mapping of benchmarks onto underlying heterogeneous PowerVR platforms and optimized support for diverse data types. TVM infrastructure was leveraged to overcome them by developing equivalents of Tensorflow tools such as graph transforms, graph partitioning and quantization tools in NNVM and Relay. While the Caffe and TFLite front-ends were developed in-house, TVM saved us from the vagaries of TensorFlow and ONNX frameworks. We have also upstreamed internal changes done in the latter frameworks. Based on the different PowerVR device capabilities, a state-of-the-art automatic partitioning tool was developed to optimally split networks across different IPs. A robust infrastructure in TVM that offers a seamless connection between Python and C++ has enabled us to swiftly connect our software development tools to the code generators. Because of the lightweight TVM runtime, developing a pipeline scheduler came easy. Acceleration of throughput sensitive applications has shown the performance benefits of up to 30%. Needless to say that easy hooks provided via packed functions made writing a runtime communication interface across devices effortless. While many diverse tools were developed in-house, the active open source development with TVM’s foundation aligned to our taste has benefited us through several customer releases for the past two years.

Benefits of TVM

  • Open source under active development with its foundation aligned to our taste that has supported us through the development of 4 NDK releases through past 2 years
  • Easy mapping of all the popular frameworks into a single intermediate representation protecting us from vagaries of the frameworks
  • Solid infrastructure for C++/Python interactions: Facilitates quick NDK tool development in Python while as a standard the compiler/codegen can be coded in C++
  • Enabled engineering to develop state-of-art heterogeneous compiler and runtime without requiring anything outside TVM infrastructure

To register or to learn more about the conference agenda, please visit their website here