Open source is good: open source standards are better.

By Peter McGuinness

The Khronos Group has launched its new interoperability standard for neural networks. The Neural Network Exchange Format (NNEF) is an open, implementation-independent way to describe neural networks designed to cut through the current tangle of framework-specific formats.

The new standard was released in provisional form in December of 2017 and, after a period of consultation with industry, is now ratified in its final form and available for immediate use. As well as the standard itself, Khronos is simultaneously releasing a suite of open source tools to allow developers to immediately begin using the format with the three most popular training frameworks: Tensorflow and Caffe/Caffe2. All of these tools are available on github in the Khronos repo.

NNEF is one of a group of standards being promoted as the “Khronos Standards for Machine Learning,” the others being OpenVX and SYCL (pronounced sickle).

Together with other Khronos standards, including Vulkan and OpenCL that provide an interface to the compute capability of GPUs, these three standards provide an ecosystem that can be used to link together a complete workflow for the training and deployment of neural networks and the vision applications that increasingly depend on them. SYCL provides an alternative to CUDA that will work across a range of systems, OpenVX enables hardware vendors to present a common vision API in embedded systems that can’t depend on a higher level OS-specific provision such as the Android NN API, and NNEF simplifies and standardizes the movement of neural networks from the training phase to deployment on any inference engine.

Khronos is moving quickly to make these standards work together as seamlessly as possible. While SYCL is a relatively general purpose programming language, the OpenVX API by its nature exposes a targeted set of functions and its heritage lies in the worlds of video streaming and what we now think of as ‘classic’ vision processing. A recently finalized neural network extension adds inferencing capability and at this year’s Embedded Vision Summit AMD showed an example implementation of a Caffe-to-OpenVX workflow that made use of NNEF as the transfer format. With NNEF in the flow, nets trained on any framework can be used as long as they can output to the NNEF format. Khronos has started the ball rolling with bidirectional translators for Caffe, Caffe 2 and both Tensorflow representations (python and protobuf).

We are strong supporters of open standards, seeing them as the bedrock of a vibrant industry, allowing competitor companies to jointly advance the technology. That view is not universal, however, and more than one company is making the case for rapid open source development based on proprietary initiatives, saying that the speed and responsiveness of open source teams is needed in the rapidly developing world of machine learning.

There is no doubt that Nvidia has done a great job with CUDA, almost single-handedly seeding the machine learning industry with the tools to accelerate neural network training. The downside is that CUDA runs only on Nvidia GPUs and the industry now needs to be bigger than one company so the emergence of SYCL is something to be welcomed. The same consideration applies to NNEF, which has a competitor in ONNX (Open Neural Network eXchange format). ONNX is a valid format but it is owned by a small clique of companies and the infrastructure of the machine learning industry needs to be more open than that.

The arguments in favor of open source standards are wider and much more profound than mere speed-of-development but it’s to Khronos’ credit that they recognize the tendency for standards to languish in committee while the outside world moves on. Their strong push for open source development, not only of the tools surrounding NNEF but the process of extending it, and putting everything on github where anyone is free to collaborate, is a masterstroke.

These standards are something the machine learning world badly needs and we couldn’t be happier that they are here.