Oracle Announces Open Source GraphPipe – Dead Simple ML Model Serving via a Standard Protocol

by ·

Machine learning is expected to change the status quo of the industry. However, its application in the enterprise is slower than everyone expected, because it is difficult for these organizations to deploy and manage their machine learning technology. Part of the challenge is that machine learning models are often trained and deployed using custom techniques, making it challenging to implement models across servers or different departments.

To this end, Oracle hopes to transmit tensor data through open source and high-performance standard network protocols – a technical means to solve the above challenges. The new standard, called GraphPipe by Oracle, makes it easier for businesses to deploy and query machine learning models from any framework.

The official interpretation of GraphPipe is that this is a collection of protocols and software designed to simplify the deployment of machine learning models and separate them from framework-specific model implementations.

GraphPipe is designed to address three particular challenges:

  • First, the Model Service API has no standards, which means that business applications typically require custom clients to communicate with deployed models.
  • Next, building a model server is very difficult, and there are few deployment solutions out of the box.
  • Finally, solutions that companies typically use today, such as the python-JSON API, do not provide the performance required for business-critical applications.

GraphPipe includes

  • a set of flatbuffer definitions
  • A guide to defining a consistent model based on flatbuffer
  • Examples of models from various machine learning frameworks
  • Client library for querying models through GraphPipe

GraphPipe Features

  • based on Flatbuffers  micro-machine learning transmission specification
  • Efficient and straightforward reference model servers for  Tensorflow, Caffe2, and ONNX
  • Efficient client implementation of Go, Python, and Java

Using these tools, organizations should be able to deploy models across multiple servers or create collections of models from different frameworks using a standard protocol. GraphPipe can help implement machine learning for IoT applications that rely on remotely running models.

Tags: