View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

Intel Open Sources a Kubernetes-Native Deep Learning Platform

Distributed deep learning using containers...

By CBR Staff Writer

UPDATED 12.15 with further details

Intel has released a new platform dubbed Nauta that lets data scientists and developers use Kubernetes and Docker to conduct distributed deep learning (DL) at scale.

Its client software, open sourced today under an Apache 2.0 licence, can run on Ubuntu 16.04, Red Hat 7.5, MacOS High Sierra and Windows 10.

It supports both batch and streaming inference, its GitHub repo shows.

(Batch inference involves processing a set of prepared input data to a referenced training model and writes the inference results to a folder. Streaming inference entails deploying a model on a system and processing singular data as it is received).

Intel Nauta: A “Production-Ready” Kubeflow

Hugely popular orchestration engine Kubernetes, originally developed by Google, makes it easier to build and deploy container-based applications.

It is, essentially, a way of efficiently running online software across a vast array of machines, stitching them together into “a big computer” and letting users oversee machines running on a range of cloud services, as well as inside private data centers.

Content from our partners
Green for go: Transforming trade in the UK
Manufacturers are switching to personalised customer experience amid fierce competition
How many ends in end-to-end service orchestration?

Intel’s Jason Knight, speaking at the company’s AI DevCon in Munich today, described Nauta as built on the back of Google’s Kubeflow tool (to which Intel is the third-largest code contributor) and was effectively a “production-ready” version of that tool.

(He also annouced the open sourcing of nGraph, an open-source C++ library and runtime / compiler suite for Deep Learning ecosystems.)

Nauta lets you use Kubernetes to manage end-to-end orchestration of ML pipelines, to run your workflow in multiple or hybrid environments (e.g. swapping between cloud and on-prem building blocks depending upon context), and to help you reuse building blocks across different workflows. The release offers greater flexibility to data scientists or developers wanting to carry out deep learning training experiments, without worrying about the pressure these will place on a given bit of infrastructure.

Training and deploying a deep neural network with Nauta

Intel Nauta: Deep Learning, Powered by Kubernetes

Nauta provides a “multi-user, distributed computing environment” for running DL model training experiments on Intel Xeon processor-based systems, using a command line interface, web UI and/or TensorBoard and powered by Kubeflow and Docker Intel said.

Developers can use existing data sets, proprietary data, or downloaded data from online sources, and create public or private folders to make collaboration among teams easier and use multi-node deep learning training experiments “without all the systems overhead and scripting needed with standard container environments.”

“Nauta is an enterprise-grade stack for teams who need to run Deep Learning workloads to train models that will be deployed in production”, Intel’s Carlos Morales said in a blog published to coincide with Intel’s AI DevCon in Munich today.

“With Nauta, users can define and schedule containerized deep learning experiments using Kubernetes on single or multiple worker nodes, and check the status and results of those experiments to further adjust and run additional experiments, or prepare the trained model for deployment” he wrote.

The release comes as a 2018 survey from Deloitte showed that nearly 50 percent of respondents had used DL, a 16 percent increase from the previous year.

While the business value continues to grow, and the interest in DL in the enterprise is palpable, it is still a “complex, risky, and time-consuming effort to integrate, validate, and optimize deep learning solutions” Intel noted.

Using Nauta, “at every level of abstraction, developers still have the opportunity to fall back to Kubernetes and use primitives directly. Nauta gives newcomers to Kubernetes the ability to experiment – while maintaining guard rails.”

 

 

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.
THANK YOU