View all newsletters
Receive our newsletter - data, insights and analysis delivered to you

Intel open sources deep learning with BigDL for Apache Spark

It will allow developers to write deep learning applications as standard Spark programmes that run on top of existing Spark or Hadoop clusters.

By CBR Staff Writer

Intel has launched a deep learning library – the open-source BigDL for Apache Spark cluster-computing framework.

BigDL, which is already running in the Databricks Spark Platform, allows users to write their deep learning applications as standard Spark programmes that can directly run on top of existing Spark or Hadoop clusters.

It allows the exporting of artificial intelligence expertise to data scientists that currently work across several applications in various fields.

BigDL is modeled after Torch, an open source deep learning framework used in scientific computing.

It provides support for deep learning, including various computing through Tensor and high level neural networks.

By using BigDL, users can load pre-trained Caffe or Torch models into Spark programmes.

BigDL leverages Intel MKL and multi-threaded programming in every Spark task for deep learningachieving high performance.

Content from our partners
Unlocking growth through hybrid cloud: 5 key takeaways
How businesses can safeguard themselves on the cyber frontline
How hackers’ tactics are evolving in an increasingly complex landscape

The BigDL can also serve as the unified data analytics platform (Hadoop/Spark) for data storage, processing and mining, feature engineering, machine and deep learning workloads, Intel said.

Intel senior vice president and general manager of the Software and Services Group Doug Fisher said: “BigDL is an open-source project, and we encourage all developers to connect with us on the BigDL Github, sample the code and contribute to the project.”

The deep learning library is part of Intel’s AI strategy, which was unveiled in November 2016 and aims to drive breakthrough performance and democratise access.

READ NOW: Transforming your business with deep learning

The strategy outlined the company’s work to make AI training and tools accessible to developers via the Intel Nervana AI Academy.

Built for speed and ease of use, the Intel Nervana portfolio serves as the foundation for highly optimised AI solutions, allowing more data professionals to tackle the major challenges on industry standard technology.

The semiconductor company intends to deliver up to 100x reduction in the time to train a deep learning model, compared to GPU solutions.

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.