View all newsletters
Receive our newsletter - data, insights and analysis delivered to you
  1. Technology
  2. Data
February 14, 2017

Yahoo & Microsoft open source data analytics tools for Spark & Graph Engine

More open source frameworks to tackle deep learning and graph databases.

By James Nunns

Yahoo and Microsoft have open sourced two data analytics tools that can help businesses to become more data-driven.

Yahoo, has decided to open source the TensorFlowOnSpark software that was created to make Google’s TensorFlow open source framework compatible with the data sets that sit inside Spark clusters.

To grossly simplify this, TensorFlow is an open source software library that users can tap in to for numerical computation using data flow graphs.

The company said: “TensorFlowOnSpark brings scalable deep learning to Apache Hadoop and Apache Spark clusters. By combining salient features from deep learning framework TensorFlow and big-data frameworks Apache Spark and Apache Hadoop, TensorFlowOnSpark enables distributed deep learning on a cluster of GPU and CPU servers.”

It’s easy to forget that Yahoo has some brilliant technical minds working at the company, what will all the issues regarding the data breaches at the Internet side of the business, but the company has history in the big data world.

The company is credited with being a founding father of Hadoop and last year it open sourced CaffeOnSpark, an open source framework that allows for distributed deep learning and big data processes on Spark and Hadoop.

Content from our partners
Scan and deliver
GenAI cybersecurity: "A super-human analyst, with a brain the size of a planet."
Cloud, AI, and cyber security – highlights from DTX Manchester
Read More: Is Google crashing the Microsoft open source party?

On other graph related news, Microsoft has decided to open source its Graph Engine, an in-memory data processing engine that the company says is: “underpinned by a strongly-typed-in-memory-key value store and general distributed computation engine.”

Both the Yahoo and the Microsoft releases now appear on GitHub, with the Microsoft repository also containing its graph query language called Language Integrated Knowledge Query. LIKQ is a graph query language on top of the graph engine, which is designed to combine graph exploration and lambda expression.

Graph databases have recently been proving themselves to be popular options for developers in the big data world.

Emil Eifrem, CEO of Neo Technology, makers of Neo4j, told CBR in a recent interview that interest in the area was because it’s: “such a natural way at looking at data – anyone doing maths knows how to look at a graph.”

Websites in our network
Select and enter your corporate email address Tech Monitor's research, insight and analysis examines the frontiers of digital transformation to help tech leaders navigate the future. Our Changelog newsletter delivers our best work to your inbox every week.
  • CIO
  • CTO
  • CISO
  • CSO
  • CFO
  • CDO
  • CEO
  • Architect Founder
  • MD
  • Director
  • Manager
  • Other
Visit our privacy policy for more information about our services, how Progressive Media Investments may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications. Our services are intended for corporate subscribers and you warrant that the email address submitted is your corporate email address.