YARN stands for Yet Another Resource Negotiator, and it belongs to Apache Hadoop technology.
To put it simply, it is a large-scale distributed operating system for big data applications.
How does YARN work?
The technology is designed for cluster management and is one of the key features in the second generation of Hadoop, the Apache Software Foundation’s open-source distributed processing framework.
YARN is a software rewrite that is capable of decoupling MapReduce’s resource management and scheduling capabilities from the data processing component.
In its structure, the processing component is separated from the resource management one. Basically, YARN helps the information stored in Hadoop Distributed File System (HDFS)to be run by different data processing engines. Some examples could be batch processing and stream processing.
YARN makes the proper usage of the available resources easier, which makes the handling of a large quantity of data easier.
What are the benefits of YARN?
The benefit of this is that it enables Hadoop to support more varied processing approaches and a broader array of applications. Such as Hadoop clusters now being able to run interactive querying and streaming data applications along with MapReduce batch jobs.
Combining central resource management with node manager agents that monitor the processing operations of individual cluster nodes has helped to increase the appeal of YARN and Hadoop.
The separation of HDFS from MapReduce with YARN has made Hadoop more suitable for operational applications that can’t wait for batch jobs to finish.
Read more: What is Java -cp?
