Current location - Loan Platform Complete Network - Big data management - What is the core of hadoop?
What is the core of hadoop?
Hadoop is an open source framework for storing and processing big data in a distributed way. The core components of Hadoop are-

HDFS(Hadoop Distributed File System)-HDFS is the basic storage system of Hadoop. Large data files running on commercial hardware clusters are stored in HDFS. Even if the hardware fails, it can store data in a reliable way.

Hadoop MapReduce-MapReduce is the Hadoop layer responsible for data processing. It has written an application program to deal with unstructured and structured data stored in HDFS. It is responsible for processing a large amount of data in parallel by dividing the data into independent tasks. This process completes Map and Reduce in two stages. Map is the first stage of specifying complex logic code, and Reduce is the second stage of specifying lightweight operations.

The processing framework in YARN-Hadoop is YARN. It is used for resource management and provides a variety of data processing engines, namely data science, real-time streaming and batch processing.