Current location - Loan Platform Complete Network - Big data management - Big data need to learn what frameworks, what ecosystem?
Big data need to learn what frameworks, what ecosystem?

You should be talking about the mainstream frameworks in the big data platform, I list:

(a) Hadoop ecosystem

HDFS: Distributed file system, to solve the storage of big data

Yarn (MapReduce): Distributed computing framework, to solve the computation of big data

Hive: Data analytics engine in Hadoop with SQL support

HBase: HDFS-based NoSQL database

ZooKeeper: Distributed orchestration service that can be used to implement HA (High Availability Architecture)

Others

(ii) Spark ecosystem

Spark Core: the core of Spark, used for offline computing

Spark SQL: Spark's data analytics engine, supporting SQL statements

Spark Streaming: Spark's streaming computation engine, but still essentially offline computation

MLlib: a machine learning framework

(iii) Flink ecosystem

Flink DataSet: Flink batch processing (offline computing) API

Flink DataStream: Flink streaming processing (real-time computing) API

Flink Table&SQL: Flink's data analytics engine, supporting SQL statements

MLlib: a machine learning framework