Nowadays almost everyone is talking about Big Data . So, do you know about Big Data? Do you know what big data includes?

Processing

Decision makers are interested in tightening the lifeblood of their organizations and achieving real-time results. They need structures that can handle the flow of data as it happens, and current database technologies are not suited to data flow processing.

For example, calculating the average of a set of data can be accomplished with traditional scripts. However, there are more efficient algorithms for calculating the average of moving data, whether it's arrivals, growth, or units. If you want to build a data warehouse and perform arbitrary data analysis and statistics, the open source product R or a commercial product similar to SAS can be realized. However, want to produce is a data flow statistics set , gradually add or delete data blocks , perform moving average calculations , the database does not exist or is not mature .

The ecosystem around data streaming is underdeveloped. In other words, if you are negotiating with a vendor for a Big Data project, you must know if data flow processing is important to your project and if the vendor is capable of delivering it.

Parallelization

There are many definitions of Big Data, and the following are relatively useful. Small data is similar to a desktop environment with disk storage capacity between 1GB and 10GB, medium data is between 100GB and 1TB, and big data is distributed storage across multiple machines and includes 1TB to multiple petabytes of data.

Distributed processing is required if you are working in a distributed data environment where data is processed in a short period of time.

Parallel processing stands out in distributed data, and Hadoop is a widely known example of distributed/parallel processing.Hadoop contains large distributed file systems that support distributed/parallel queries.