Currently, more and more companies are using big data analytics as the basis for their judgment of future development. At the same time, the traditional logic of business prediction is increasingly being replaced by new big data predictions. However, we need to carefully manage everyone's expectations of big data, because massive data can only further develop its business value if it is effectively governed.
The most widely recognized definition of Big Data is the 3V characteristic of Big Data given by Gartner: huge volume of data (Volume), fast processing of data (Velocity), and varied data structures and types (Variety). According to this definition, the first thing that comes to mind is the unstructured data that has been difficult to deal with in IT systems but cannot be ignored. In other words, Big Data not only deals with the analysis of transactional data, but also incorporates information from social media, e-commerce, and decision support. Now, the distributed processing technologies Hadoop and NoSQL have been able to store, process, analyze, and mine unstructured data, but have failed to provide a comprehensive solution to meet customers' big data needs.
In fact, the scope of Big Data in a general sense is much broader, and any complex calculations involving massive amounts of data and multiple data sources fall within the scope of Big Data and are not limited to unstructured data. Therefore, such as telecom operators have a huge number of users of all kinds of detailed data, cell phone switching information, cell phone in the network registration information, cell phone call billing information, cell phone Internet detailed log information, user roaming information, user subscription service information and user basic service information, etc., can be categorized as big data.
Compared to cloud computing, which emerged a few years ago, big data may have a longer road to travel to realize its business value. But business users can't wait, and more and more executives tend to use big data analytics as an important basis for their business decisions. In this context, we must find a comprehensive big data solution, not only to solve the problem of unstructured data processing, but also to extend the function to the storage of massive data, distributed collection and exchange of big data, real-time fast access to massive data, statistical analysis and mining and business intelligence analysis.
A typical big data solution should be a platform solution with multiple capabilities, which include storage, computation, analysis, and mining of structured data, storage, processing, and handling of multi-structured data, as well as business intelligence analysis of big data. This solution should have the following four characteristics in the technology: software and hardware integration of big data processing, the ability to fully structured data processing, the ability to large-scale memory computing, and ultra-high network speed access.