Shuo Wen Jie Zi Lesson 2: Data Mining

After an initial understanding of big data, we will then look at how to use big data information to create value, we might as well imagine big data as a mine, how to dig out valuable gold from the mine, is the focus of data mining.

Finding the gold in data

Data mining is translated from Data Mining, which is a concept that can't be fully interpreted in both Chinese and English, because data mining is not just about finding the data itself, but also about analyzing the data from a large amount of data through automated computer analysis and identifying the potential information that has not yet been uncovered, and using it as a reference for decision-making. It is also used as a reference for decision making. If we use gasoline as an analogy, data mining involves searching for an oil field (finding the source of the data), extracting the oil (collecting the data), refining the oil (finding the potential information in the data), and using the gasoline as fuel (using the potential information to create value).

As a simple example, the recommended products on a shopping site are consistent with this concept. When a consumer buys a product on the site, the system can record all the items purchased and analyze them as consumption habits, such as how often and how much of a particular product is purchased, the color preference of the clothing, and the type of music that the consumer likes, etc. This way, the system can be used to analyze the consumer's behavior. In this way, the shopping site can use the analyzed data to estimate the time of the next purchase of the user's daily necessities, and automatically send out advertisement emails when it is close to the time, or automatically notify the consumers when there is a promotion of the product they may be interested in, so as to increase the willingness of the consumers to buy it.

▲Amazon is one of the world's largest shopping sites, and its recommendation feature is the result of data mining.

As computers, mobile devices, and Internet of Things (IoT) devices become more and more prevalent in our lives, the way we look for and collect data is changing dramatically, and we may not even realize it's being collected at all. For example, when we use the smart phone's navigation function, the system will be able to use the phone's built-in motion sensors to calculate the speed of each section of the road to estimate the size of the traffic flow, and with the time and date of the day for analysis, so you may be able to get to work on weekdays to go to the highway will be congested, or every Saturday in the city of which sections of the road is particularly congested and so on, road conditions, and so on.

Multiple ways to utilize

Sometimes the way data is collected and used can be unexpected. For example, if you make a typo in a search engine query, the system displays the results of the search for the corrected keyword, and when the user clicks on any of the results, the system knows that the corrected result is the correct one. If you click on any of the results, the system will know that the corrected result is correct, and if the user chooses to search for the original keyword, the system will also know that the word that was originally recognized as an error is the "real word".

▲Taking Google search as an example, when a user enters a keyword that may be incorrect, the system will automatically correct it.

The first step in the process is to make sure that you have the right tools for the job, so that you have the right tools for the job.

However, human beings are born with the mind is a good thing, through logical judgment, empirical analogies and other ways to analyze the causal relationship of events, and data mining, to put it bluntly, just through the statistics, analyze a very large amount of information to find out the highly correlated events, so in the process of data mining, do not correlation override causation, or else you may be able to make a lot of inverted results for the cause, The first thing you need to do is to make sure that you have a good understanding of what you are doing.

Example of how to use Saying Wenzheng Jiezi: data mining

O: Data mining requires creative thinking to uncover latent data and work with it flexibly.

X: On Sunday, we're going to take a shovel up to the mountains to do some data mining.

(Cover image via Flickr, this image is licensed under a Creative Commons CC name tag - do not modify for sharing, by Mathematical Association of America)