Why is big data so hot in recent years?

Mankind entered the era of big data, similar to biology ushered in the microscope, astronomy discovered the telescope, because the network transmission and computer storage computing power to improve, handed us a magnifying glass of information, and since then our observation of the phenomenon into a new field.

In fact, since ancient times, there have been multi-dimensional data mining behavior, the development of the calendar process may serve as a good example, the jianghu is now occasionally about the legend of Lin Yuanshuai Zhuge military master, since the birth of computer technology, the use of data and processing has been in synchronous development, whether it is the distribution of processing or parallel processing, not a day to pop up today's technological heights. Many of our scientific discoveries have been made in the last 30 years thanks to this.

But why has "big data" suddenly become so popular in the last few years? The reason is simple: the proliferation of smartphones around the world.

With the improvement of the information processing capability of mobile terminals, the interactive interface with the user is not only more viscous, but also realizes a full range of full-time interaction, at which time everyone's mobile terminal actually becomes a data logger. It is more personalized than the information that can be obtained by PC, not only exposing the details of the person's life, location trends, but also recording his consumption habits, and for the first time, human beings have so many data producers. Each metadata can be directly linked to a specific amount of expenditure, each figure can be monetized, the commercial value of big data and the revenue of each enterprise can be almost directly linked. So the storytelling around "big data" is rapidly becoming mainstream.

But behind the gimmicks of the media, you will find that the domestic research on complex systems is still in the conceptual rather than the application stage, most of the industry has no way to grasp the relationship between the data in the linear, closed system, not to mention the transformation of big data into valuable information. In the film and television industry, the industrialized system is in its infancy, and many practitioners can't even understand the basic data such as financial statements, and it's even more inconceivable to understand the value of big data.

The significance of big data in the domestic film and television industry

Big data technology as a tool, the direction of its application, there are only three aspects, one is the understanding of too much and once, the second is the cognition of the future and the future, the third is to judge the current and real-time processing, the film and television industry, if you want to have a long-term development of the application of big data technology, then in the three aspects will be faced with a number of problems to be solved. The application of big data technology in the film and television industry will face some problems in these three aspects if it wants to have a long development.

Understanding the past and the past

Since it is a judgment of what has happened, it involves data collection, and this part often leads to debates, with the central theme being: how big is big, GB or TB, PB or EB?

If I want to know why To Our Lost Youth, a movie that can be almost categorized as a literary film, has only grossed a few dollars at the box office in the first half of 2013, it would have been a big hit. why it was second only to Journey to the West - The Descending of the Demons at the box office in the first half of 2013, should I mine the data solely on the basis of the efficiency of social media distribution, or should I trace it back to the youth in the original novels and the memories evoked by the movie's publicity?

Putting the scope of data mining in the context of social media, then by combing through the promotion process of a movie, it is easy to work out a minute-by-minute reference and what kind of promotional content the audience will be attracted to, but it is still only describing the surface.

If the observation stops at the surface, it will be impossible to find the ultimate cause and effect. We must make an argument about the mechanisms that drive the phenomenon, so what volume should we use to store and analyze the memories of viewers in order to find the relationship between individual experience and collective ****ing?

In this respect, analyzing correlations using only social media data is not very different from, or even as flexible as, the emotional derivations we make every day, and it is easy to make "black swan" mistakes (before the discovery of Australia, the West thought there were only White swans). It is necessary to go back to the causal stage with larger external data, such as the consumer preferences and social experiences of the main audience groups over a ten-year period, as well as to statistically analyze their psychological activity prior to the viewing of the film. Would that be too complicated? But from a data mining perspective, only an effort in this direction is likely to provide substantial value.

Or we can keep it simple and rough, as Toni Chen, the head of Metro, said, the most important factor in the success of To Youth is because of Zhao Wei's dedication and commitment, as well as her personal accumulation in the industry.

You can think about which of these three perspectives would be closer to the heart of the matter.

Perceptions of the future and the future

Although big data technology allows human beings to probe deeper into the understanding of phenomena, when predicting the future of domestic film and television projects, the first problem to face is that we are still in a period of continuous fluctuations in the audience.

In the North American market, the audience that contributes 50% of the box office is about 10% of the population, or about 30 million people, and this segment is essentially fairly stable in structure. In the late 1970s, when the average production budget for a North American movie began to climb above $10 million and promotional expenses reached $5 million or more, the monitoring of the audience gradually shifted from stage-by-stage research to regular surveillance. In the period when computers are still a myth, "Hollywood" is the form of manual + letters, the establishment of the earliest large-scale audience research model, these historical data through the accumulation of several decades, has made a movie and the audience between the connection has become very transparent. However, even with such rigorous market monitoring, in recent years, because of the impact of mobile Internet, the audience went to the theater to watch a movie behavior randomness gradually increased, resulting in the traditional audience research model frequently appear some problems.

On the other hand, in the domestic movie market, almost two-thirds of the screens have appeared in the last three years, and in 2010, we had only 6,223 modern screens, while today, the number is almost 17,000. This behavior can not be called a habit.

So the current fluctuations in the Chinese movie market are difficult to monitor through existing technical means, because there are other variables that we don't know, which leads to the opposite result, which is called "variable omission bias" in statistical regression analysis, and the current scope of big data technology does not help us solve this problem. The current scope of big data technology does not help us solve this problem. We need time to revise our understanding of the market data, and the audience needs time to develop the habit of watching movies in theaters.

In the first half of 2013, almost all practitioners were overly optimistic about movies with action elements, and in the second half of the year, all practitioners, including me, were overly optimistic about movies with romance elements. From the big data monitoring of some companies, this change in audience consumption behavior has been fed into the data that can be captured, but we don't know how to quantify its impact. In other words, we can see the trend, but it is difficult to determine the results.

So, in today's Chinese movie market, it might be more relevant to shift the direction of big data technology from predicting the future to allowing us to avoid operational mistakes.

Judgement of the moment and real-time processing

Now on the understanding of big data, often entangled in the first word "big", while ignoring another important feature of its "fine", in fact, the latter is the most important, because it will create the real practicality of big data. In fact, the latter is the most important, because it will create the real practical value of big data.

Social media-based data mining, in fact, has been able to do so that we will be the classification of the audience from simple age, gender, occupation and other dimensions, the implementation of the region, activity space and personality traits and so on more rich in detail, based on such a basis, we have to do is how to give the audience to provide personalized impact, and no longer to the film for the position of the **** sexual publicity.

For example, when a male viewer sees a movie poster on a certain medium, it may be long legs and a small buttock, but when a female viewer is exposed to this medium at the same time, what they see may be a cute uncle. When a trailer for a Hollywood blockbuster is slapped in front of a promotional position, it may be used primarily to convey emotion or to highlight the funny, but the same minute or so of the trailer that goes live on a video site is split into several versions that correspond to the profiles of the users behind each click. In this way, the viewer joins in the production process, and the rapid processing of viewer preferences ultimately creates a message that is better suited for distribution.

At present, the data research company involved in the process of movie promotion, what it does is still only a statistical work, the decision is in the film or PR company, in fact, the decision-making mechanism can be the same with the data to refine, to become a real-time interaction, to reduce the loss of time, and to improve the efficiency of the film promotion. We used to promote the movie, often for how to take care of the majority of the audience's interest in the headache, then a different way of thinking, with the existing audience data for the segmentation of the group, to provide different information to different groups of viewers, the sea of hundreds of rivers than the radiance of the light may be more in line with the essence of the current social marketing.

But all of this is just an idealized vision, and the reality is that the Chinese film industry is still in an extremely primitive state.

Just from the perspective of movie investment costs, the average investment in a movie released today is less than $5 million, or less than 30 million RMB, which is equivalent to the level of the early 1970s in North America, without taking into account inflation or the income of the audience. In the face of such a market environment, many of the issues are actually rather empty, because although there is a great risk in making decisions, the cost is very low.