Network security and the application of big data technology to explore the paper

Abstract: With the rapid development and popularization of Internet technology, nowadays Internet technology has been widely used in people's work and life, which brings unprecedented convenience, but at the same time, a variety of network security issues have also emerged. Based on this, this paper mainly introduces the specific application of big data technology in the field of network security, hoping that the research in network system security at the same time, can provide feasible theoretical reference for the sustainable development of the Internet business.

Keywords: network security; big data technology; application analysis

Foreword

With the recent years of the Internet technology continues to deepen, the network security accidents also occur frequently. Out of the importance of network information security, China established the National Security Council in 2014, officially upgraded network security as a national strategic deployment, which also indicates that the situation of network information security in China is not optimistic, and network attacks are in a high state. The number of Trojan horse zombie viruses, malicious ransomware, distributed denial-of-service attacks, theft of sensitive user information, and other types of cyberattacks are at the forefront of the world. The occurrence of mobile malicious programs, APT, DDOS, Trojan viruses and other network attacks will not only seriously impede the network bandwidth, reduce network speed, and also have a certain impact on the corporate reputation of telecommunications operators. According to a large amount of data, relying only on traditional network preventive measures has been unable to cope with the new generation of network threats, and through accurate detection and analysis and thus early warning, has become the key to the current stage of network security capabilities.

1 network security problem analysis

Network security issues not only involves the privacy and information security of citizens, but also related to national security, such as Yahoo's information leakage, resulting in at least 500 million pieces of user information was stolen; the United States Prism Gate and Hillary email door and other events have made network security issues further escalation, expansion. With the increasing complexity of the Internet architecture, the amount of data for network security analysis is also increasing day by day, and in the process of moving from terabyte to petabyte, not only is the source of data rich and the content more refined, but the dimensions required for data analysis are also more extensive. Accompanied by the growth of network performance at this stage, the data source sending rate faster, the higher the speed requirements for security information collection, version update delay and other vulnerabilities caused by Odav and other vulnerabilities are increasing, the scope of the impact of network attacks will be further expanded; for example, APT this kind of organized, targeted and long-term latent infiltration of multi-stage combination of attacks is more difficult to guard against, the only way is to analyze more kinds of security information and integrate a variety of means to detect and defend against the attack. The only way to prevent them is to analyze more kinds of security information and integrate multiple means of detection and defense. In the traditional technical architecture, most of the use of structured databases for data storage, but due to the high cost of data storage, the system will often standardize the original data processing and then stored, which can easily lead to data loss and distortion as well as difficult to save the historical data caused by the difficulty of tracking and tracing; and at the same time, for the execution of analysis of noisy, large, unstructured data sets and complex query is very inefficient At the same time, the efficiency of analyzing large, unstructured and noisy data sets and complex queries is very low, which makes it difficult to guarantee the real-time and accuracy of the data, and the efficiency of security operations is not high, so the traditional network security technology has been difficult to meet the new requirements of network security analysis at this stage. The concept of big data technology was originally developed by Victor Mayer. Myer. Sch?nberg and Kenneth Kukier published in 2008. Kukier in 2008 published the book "Big Data Era" proposed, big data refers to not using random analysis method, but all the data for comprehensive analysis and processing. As the first trend in the development of information architecture at this stage, big data technology, with its unique characteristics of high speed, diversity, variety and low value density, has been widely used in many fields of the Internet in recent years. The strategic significance of big data lies in the ability to master huge data information, so that the storage and analysis of massive raw security information can be realized, the storage cost of distributed database compared with traditional database can be reduced, and the data is easy to expand the level of inexpensive hardware, which greatly reduces the cost of security inputs; and accompanied by the substantial increase in data mining capabilities, the collection and detection of security information to respond more quickly, heterogeneous and massive data can be collected and analyzed in a more efficient manner, and the data can be collected and analyzed more quickly. The support of heterogeneous and massive data storage creates the basis for multi-dimensional and multi-stage correlation analysis, and improves the depth and breadth of analysis. For network security defense, through the integrated management, processing, analysis and optimization of data from different sources, it is possible to lock the target data in a massive amount of data at high speed and provide real-time feedback on the analysis results, which is crucial for network security defense at this stage.

2 Application of big data in network security

The use of big data in network security analysis can not only achieve data optimization and processing, but also comprehensive processing of logs and access behaviors, thus improving the efficiency of event processing. The effect of big data technology in network security analysis can be specifically analyzed from the following points:

2.1 Data collection efficiency

Big data technology can be distributed to collect data, and can achieve hundreds of megabytes/second collection speed, so that the data collection rate has been greatly improved, which also lays the foundation for the subsequent correlation analysis.

2.2 Data storage

In the network security analysis system, the storage of raw data is crucial, big data technology can be used for different data types for different data collection, but also to actively use different ways to improve the efficiency of data query, for example, in the query of the log information is suitable for the use of columnar storage, and for the analysis and processing of standardized data, it is suitable for the use of distribution of data. Standardized data, it is suitable for the use of distributed mode of pre-processing, after the data processing can be stored in the columnar storage results; or can also be established in the system MapReduce query module, in the query can be placed directly on the specified nodes, the completion of the processing and then sorted out the various nodes, so as to ensure that the speed of the query and the reaction speed.

2.3 Analysis of real-time data and subsequent data processing

In the analysis of real-time data, correlation analysis algorithms or CEP technology can be used to analyze, so that the data collection, analysis, processing of the integrated process, to achieve a higher speed and higher efficiency of the processing; and for the processing of statistical results and data, due to the timeliness of this processing requirements are not high. Requirements are not high, so you can use a variety of data processing techniques or the use of offline processing, so that you can better complete the analysis of system risks, attacks.

2.4 Analysis of complex data

In the analysis of different sources, different types of complex data, big data technology can better complete the analysis of data and query, and can effectively complete the complex data and security risks, malicious attacks and other aspects of the processing, when there is a malicious damage to the network system, attacks, can be used to big data technology from the perspective of the traffic, DNS, and the network, the data can be used to analyze and analyze the data, so that the data can be used to analyze and analyze the data.


Auto Sync with Cloud
No manual interaction required to synchronize with cloud services.

3 Based on big data technology to build network system security analysis

The introduction of big data technology in the network security system, mainly involves the following three modules:

3.1 Data Source Module

Network security system `data and data sources will be multiplied with the progress of Internet technology technology. Can be distributed collector in the form of hardware and software in the system for information collection, in addition to firewalls, detection systems and other software, the requirements of the equipment hardware is also increasing, such as the server, memory inspection and maintenance work.

3.2 Data Acquisition Module

Big data technology can analyze the data in pairs, so as to build a distributed data base, able to do the original data from the emergence of the deletion of a certain description of the real realization of the data access, retrospective function, especially for the amount of data is increasing day by day today, the distributed data storage can be better realized to improve the stability of the database.

3.

3.3 Data Analysis Module

For the operation of the network security system, the user's business system is the ultimate security object, big data analysis can be in the beginning of the user's data, timely analysis, feedback, so that network users can get a more personalized service experience. And for the user, to get what they want will also be more trusting of the network system and big data technology, for personal security and privacy information stored on the system of doubt will also be greatly reduced. Currently the field of network and information security is facing a new challenge, enterprises, organizations, individual users will produce a large amount of security data every day, the existing security analysis technology has been difficult to meet the needs of high-efficiency, accurate security analysis. And big data technology, flexible, massive, fast, low cost, high capacity and other unique network security analysis capabilities, has become the industry trend at this stage. For Internet companies, the realization of the depth of data "processing" is the key to achieving data value, which is essential for business operations.

4 Conclusion

In the current era, information data has penetrated into various industries and business fields, and has become an important factor in social production. Because of this, the number of Internet data generation is also multiplying day by day, which brings a certain degree of difficulty and pressure to the network security analysis work, and big data technology can be very good to improve this problem. The application of big data technology in the network system is not only able to meet the high efficiency and accuracy required for data processing, and can be built on this basis, a relatively perfect set of prevention and warning system, which plays a very key role in maintaining the safety of the network system, I believe that big data technology can be more widely used in the future.

References:

[1]Lu Wansheng. Introduction to the application of big data technology in network security analysis[J]. Digital World, 2017.

[2]Wang Shuai,Wang Laifu,Jin Huamin et al. Application of big data technology in network security analysis[J]. Telecommunication Science, 2015.

[3]Sun Yu. Introduction to the application of big data technology in network security analysis[J]. Network Security Technology and Application, 2017.