Compared to batch computing, the stream computing approach works well to analyze large-scale flow data in real time during the ever-changing movement, capturing potentially useful information and sending the results to the next computing node. At present, the computing method has been practiced and applied in Web applications, network monitoring, sensor monitoring and other fields. The big data security experts of ToppayShield have successfully applied the principles of stream computing technology to the field of big data security, creating a stream computing engine with superior computing performance, high concurrency, tens of thousands of throughput, and milliseconds return, which provides technical support for the financial industry to improve the management of risk control.

In the context of the Internet era, the highly fragmented data gathered by financial consumers, the scale of which is constantly increasing, financial institutions and enterprises can use these data for calculation, processing and judgment, to promote the formation of intelligent risk control in the financial industry. As a result, today's intelligent risk control focuses on big data, algorithms and computational capabilities, emphasizing the correlation between data.

The big data security team at Autopay continues to focus on the field of big data computing and storage technology, and has found that a high-performance computing engine based on massive data processing is crucial in the repeated research and exploration of big data security technology. In terms of timeliness, there is a significant lag, unable to meet the current demand for real-time financial big data computing.

Taking the financial wind control anti-fraud as an example, the deployment of the "flow computing engine" of the Tongfaidun risk monitoring and early warning platform can be based on massive risk data, real-time calculation of complex wind control models and strategies, and efficiently output the wind control results of the intelligent wind control system, the performance is significantly better than that of the traditional intelligent wind control platform.

To solve the mystery of the superior performance of the stream computing engine, we need to start from the technical principle of stream computing itself.

Streaming computing adheres to the basic idea that the value of data decreases with the passage of time, as in the case of user clickstreams. Therefore, events should be processed as soon as they appear, rather than cached for batch processing. In order to process streaming data in a timely manner, a low-latency, scalable, and highly reliable processing engine is required. For a streaming computing system, it should fulfill the following requirements:

1) High performance

2) Massive

3) Real-time

4) Distributed

5) Ease of use

6) Reliability

As compared to traditional static data, technicians utilize data mining and OLAP analytics tools to find the most important data for the business from the static data to find valuable information for the enterprise, the processing of streaming data corresponds to a different mode of computing: real-time computing.

Real-time computing is generally performed on massive data, and is generally required to be at the second level, while the streaming computing engine currently used by the big data security team of CommuniLink has reached the millisecond level of real-time computing capability for massive data, and the key technology lies in the streaming computing capability of the engine:

(1) Unlike the batch computing that slowly accumulates data, streaming computing spreads a large amount of data to each point in time, and continuously transmits it in small batches, with a continuous flow of data, which is discarded after computation.

(2) Batch computing maintains a table and implements various computational logic on the table. Streaming computing, on the contrary, you must first define the calculation logic, submitted to the churn computing system, this calculation job logic throughout the runtime is unchangeable.

(3) Calculation results, batch computing on all the data to calculate the results of the transmission, streaming computing each time a small batch of calculations, the results can be immediately delivered to the online system, to achieve real-time presentation.

Formally, compared with the traditional relational database collection of financial big data for risk early warning, the streaming computing technology of the Tongfaidun intelligent risk control early warning platform has the following major advantages:

1) pre-built massive risk control models, intelligent identification of business scenarios risk types, rapid matching and push risk control strategies to improve the efficiency of risk control;

2) support for real-time, quasi-real-time, offline, and other data management systems, including the use of the Internet and the Internet. Support for real-time, quasi-real-time, offline and other risk control modes, to achieve 10,000 throughput milliseconds response to protect real-time transactions;

3) real-time risk disk early warning, timely perception of the risk situation, the flexibility to adjust the risk control strategy, to improve the timeliness of the risk control;

Not only that, based on the "flow computing engine" of the risk monitoring and early warning platform of the Tongfaidun use of big data, artificial intelligence and other advanced technologies to effectively integrate anti-fraud technologies, and to improve the effectiveness of risk control and risk management. In addition, based on the "stream computing engine", the risk monitoring and early warning platform of Tongpaidun utilizes big data, artificial intelligence and other advanced technologies, and effectively integrates anti-fraud technologies such as device fingerprinting, terminal threat perception, data governance, situational awareness and other technologies to build intelligent risk control strategies for different scenarios, which not only meets the requirements of the regulatory agencies for risk prevention and control of financial businesses, but also meets the needs of financial enterprises to effectively carry out innovative businesses.

Currently, in the direction of big data development, machine learning is gradually developing from batch processing and offline learning to real-time processing, and real-time is turning into a trend, realizing the big data intelligent system of perception, analysis, judgment, decision-making, etc., which need the support of streaming big data real-time processing platform; in addition, streaming big data real-time processing can provide computing for big data-driven deep learning.

Streaming big data real-time processing can provide computing framework support for deep learning driven by big data.

Streaming computing in the content of the financial and scientific computing in the data for faster computing and analysis needs, will become the next generation of computing engines. The big data security team hopes to utilize streaming computing research and development results to create a truly intelligent risk monitoring and early warning platform to serve more enterprise customers.