1, from the database import

Before big data technology took off, relational databases (RDMS) were the main way to analyze and process data. The development of database technology has been quite well developed so far, and when big data emerged, the industry was considering whether it could apply database data processing methods to big data, and so Hive, Spark SQL, and other big data SQL products were born.

2, log import

Logging system will be our system to run every condition of information are recorded using text or logs of the way, this information we can understand as the business or the traces of the behavior of the device in the virtual world, through the logs of the business key indicators and equipment operation status information analysis. The logs are used to analyze information such as business key metrics and equipment operating status.

3, front-end buried

Why do we need to bury? Now the Internet company is increasingly concerned about the conversion, new, retention, rather than simple statistics PV, UV. these analytical data sources through the buried point of access, front-end buried point is divided into three kinds: manual buried point, visualization buried point, automated buried point.

4, crawler

To date, the crawler's data has become an important strategic resource for the company, through access to peer data and their own data to support the comparison, managers can make better decisions. And the harder it is to crawl to get competitors' data, the more valuable it is for the company.