New trends in the development of the software testing industry:

Artificial Intelligence AI+Testing

The use of AI (deep learning, etc.) systems to assist in testing is definitely one of the hottest testing trends of the last few years, which includes automatic generation of test cases, test data and test code, large-scale test results analysis, automated exploratory testing, defect localization, etc. Several companies in the U.S. have already launched commercial AI testing tools.

Although there are already many companies starting to research AI-assisted testing, and there are also many tools coming out, but they all have a big problem: the accuracy is not high enough. Due to some issues with the AI learning algorithms themselves right now, the test cases and validation conditions they learn and produce are not very accurate.

AI-assisted testing is divided into three steps:

1. Automatically produce the input of test cases through deep learning models and manually verify the output.

2. Automatically produce the input of test cases through the deep learning model and automatically accept the output through the rule model.

3. Automatically produce the input and output of test cases by deep learning models and automatically validate the output.

Now the industry can basically realize the first step, some companies have been able to achieve the second step, and for the third step, only a small number of large companies to achieve, and the accuracy is not very high, so AI-assisted testing still has a long way to go.

Two, the testing of new business and architecture systems

With the emergence and prosperity of AI systems, blockchain, microservices, and big data, special testing for them will also be a trend in the testing field. This will involve some new challenges, such as testing AI systems may use completely different methods and theoretical systems than before; how to simulate the real environment for testing when testing blockchain; micro-services in the case of getting larger scale, is it necessary to use different testing strategies and some special testing techniques than before; how to test big data systems in the test environment, simulate real large-scale data, and business testing and so on.

Test infrastructure

With the increase in the size of the software system, the construction of the test environment has become more and more complex, and its cost is getting higher and higher. And in many actual projects there are some specific needs for the test environment, such as before each round of testing, rollback of the last round of testing affects all the data; or can be quickly rolled back to a previous version of the system under test on multiple nodes in the cluster, and so on. With the increase of these needs and cost pressures, the construction of a modern and efficient testing infrastructure has become an inevitable trend for a large system. For example, you can use Docker, Ansible, etc. to build efficient testing infrastructure, etc., and then it has another new name: TestOps.

Four, testing in the product environment

Last year, different types of online failures occurred in both the AliCloud and Tencent Cloud, in which the AliCloud for their in June 27, 2018 online failure of the description wrote: "This function in the test environment verification did not occur in the problem, online to the automated operation and maintenance system, triggered an unknown code bug". This shows that for large-scale, high-complexity server systems, testing in the test environment alone can no longer meet the quality requirements, and how to test in the product environment will certainly occupy an important position in the current and future cloud era.

V. Failure Injection Testing (Chaos Engineering)

As cloud platforms become more and more large and complex, it is difficult to satisfy the demand for high availability with ordinary test cases, so FIT-Failure Injection Testing (FIT) is becoming more and more important. FIT-Failure Injection Testing (FIT-FIT) is becoming more and more important. Netflix even uses FIT on a large scale in its product environment, not just in the test environment. netflix has published several articles about Failure Injection Testing in its official blog. In an era of more and more complex systems in the future cloud, fault-based injection testing is surely essential for a quality-driven system.

Six, the security development process and automated security testing

Nowadays, when it comes to security, generally talk about penetration testing or security protection, such as WFA, etc., and very few people talk about security development and security testing in the development process. In fact, security development is the most effective approach to security protection, such as as early as the beginning of this century, Microsoft has put forward its own security development process SDL (Security Development Lifecycle), but due to Microsoft's SDL is very bulky and heavy, the cost is very high, resulting in its current need for agile and rapid development of software in the Internet era is difficult to promote.

BSI is a security development process that combines agile methodologies and practices, and requires embedding a variety of different types of security testing into the development process, such as security testing based on business functionality, white-box security testing based on threat modeling constructs, and security scanning and testing based on a variety of black-box automation, which is particularly suitable for agile teams that require continuous delivery.

Seven, testability analysis and design

Nowadays, very few companies will testability analysis and design of the system, and only some of the large companies of some of the large systems will use it. In fact, testability is a very old topic, just as contract testing was introduced back in the 1980s, but few companies use it due to its low ROI for small and medium-sized projects. Until the emergence and prevalence of scaled microservices, contract testing, an old favorite, has gradually become a new hot topic. But as the scale of software increases, and especially as larger systems such as cloud platforms become a trend, so will making their various tests more effective by improving their testability.

Eight, agile testing

Agile testing may be in many foreign companies (such as Atlassian, Netflix, Google, etc.) has been the norm

, but in the country, in the future for a long time is still a luxury. Among them, the concepts and practices of test-forward, test-driven development (ATDD at the business function level), and prevention of defects over detection of defects make agile testing effective in reducing defective code and rework, which effectively saves development costs and improves the speed of delivery and software quality, so the various practices in agile testing will surely become a trend in China as well.