Continuous Testing – Everything You Need to Know

To know what Continuous Testing is, you should first see where development as a process came from. Earlier in the day, there was a Waterfall development style: a hierarchical structure, manual code builds, and a low tolerance for failure. Everything used to be step by step, in one direction, hence the name. Testing was something that used to happen near the end of the software development lifecycle. People designed, built code – you tested it right before it went into production. Performance testing was the last thing you did right before you went live.

The new culture is characterized by Agile, continuous delivery with a new, integrated DevOps pipeline and durable self-sufficient teams able to make decisions on every stage of the process. Everything that can be automated (and what makes sense being automated) is automated. You fail fast and innovate.

“Fail fast” became a buzzword. In the wrong context sometimes it is used as an epitome of volatile startup culture and Silicon Valley bubble that is supposedly bound to burst. The true meaning of fail fast is key to understanding what CT is all about.

You must find and eliminate everything that doesn’t work as fast as possible before you build anything on top of it. This way you don’t waste time, resources, and creative power on something not viable, something that is doomed right from the start.

How do you find what won’t work then? The answer is Continuous Testing. Continuous Testing is the backbone of DevOps approach to software development that allows early detection of software bugs and fixing the issues at earlier stages of the development lifecycle. Without it, the acceleration of the development process would lower the quality of the software instead of improving it.

Why Continuous?

Now, “continuous” is a flexible term and for different companies, it means different things, depending on their products and how fast they are. Google, Amazon, and Netflix integrate and test multiple times a day. For other companies, where software is not the end product but a means to an end, for example for a research paper writing service, testing within two-week sprints will be regular and “continuous” enough. The key is the automation of testing that can be launched from various places of the DevOps pipeline.

In continuous testing test scripts are written before the coding even begins (unlike the automated testing, where first the code is integrated, and only then test scripts are developed and run automatically). This way, the automated scripts can be run one after another continuously each time the code is integrated. You can schedule unmanned overnight testing procedures, and if something is mucked up in newest pieces of code, you are going to know about it the next day – not when everything is already in the production.

How You Do It?

Testing has been considered a bottleneck in the software development lifecycle – CT tackles this problem since there is no gap in the process where anything awaits human intervention.

However, for developers who only begin their transition from Waterfall to Agile, the major doubt is whether writing all the test scripts before coding begins is practical (or even possible). The answer is “Yes”, although it comes with its pitfalls and some counterintuitive stuff:

• Agile continuous delivery means, even more, planning for testing that it used to be in Waterfall delivery.
• With automation, failure is good! When something fails in the script and you know where exactly – you go just right there and figure it out, you don’t go through all the steps manually. You reap these failures.
• The number of test cases should not be large; they don’t help provide fast feedback that is expected. Less is more in this case. You need to prioritize test cases to get results fast.
• Automate the highest value (not every manual script). Figure out your critical functionality and your business priorities with your Product Owner and automate tests of that critical functionality.
• Use software for automation. You cannot use programmers for automation of all scripts – that’s not scalable.
• You may need different sets of automated suites for different purposes, environments, and execution times.

Also, it’s important to remember the little things:

• Come up with a concrete definition of “done” for every stage of development.
• Less documentation means no documentation at all.
• Test planning should be incorporated into backlog refinement.
• Test inventory should be kept current.
• Implement version control.

[the_ad id=”2867″]

DevOps is About People

If your company is going through the changes towards DevOps mode, the most important thing you should do is to integrate your testers (both manual testers and script automation) into your Scrum team to make organic knowledge sharing possible.

That’s the heart of the DevOps approach. Everyone in the team is on the same page in the process. No downtimes. No knowledge transfer problems. No “us” versus “them”. Developers and testers are on the same side, feedback is immediate, and ideas are shared on the spot. As a result, the functional and technical aspects of the software are integrated, which makes it:

1. Better
2. Faster to develop
3. Cheaper to develop

This was the initial goal of this whole Agile approach!

Way to go

Many teams are struggling to switch to Agile mode and Continuous Testing. Therefore many companies are somewhere in between, in a mix of DevOps and Waterfall. Some are starting their journey, some are somewhere in the middle. It’s a daunting task since it involves unlearning things and ditching practices that are familiar and have worked earlier.

The fastest way to get there is taking one step at a time. The goal is to make progress. Change cannot be pushed and forced; the shift of paradigm should be nurtured to induce organic lift. That’s exactly why many leaders in the industry stress that DevOps is about people rather than technology.

Since the pressure to get products out there as fast as possible will only grow in the foreseeable future, the importance of Continuous Testing will keep growing as well. One way it can evolve is integration with AI. Machine learning can be used in the future to come up with relevant test scenarios and to generate test scripts with less human involvement. AI can also be responsible for evaluating the critical areas that need to be tested, thus minimizing the manual intervention and the time it takes to build the automated testing framework for the project.