Understanding the Agile Testing Framework
Table Of Contents
What is Testing?
Testing is the process of finding defects in a product/process to notify behaviour that isn’t acceptable for a user, to highlight how much of the product conforms to the expectations and what are the deviations. Testing ensures the quality of a product and forms a major component of the Software Development Life Cycle.
What is a Testing Framework?
A testing framework is a set of rules that define how to do the testing. It incorporates tools and practices that can make testing easier, effective and efficient. Broadly, it provides the pillars on which testing can be planned and executed.
What is an Agile Testing Framework?
Agile is all about embracing the fact that change is constant. Software products that follow Agile Methodology are always on the roller coaster ride of constantly changing requirements, technical stack, timelines, deliverables, etc.
For software developed using Agile methodology, small iterations of the product are built and evolved at regular intervals(called sprints). These small iterations of the product also need to be tested to ensure that quality issues are identified and resolved timely.
A framework that enables testing of these small iterations of the product makes for an Agile Testing Framework.
Does an Agile Testing Framework need to be agile itself?
Yes. As the product adapts to the changes getting introduced, the same needs to be done for its testing framework. The testing framework for an agile product needs to be constantly revisited to weed out obsolete tasks and practices and incorporate new ones. Also, not all components of the framework are needed from the start. You have to add and remove from the framework as per the need.
Also, you should start thinking about testing from the beginning. Not only in the Development Phase, testing should be incorporated in the Requirements and Design Phase also. More about this on ‘Is your automation testing tool ready for Shift Left testing?‘
The nature of testing performed in these phases could be a bit different. For example, in the Requirements Phase, testing could be related to business analysis.
In the Feasibility Analysis phase, testing could be related to legal and technical analysis. In the Design phase, testing could be related to user acceptance and security. In the Development phase, testing could be actual unit testing, smoke testing, acceptance testing, performance testing, stress testing, etc.
What makes a good Agile Testing Framework?
An Agile Testing Framework should be good enough to cover the breadth and depth of a product. Following are some of the features that are crucial for a good Agile Testing Framework:
Common vocabulary
The business and technical teams must share the same vocabulary for terms and concepts related to the product even if it means developing a few new words. Using a common vocabulary eases communication among different teams and even within the same team.
Test-driven development
Having a test case that defines the expected behaviour even before the behaviour is implemented helps in clear understanding. You precisely know what should be implemented and are not biased by the implementation details. As long as the test fails, you know the behaviour is not correct. As soon as the test passes, you know you are on the right track. Here the preciseness of the test becomes very important.
Identifying what all should be verified as a part of the test to ensure correct implementation of the behaviour is critical. At the same time, care needs to be taken to keep the test easily maintainable, loosely-coupled with the system and finite scope so that it doesn’t end up testing everything or nothing useful.
Different tests for different teams
Different teams on the product should have different testing criteria for the product. The way business teams test the product is and should be different from the way technical teams test the product. Business teams are better aware of the user scenarios and can better judge the product for user acceptance. Similarly, technical teams are better aware of the implementation details of the product and can better test the code quality through unit tests.
Domain experts
It is good to have domain experts, as needed, on the team. Domain experts, be it business experts or security experts can provide a different viewpoint for the product. The way business experts aren’t good with technical details, the technical team is not good with the business domain. Having business experts regularly review the product deliverables helps to bring in crucial feedback for the product that could be hard to incorporate towards the end of the product development.
Also, business experts are more likely to take the product iterations to customers and gather feedback that can prove crucial in shaping the product further. Similarly, security experts on the team can help close security loopholes in the product early on. Keeping security in mind from the start is important. Following security guidelines while designing the product can help in avoiding flaws that can prove fatal later.
Exploratory testing and Extreme Roleplays
Testing is more about mindset, less about a job. As you become familiar with a product, your testing skills start getting influenced by the knowledge of the product. You start making assumptions on how it should be used, which means that you won’t try to use the product differently. This leaves out the scope for testing the product with a different user’s perspective.
A user who is a novice (like Charlie Chaplin) may have a different approach for using the product than a user who is fond of pushing the product to extreme limits while using it (like Big Bunny who is known for breaking the products).
Users from different levels or background may approach and expect the product to behave in different ways. Performing such extreme role-plays during testing can help in identifying issues with the product that cannot be noticed otherwise.
Tests Organization
When you are dealing with a large number of test cases, that span across teams and domains, daily, it becomes crucial to organize them in a trackable manner. Agile Testing Matrix could be of help here as it not only provides a skeleton to organize the test cases but also becomes the guide to understand the purpose and objective of the test cases.
This is how a basic Agile Testing Matrix looks like:
When you try to fill in this matrix with your test cases classified as unit tests, functional tests, acceptance tests, security tests, etc. you tend to realize what purpose each test case is serving for the product and how strongly your test cases are covering the product quality.
With the help of this matrix you can eventually also see which areas of the product have the least focus from a quality perspective and that can give you guidance on where to channelize your testing efforts.
Sprint demos
Small iterations of the product built every sprint should be regularly demonstrated to the technical and business teams to check the gaps between what is expected and what is coming out. This helps in catching and correcting any differences in expectations and deliverables early on. Sprint demos also help in building confidence in the team for what they are building. It becomes easier to look at the overall picture when things translate from imagination to real-world and you start seeing some essential pieces of the structure. Sprint demos help achieve that.
User facing documentation
Although it can be argued that user-facing documentation is/should not be the responsibility of the technical team, the truth is that documentation plays an important role in user experience. Documentation is the go-to place for a user with doubts/queries. When you write and/or review the documentation from a user’s perspective who is not aware of the internal details of the product, you can often see the missing pieces and problems.
When you are making someone else understand, your understanding becomes better at what should be, why it should be and where it should be. This helps in identifying a different set of issues with the product, which could then be categorized as bugs or feature enhancements
Agile methodology helps in building products faster but a velocity without quality has no significance. Quality should be held as most important. To achieve velocity with quality, every team member should be concerned about quality. Be it design discussions or actual development of the product, every team member should wear the testing hat and notify issues early. The objective of a product is to serve quality and quality should be everyone’s responsibility.
Further Reading: Top 13 Best Practices For Agile Teams
References
