Automated UI testing is insanely hard to get right—and extremely easy to get wrong.
One pattern that can drastically improve test automation success is to keep it simple. But even today, people are constantly recreating test automation frameworks from scratch when usable frameworks already exist—some for at least 20 years.
The result: Testers find themselves struggling with basic framework development instead of creating the most stable automated tests possible.
Recreating the wheel can result in tests that don't work. In fact, one vendor's analysis of tests running in a cloud-based service found that fewer than 20% of us can execute a test that will pass more than 90% of the time.
Even scarier, the largest cohort had pass rates of between 50% and 75%. A 50% pass rate means you are doing no better than if you flipped a coin to decide whether your test automation will pass or fail.
Follow my test automation framework checklist and you'll never need to start from scratch again.
Create a base test class with 'setup' and 'teardown' hooks
Every automation framework should have a parent test class from which child test classes inherit some implementation details. At a minimum, your base test class should contain setup and teardown hooks (also called before and after hooks) for every test method. These hooks will allow your tests to create and destroy the resources you use for every test.
Furthermore, the base test will contain common methods and properties that occur in all your tests. One property you will absolutely need: the driver that manages your browser. Instantiate it in your setup hook, use it in the test, and discard the resource in your teardown hook.
I also recommend adding some logging to the setup hook. Any test data creation or page state management might go into this method as well. The setup hook must exist, since it's a standard approach to control the setup of every test.
A sample setup hook looks like this:
For its part, the teardown hook will kill your browser and automation session. If you're running in a cloud provider, the teardown method should also update any test status and third-party information that is relevant to the test. Examples could be updating the test status in a service such as JIRA, updating test status in an HTML report, and even deleting test data that you may have created in the setup method.
There are a few benefits to having setup and teardown hooks in your test automation. First, it is a common pattern that the majority of the testing industry follows. This means that anyone with a basic understanding of testing will understand these conventions.
Second, this functionality comes with the test runner and has been created by smart engineers for you to use. That means you can use those rich features to make your tests work as desired, rather than trying to reinvent the wheel with your own conventions.
Ensure your framework has a BasePage class
The BasePage reduces duplication and stores common components related to all of your page object classes in one place. Make sure that your BasePage class contains common components used by all of the page objects, not just a few of them. Otherwise, you'll be breaking the Liskov Substitution Principle.
What's more, having a bunch of random functions and properties in your BasePage class leads to confusion about which things belong in which class.
Here's a good example of a BasePage class:
Some items that will be required by most of your page objects are:
- A class that is used for synchronization of elements
- A JavaScriptExecutor (if you are using Selenium WebDriver)
- The property that stores the driver
There might be a few other components that need to exist based on your requirements. But it will not be much more than the above list.
The BasePage object example is a production-level class that's executed multiple times per day. Notice how small it actually is.
Generate test classes
For most testers, this checklist item will come as no surprise. Unfortunately, for a portion of the population, there are still frameworks that don't contain test classes. I'll get back to this in a moment.
A test class is one that contains some of your test methods. Typically, a test class is linked to a small feature in your application. For example, LoginFeature, LogoutFeature, and ShoppingCartFeature are some test classes that you might have.
You'll typically have a few of these, with each test class containing one or more test methods. Here's an example:
Some testers don't have test classes because they might be doing something such as keyword-driven testing. This is a relic of test automation that you should avoid at all costs.
Establish page objects
A page object is an abstraction layer of either entire HTML web pages or small components of HTML. Sadly, "page object" is a poor and confusing name, since it doesn't have to represent an entire HTML web page.
Ultimately, you want to use a single class to update your element locators and methods when the user interface changes. The only thing constant in software development is change, and the best way to fight change is to decrease the amount of code you must manage.
A good page object has three components:
- A descriptive class name that makes it clear which HTML component or page is being abstracted
- Element locators that let you interact with that HTML page through your automation code
- Methods that represent interactions that an end user would perform on your component
If you're missing the final two points above, then you don't have a page object. You have something different that's likely worse.
As always, keep it simple and avoid reinventing the excellent concept of page drivers, which has been around since 2004. Use page objects as defined here, and don't deviate. Automation engineers have much harder challenges to tackle, such as getting that pass rate up to 99.5%.
Execute tests in parallel
If your automated testing framework can't run in parallel, it will fail. The longer you work on test automation, the larger your suite will grow until, at some point, the automation suite will take too long to execute. There are several actions you can take to decrease suite execution time, but parallelization is the most effective.
In one test case, I used parallelization to execute a suite of 180 tests in under four minutes. Without parallelization, this suite was taking over 20 minutes to execute. Furthermore, the average test run time for the parallelized suite was approximately 1.76 seconds per test.
There is no other activity that will get you a 98% reduction in test case execution time. That's why parallelization is critical.
Make your job as an automation engineer as simple as possible by using the right tools to make parallelization easier. For example, tools such as Cucumber or SpecFlow make parallelization much harder than necessary because they parallelize at the feature level, while you want parallelization at the scenario or test level. (If you parallelize at the feature level, you can only run as many threads in parallel as you have features.)
Testing frameworks such as TestNG and XUnit also make parallelization harder. TestNG isn't thread-safe natively, so you have to do weird things such as using ThreadLocal and creating a thread-safe architecture on your own.
Just writing a sentence about thread-safety scares me. I would be waking up with night terrors if I had to implement it correctly.
Follow the patterns
The best test automation frameworks follow the same industry-wide patterns, and you can boil down these ideas to a checklist you can use.
When creating or updating a framework, make sure that your framework has page objects, a BasePage object, test classes, base test class, and setup and teardown, and that it can run in parallel. These are the minimum requirements that you must have in place to ensure successful test automation.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
