4 Most Common Myths Around Test Automation

A common theme among most software testing organizations is their escalating interest in Test Automation. While test automation has grown in popularity, there are still many myths and biases surrounding it. Such myths can unknowingly create a self-limiting boundary and negatively impact the possibilities of test automation.

In this article, I will discuss some common test automation myths and biases that people have towards test automation.

Note: “!=” operator has been used throughout this article to denote inequality. If you are not familiar with this operator, please read it as “not equal to”. 

Test Automation != Test Execution

Automated testing is often viewed as executing tests. However, this is one of the most common misconceptions. Automation is defined by Wikipedia as a wide range of technologies that reduce human involvement in processes. It is often the execution of tests that consumes the largest portion of the overall testing phase. Automating this phase can certainly save a great deal of time. 

However, Test Automation goes beyond Test Execution. Test Automation covers a broad range of possibilities. At each step of the testing lifecycle, there are a number of activities where test automation can reduce human effort. Here are some examples:

• Test Environment Setup:
  • Installing libraries using scripts
  • Setting configuration variables, etc.
    
    
• Mocking Component Behavior
  • Useful in IoT & Embedded Software Testing
  • Can also reduce (or eliminate) actual hardware/component need
    
    
• Test Reporting
  • Generating summary report/email.
  • Linking screenshots/logs to the reports.
  • Triggering emails on test execution, completion, etc.
    
    
• Rerunning failures
  • It is helpful to automate the re-execution of failure/flaky tests.
  • It also helps to examine the consistency and probability of the failure.
    
    
• Test Data Preparation
  • Creating synthetic test data
  • Creating dynamic & random test data
    
    
• Scheduling Test Runs
  • Based on schedules
  • Based on continuous integration rules
    
    
• Test Build Generation
  • Preparation of release candidates for testing
    
    
• Updating results to ALM Tools
  • Ex: JIRA, Team Foundation Server, Polarion
  • Using API endpoints
    
    
• Monitoring Production Logs
  
  
• Comparing Files
  
• Cleanup of Environment, etc.

“Just because you’ve counted all the trees doesn’t mean you’ve seen the forest.”

Anonymous

Test Automation != Quality Assurance

Over the years, Testing has been mistaken for quality assurance. A similar outlook is shared by Test Automation, which is often regarded as a quality assurance activity. Let’s look at the definition of quality to see how test automation isn’t quality assurance.

• One of the popular definitions of Quality is that it is “value” to someone “who” matters. Depending on context and personas, “value” and “who” take on different forms. Thus, quality is a multidimensional concept.



• By performing checks through tools, Test Automation will help you find information. Having such checks will enable you to find regression issues.



• With Test Automation, you can define checks to detect:
  • Functional Issues
  • Performance Issues
  • Security Issues
  • Accessibility Issues
  • UI, Responsive Issues, etc.
  
  
  
• Quality will, however, mean different things to different people. Below are some typical software user personas:
  • Marketing Team
  • Tech Writers
  • Testers
  • Programmers
  • Managers
  • Customer Service
  • Business Specialists
  • Support/Sales
  
  
• Different notions of quality will apply to your context according to the persona of your customer. Here are some of the possible quality attributes: 
  • Reliability
  • Speed
  • Usability
  • Scalability
  • Maintainability
  • Testability
  • Documentability
  • Trainability
  • Functionality
  • Supportability
  
  
• While you may be able to control some of these attributes, the majority are beyond your control or scope as a test automation engineer.

“Quality is not what happens when what you do matches your intentions. It is what happens when what you do matches your customers’ expectations.”

Guaspari

Test Automation != Test Scripts

• Newbies in test automation often focus heavily on the writing of test scripts.


• Even though Test Scripts are one of the visible outcomes of test automation, there is more to it than that.


• Automation engineers perform several activities that are often overlooked. They include:
  • Modeling application
  • Test data generation
  • Developing framework / reusable libraries
  • Work towards CI-CD-CT
    • CI = Continuous integration
    • CD = Continuous delivery/deployment
    • CT = Continuous testing
    
    
  • Documenting automation / Preparing readme documents.
  • Logging automation
  • Test design
  • Defining test execution strategy
  • Developing accelerator tools
  • Code review
  • Version controlling & tagging
  • Configuring / Parameterizing
  
  

  “Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”

  Martin Fowler

  Test Automation != Coding

  

  What programming language do I need to learn to be successful in test automation?”. On a daily basis, I hear this question from people. Yes, programming has historically been at the heart of automation. It is no longer the only way to automate.

  • With the rise of no-code test automation tools, the obvious equation of automation = programming has changed.
  
  
  • Today, You can automate test execution using coded tools as well as no-code tools.
  
  
  • Automating reduces human intervention, and there are various ways and tools to achieve it without or with coding.
  
  
  • If you are new to no-code tools, you can check out Testsigma which is a popular open-source no-code tool Testsigma. Here is the link to the open-source version of Testsigma: testsigmahq/testsigma: Build stable and reliable end-to-end tests @ DevOps speed. (github.com)
  
  

  Build stable and reliable end-to-end tests, the easy way, for free.

  Get Testsigma Open Source
  
  
• Even if you are a beginner in the field of automation, no-code tools can certainly be used as a starting pathway for learning automation.

“No-code is the first most accessible step of web and software development. lt allows anyone to create software, visually, without writing code”

Ben Tossell

Other Popular Myths

Well, the myths don’t end here. Test Automation is associated with countless other myths. Here are a few popular ones:

• Test Automation = Automation Tools
• Test Automation = No Manual Testing
• Test Automation = Test Team Downsizing
• Goal of Test Automation = 100% Automation

Check Out – Code less automation myths

“Everything really interesting that happens in software projects, eventually comes down to people.”

James Marcus Bach

For all our visual readers, I have summarized this entire article in the form of this mind map. Check this out here:

Hopefully, this article has given you an idea of the various possibilities and opportunities you can explore if you are interested in test automation. These ideas are often overlooked amidst myths, biases, and unrealistic expectations surrounding test automation. It’s always a good idea to explore and consider possibilities that may work for your context.

Happy Exploring! Happy Myth Busting! Happy Test Automation!

PC: Background vector created by redgreystock – www.freepik.com