Smoke Testing vs Sanity Testing Explained
Table Of Contents
Introduction
The life of a QA Tester will be considered incomplete if the terms ‘Smoke Testing’ and ‘Sanity Testing’ are not infused into it. Though these are some regularly used terms, there are some common misconceptions around them too.
Sanity Testing vs Smoke Testing – Myths and Misconceptions
Before we begin into the details of smoke testing vs sanity testing and what they actually mean, lets go through some common misconceptions and myths around them:
- Smoke Testing and Sanity Testing are the same and can be used interchangeably : This is not true and should never be done, why? we shall discuss that in this Article.
- Sanity Testing is equivalent of acceptance testing: Acceptance Testing is done to ensure that the build meets all the requirements that the client specified before the release while Sanity Testing is done to ensure the product is sane, rational for more detailed testing. These are not and should not be used interchangeably.
- If I am doing Smoke Testing, I can skip sanity testing: Smoke testing is a very high level testing, as in, this testing is done to ensure this build is suitable to begin any other type of testing. For example, this could be just a test to ensure the build installs and then the Sanity Testing can begin. Though sometimes, the smoke test cases are run together with Sanity Test cases, these should not be confused to be the one and the same thing.
In this article, we will try to clear above confusions once and for all. In case, you wanted to read more about Manual Testing, do refer to the link here.
Note: Because we will be using the term ‘Software Build’ many times in the article, lets define it here.
What is a Software Build?
‘Software Build’ is a process of converting source code, into a user application, after multiple revisions and code changes and Software Build creation involves multiple processes like, “Version Control, Code Quality & Compilation” and software build is also the result of this building process. In this article, a build will be referred to as a testable version of the software.
Smoke Testing vs Sanity Testing – What are the Differences
Smoke testing is performed in the initial phase of the software development life cycle by the developers to ensure the code module is successfully compiling and the software’s core functionality is working properly. Any other level of testing is done after successfully passing the smoke testing.
Smoke testing will fail if the required environment is not present, other required libraries are not imported in the code itself, or any major bug issue breaks the software build and does not let it run, etc.
Automation & Smoke Testing:
Usually this is the type of testing that is executed before actual automation test cases can run. For organizations that have continuous testing built in, smoke testing is equivalent to successful installation of the build for running test cases or execution of the first test case. So, this is not a type of testing that is deliberately automated but if test automation is put into place, the test automation can only run successfully once the software has passed smoke testing. Or otherwise, the first test case that executes might fail.
Sanity testing ensures that bug fixes, newly added functionality or any other code change doesn’t affect the stable software, build other functionality or introduce any type of bug issue. It also evaluates the software and ensures whether it’s ready for the next level of testing or not.
Automation & Sanity Testing:
Considering automation for Sanity Testing test cases, a recommended approach is to execute these test cases before running the complete regression test suite. The benefit is that if there are any errors in the sanity test cases, then errors can be reported sooner rather than later.
Some companies have benefited from automating their sanity test cases. Here is a case study where Freshworks – a feature packed product suite for businesses of all sizes – was able to automate 100% of their sanity test cases in 90 days using Testsigma: Testsigma+Freshworks – A case study.
Differences Between Smoke Testing vs Sanity Testing
| Smoke Testing | Sanity Testing |
| Performed on initial builds | Performed on stable builds |
| To test the stability of new build | To test the stability of new functionality or code changes in the existing build |
| Covers end to end basic functionalities | Covers certain modules, in which code changes have been made |
| Executed by testers & sometimes also by developers | Executed by testers |
| A part of basic testing | A part of regression testing |
| Done usually every time there is a new build | Planned when there is not enough time for in-depth testing |
Key Points of Smoke Testing vs Sanity Testing
The difference between smoke testing and sanity testing is, developers carry out smoke testing in the initial phase of the software development life cycle. It ensures software core functionality is working correctly. Testers perform sanity testing after the smoke testing to ensure the software works correctly after code changes, bug fixes, or new functionality is added. Evaluate whether the software is ready for the next level of testing.
- Smoke testing is performed in the initial phase of the software development life cycle. In contrast, sanity testing is done after smoke testing.
- Smoke testing ensures the software core functionality is working correctly, where sanity testing is performed to test if the software is working properly after the new addition of functionality and evaluate whether it’s ready for the next level of testing.
- The development team carries out smoke testing, but the QA and test teams perform sanity testing.
Summary
- Smoke and Sanity testing helps the QA team save time by quickly testing to make sure if an application is working properly or not. Also, it ensures that the product is eligible for further testing.
- Smoke Testing is done by both the dev team or by the QA team and can be taken as a subset of rigorous testing. Whereas Sanity testing are done only by the QA team. Also, Sanity testing can be considered as a subset of acceptance testing.
- Smoke testing is executed at the initial stage of SDLC, to check the core functionalities of an application. Whereas Sanity testing is done at the final stage of SDLC, to check the main functionalities of an application.
- As per the requirement of testing & time availability, the QA team may have to execute Sanity, Smoke on their software build. In such cases, Smoke tests are executed first, followed by Sanity Testing .
In practice, all QA teams need to do Smoke and Sanity testing. These testing types have a pre-defined number of test cases that get executed multiple times. This repetitive execution also makes them an ideal candidate for test automation. When looking for automation, you are recommended to use a tool that provides you ROI on automation from the initial stages. Testsigma is one such tool.
Frequently Asked Questions
Which comes first, Sanity or Smoke Testing?
Smoke testing is carried out early in the software development life cycle to ensure the software’s core functionality is working correctly. Smoke testing rejects badly broken software in the initial testing stage of software development.
After the smoke testing, sanity testing is carried out to ensure the newly added functionality in the software is stable. It does not affect the software’s existing functionality.
Which testing is done manually Sanity or Smoke?
Smoke testing is done manually without any test automation tool because smoke testing is about testing the software core functionality are working properly. Sanity testing is performed using an automation tool which evaluates whether the software is ready for the next level of testing.
Why is Smoke Testing called Smoke Testing?
The name smoke is somehow adopted from the plumbing community. Initially, people used smoke to check for leakage in the pipeline, then adopted it in the software testing. In the same way, smoke is used to check for leakage, and smoke testing finds major leakage in the software.
Choose a Tool that Lets you Automate from Day 1!
