top of page
Writer's picturePéter Földházi Jr.

Sanity vs Smoke vs Regression testing



Sanity and Smoke testing are often mixed up by interview candidates. The main reason for that is that many organizations misuse these terms. What is a smoke test is often called as a sanity test. And smoke test is described as a regression test suite. All wrong!

Let me clear the picture for you.


Smoke testing: run after every build to ensure that the build is stable enough to execute further testing, and to continue with the build process. Ensures that the main functionality of the software is working correctly. Nowadays, it is usually automated. Wide and shallow (not deep) testing.


Sanity testing: is a narrow testing, which is always performed manually and is not scripted (no low level test cases are available). Ensures whether the requirements are met or not. Narrow and deep. May resemble to targeted regression but its goal is to validate (sometimes new) features instead of finding regression issues. With sanity testing you basically answer the question: does it make sense?

Never state that you have automated your sanity test suite, it is a false statement! Sanity testing cannot be automated. You most probably wrongly call your smoke test suite as sanity test suite.


Full regression test suite: all of the test cases are executed. If your regression test suite is completely or mainly manually executed, then it is recommended to execute full regression testing for the first some releases until the software under test becomes stable.

Once you have all the test cases automated (the ones that can be automated), then you can return back to executing the full package.


High priority regression test suite: often mixed up with smoke testing, as in the latter the most critical cases are executed and is basically like a shorter regression. However, when executing high priority regression, all the critical and high priority cases are executed, which is a deeper testing compared to smoke testing.


Targeted regression test suite: often mixed up with sanity testing, as in both only select features of the software under test is examined. However, prior to execution of the targeted regression package, the scope of regression is determined according to time and testers available for execution and the most impacted areas of the software by the new features and other changes. This is in order to ensure, that even with the limited time spent on testing, the most amount of regression issues can be found. Sanity testing on the other hand is to validate features, not to necessarily find regression issues.

163 views0 comments

Recent Posts

See All

Comments


bottom of page