Smoke testing (software)
In computer programming and software testing, smoke testing (also confidence testing, sanity testing,[1] build verification test (BVT)[2][3][4] and build acceptance test) is preliminary testing or sanity testing to reveal simple failures severe enough to, for example, reject a prospective software release. Smoke tests are a subset of test cases that cover the most important functionality of a component or system, used to aid assessment of whether main functions of the software appear to work correctly.[1][2] When used to determine if a computer program should be subjected to further, more fine-grained testing, a smoke test may be called a pretest[5] or an intake test.[1] Alternatively, it is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team.[6] In the DevOps paradigm, use of a build verification test step is one hallmark of the continuous integration maturity stage.[7]
For example, a smoke test may address basic questions like "does the program run?", "does the user interface open?", or "does clicking the main button do anything?" The process of smoke testing aims to determine whether the application is so badly broken as to make further immediate testing unnecessary. As the book Lessons Learned in Software Testing[8] puts it, "smoke tests broadly cover product features in a limited time [...] if key features don't work or if key bugs haven't yet been fixed, your team won't waste further time installing or testing".[3]
Smoke tests frequently run quickly, giving benefits of faster feedback, rather than running more extensive test suites, which would naturally take longer.
Frequent reintegration with smoke testing is among industry best practices.[9][need quotation to verify] Ideally, every commit to a source code repository should trigger a Continuous Integration build, to identify regressions as soon as possible. If builds take too long, you might batch up several commits into one build, or very large systems might be rebuilt once a day. Overall, rebuild and retest as often as you can.
Smoke testing is also done by testers before accepting a build for further testing. Microsoft claims that after code reviews, "smoke testing is the most cost-effective method for identifying and fixing defects in software".[10]
One can perform smoke tests either manually or using an automated tool. In the case of automated tools, the process that generates the build will often initiate the testing.[citation needed]
Smoke tests can be functional tests or unit tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods. Functional tests may comprise a scripted series of program inputs, possibly even with an automated mechanism for controlling mouse movements. Unit tests can be implemented either as separate functions within the code itself, or else as a driver layer that links to the code without altering the code being tested.[citation needed]
Etymology
[edit]In Lessons Learned in Software Testing, Cem Kaner, James Bach, and Brett Pettichord provided the origin of the term: "The phrase smoke test comes from electronic hardware testing. You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don't have to do any more testing."[3]
See also
[edit]References
[edit]- ^ a b c ISTQB® Glossary for the International Software Testing Qualification Board® software testing qualification scheme, ISTQB Glossary International Software Testing Qualification Board.
- ^ a b Dustin, Rashka, Paul. "Automated Software Testing -Introduction, Management, and Performance". Addison-Wesley 1999, p. 43-44. ISBN 0-201-43287-0.
- ^ a b c Kaner, Cem; Bach, James; Pettichord, Bret (2002). Lessons Learned in Software Testing. Wiley Computer Publishing. p. 95. ISBN 0-471-08112-4.
- ^ "How to: Configure and Run Build Verification Tests (BVTs)". MSDN Library for Visual Studio 2005. Retrieved 2010-11-20.
- ^ 2013-03-20 ISTQB Terminologi for test av programvare, versjon 2.2 Glossary Working Party, International Software Testing Qualifications Board, Erik van Veenendaal (engelsk), Ernst von Düring (norsk) "intake test: A special instance of a smoke test to decide if the component or system is ready for detailed and further testing. An intake test is typically carried out at the start of the test execution phase. See also smoke test."
- ^ Samuel Menaker; Sheetal Guttigoli (14 December 2014). Managing Software Development. Samuel Menaker, Sheetal Guttigoli. p. 40. GGKEY:JH61NP21TXJ.
- ^ PowerShell Magazine, DevOps, Infrastructure as Code, and PowerShell DSC: The Introduction, by Ravikanth C, January 5, 2016
- ^ Cem Kaner, James Bach, Bret Pettichord, Lessons learned in software testing: a context-driven approach. Wiley, 2001
- ^ McConnell, Steve. "Rapid Development". Microsoft Press, p. 405
- ^ "Guidelines for Smoke Testing". MSDN Library for Visual Studio 2005. 26 June 2007. Retrieved 2010-11-20.