Template:Smoke test/How-to

A Smoke test page is a subpage of a template /testcases page that works in tandem with development in a template sandbox. It consists of a relatively small set of test cases and includes some tests designed to test the features of a particular set of changes or proposed changes in a template sandbox.

Features

The advantage of a smoke test and what distinguishes it from a standard /testcases page is that a smoke test is designed to be transcluded from a template /doc page, so that when sandbox development is in progress, every preview mode refresh re-executes the smoke tests, providing instant feedback about the code changes without having to save the sandbox, thus enabling the template editor to make a series of incremental edits and publish when ready, knowing what the test results will be in advance.

Contents of a smoke test

The purpose of the smoke test page is to assist a template editor in rapid and efficient testing of sandbox changes. A smoke test page is located at subpage /Smoke test of the standard template /testcases page. Any tests may be placed there, but a set of paired tests that compare live and sandbox template invocations of the same parameter set are the norm. These could be copied or styled after some of the tests on the /testcases page. In addition, some targeted test cases should be added to test the specific functionality changes being contemplated by the sandbox editor.

It is up to the sandbox editor to decide what test cases to place in the Smoke test. As with sandbox contents, the contents of a Smoke test page for a given template may change every time the sandbox is used for some new or changed feature development.

Presentation

One presentation suggestion is to have two major sections, the first being a classic smoke test consisting of a handful of regression tests testing the most basic template features, just to make sure that sandbox changes don't completely break the template. These don't have to be detailed or numerous or duplicate the /testcases page in detail; something like a missing curly brace or a misplaced #if: will likely make everything go sideways, so intricate or detailed testing is not necessary. A good name for this section is ==Regression==.

The second section should be directly targeted at the new or changed functionality. If you are adding a new parameter, the second section is the place to add a test for it. Name it anything you want; something that will bring the purpose to mind and distinguish it from other sandbox development efforts would be helpful; e.g., == New param xyz. After the sandbox is moved to live, the second section should get moved to the parent /testcases page, and form the kernel of a new set of test cases for the new or changed functionality.

Outline

Here is a step-by-step outline of how to create a Smoke test and incorporate it into sandbox testing:

create a page named /testcases/Smoke test under the template whose sandbox you are testing
copy two or three of the more generic tests from the testcases page into Smoke test, add section header ==Regression== over it
add another section labeled after your new/changed functionality, add a few testcases that test it. Each test case should invoke the live template and the sandbox with the identical parameter set
edit the template /doc page #Examples section, tack {{smoke test}} onto the end of it and save it.
edit the sandbox, hit Preview button; does everthing look good?
- yes: cancel out and continue
- no: fix the Smoke test to resolve issues; back to #5
if desired, add more complex tests with parameters:
- edit /doc, change the {{smoke test}} invocation by setting optional positional params 1, 2, and 3 to whatever is useful to test
- edit Smoke test to interpret positional params 1, 2, and 3 passed to it, to alter how the tests work; go back to #5

Use as a template

As explained at Template:Smoke test#Notes, the Smoke test is designed in tandem with a set of template sandbox changes. Template {{Smoke test}} is designed as a wrapper and a link between the /doc page of the sandbox in testing, and its /testcases/Smoke_test page. The Smoke test may be just a page with some tests on it, like a /testcases page, or may itself be designed as a template that accepts parameters.

Pass-thru parameters

When invoked via {{Smoke test}}, then you can pass up to three positional parameters to the wrapper from the /doc page of the template sandbox undergoing development, which in turn passes the three parameters through to the /testcases/Smoke_test.

If designed as a template, there is no predefined meaning to any of the parameters, and the editor performing the sandbox development can assign any meaning to the parameters that helps generate the appropriate test case execution in the second section of the smoke test.

Testcases parent page

If desired, it may be transcluded onto its parent /testcases page, for visibility and convenience, but it doesn't have to be.

Examples

Notes

T.b.a. – Add an editnotice when a page title matches /testcases/Smoke test$ and link to this How to.