What Is Exploratory Testing?
Exploratory testing was first coined by Cem Kaner who defined it as
a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.
Exploratory testing relies more on the experience and skills of the individual tester to identify defects not easily detected by other types of tests. It is an approach that is often described as concurrent learning, test design, and execution.
A Brief History of Exploratory Testing
Before Kaner coined the word in his book, Testing Computer Software, it had been known as Ad hoc testing. When it was still known as Ad hoc testing, it was looked down on as sloppy and careless work. For this reason, a group of testers began using the term “exploratory testing” to emphasize the thinking process involved.
However, the term was only popularized when Cem Kaner in his (now out of date) book Testing Computer Software. The introduction of which is as follows:
No matter how many test cases of how many types you’ve created, you will run out of formally planned tests. You can keep testing. Run new tests as you think of them, without spending much time preparing or explaining the tests. Trust your instincts.
Testing Computer Software
Exploratory Testing: How Does It Work?
In order to understand how exploratory testing works, there are four concepts that need to be discussed:
- Test Mission
- Test Charters
- Test Sessions
- Test Heuristics
Test Missions
This provides the reason for why you are doing the test and what you seek to. The tester creates the mission, designs tests to accomplish that mission, and does the testing.
Test Charters
Is the structure on how to carry out the test. It is the mission broken down into actionable items. It should suggest what to test, how it can be tested, and what needs to be looked at.
Learn how to write Test Charters.
Test Sessions
It is the uninterrupted time when the test is conducted. It allows identifying a starting point for testing.
Test Heuristics
According to an article written by Yaron Kottler and Bernie Berger test ideas are usually driven by a set of heuristics, which have been defined as “a fallible idea or method which may help you simplify or solve a problem”. In other words, heuristics can be thought of as rules-of-thumb which can be used to drive your test ideas.
Exploratory Testing vs. Scripted Testing
The definition came up in comparison with script testing. If exploratory testing is considered freestyle, testing on the fly and thinking type of testing while scripted is testing software based on pre-determined outcomes.
In scripted testing, the test cases are designed first and then they will proceed to test execution will proceed. While in exploratory testing, it will be done at almost the same time.
Scripted test execution compare actual results with expected results while exploratory testing uncovers defect during the test that is not within the expected results.
Exploratory Testing Examples
Exploratory testing is used for trialing a mobile gaming app where the focus of the test is whether the players will be able to use the update in real live game. Exploratory testing will be able to detect bugs that will appear in real world play.
In the early life of software, exploratory testing is the most effective to determine how it will perform in actual usage. The tester should have knowledge of who the product is for and what is it intended to be used for.
Exploratory Testing in Agile
Exploratory testing fits the Agile methodology in that it does not follow a single predetermined methods. It places emphasis on reacting to change and collaboration. Testers are encouraged to freestyle, play around with software. Furthermore, in Agile environment, there is little time to create test designs and scripts which makes the flexibility of exploratory testing all the more befitting.
Advantages of Exploratory Testing
The following are some of the advantages in using exploratory testing:
- It finds important bugs fast.
- It does not need too much preparation.
- It is more intellectual stimulation than scripted tests.
- Testers can devise new test from what they learned from the previous tests. They do not need to complete steps are not necessary to the investigation that they are doing.
Not only that, according to the article written by Tom Totenberg statistics about exploratory testing shows:
- On average, exploratory testing discovers 11% more overall defects than scripted testing.
- For defects that should be immediately obvious, such as a missing button in the UI, exploratory testing discovers 29% more vs. scripted.
- When it comes to “complex” bugs (bugs requiring three or more user actions to cause an error or failure), it jumps to 33% more defects found.
Disadvantages of Exploratory Testing
- Since it is more freestyle the test performed will be hard to review before execution or the kind of tests that were done is hard to replicate (unless you record it).
- It depends more on the tester and the tester’s knowledge.
- Because it depends on the tester, the time that the tester is brought up to speed may be a factor in the completion of the test.
Choosing between exploratory and scripted testing must not be because the former is better than the latter. The best measure is what will work best for the software to be tested. Knowing the nature of both test methodologies, which one will work the best.