• info@helpingtesters.com
  • helpingtesters

What is Exploratory Testing | HelpingTesters

October 18, 2017 Types of Testing
Exploratory Testing, Exploratory Testing in Agile, Exploratory Testing Example, Exploratory Testing meaning

There are cases when testers don’t get the requirement document but they have to finish all testing activities within the limited time frame. Within this strict timeline, they have to explore, learn the application, do test planning, test the application and present a detailed test analysis. So, how can all this be achieved within such a short span?

Hence the need for exploratory testing. Exploratory testing follows the approach of minimum planning and maximum test execution, which involves both logical and critical thinking, as there is neither test case to just read and execute nor automated test scripts to run as designed. Exploratory testing is one of the strongest ways to understand the application and then test it with logic. This enhances the quality to the higher extent, and which can be combined with other testing practices simultaneously.

In an article, James Bach wrote, “Exploratory software testing is a powerful and fun approach to testing. In some situations, it can be orders of magnitude more productive than scripted testing.”

The diagram below would give you a rough idea of exploratory testing.


One way of putting it is, exploratory testing is a documented form of random testing. This is the opposite of formal/scripted testing process which covers all phases of software testing. In Exploratory testing, testers are not going to follow any particular lifecycle or approach. They are simply going to validate the application by performing various tests on the application in their way, be familiar with the application and capture any errors found during the testing process.

Exploratory testing is sometimes referred to as ad hoc testing or monkey testing. All these terms focus the same but are not exactly the same.

  • Monkey testing is the random way of testing the application. Once the issue is observed, analysis on the issue for what could have caused it comes into the picture. This is sometimes called Random Testing. While performing monkey testing, the test might detect a handful of anomalies from the application. But are you sure that the application is tested for user scenarios? The answer is NO (So it can’t be exactly termed as exploratory testing). Even though it is random testing, bit does follow some systematic approach while a tester performs it.
  • Ad Hoc testing is testing the application randomly but with complete knowledge of the application and/or domain. Here tester focus on each and every scenario being tested and the way it is being tested. Also, alternate scenarios are focused and tried out in case of issues. But again, this does not have any systematic approach as such. It is generally done by experienced testing professionals, and require a good imagination and analytical skills, but this is not the learning phase.

Exploratory Testing in general means “Testing application without Requirement Specifications”. But it is more than that:

  1. Skilled testers perform exploratory testing once scripted test execution is done and brainstorming is done to find more scenarios.
  2. If the time to test an application is less, then exploratory is the best way to achieve test coverage. Normally we have seen the sprint cycles have even reduced to 1 week so in these cases, it becomes difficult for the tester to write test cases and execute them.
  3. Exploratory means NOT to test with scripted test cases, but you should aware of full requirements and think beyond it.
  4. Any tester testing without referring scripted test cases is performing exploratory testing.
  5. Corner areas can be reached through efficient exploratory testing.

“Discovering the unexpected is more important than confirming the known.“— George E. P. Box  

Advantages of Exploratory Testing

Some of the striking advantages of exploratory testing are:

  • Testers get familiar with the application and at the same time, check whether the application under test is behaving as per the expectation. In fact, the tester is doing the same thing as traditional tester do, but without documentation.
  • The tester can find the bug quickly as compared to traditional testing. Results have shown that the rate at which new bugs are found is much higher for exploratory testing, as compared to scripted testing.
  • Many times, it’s not mandatory to follow traditional approach and prepare test cases for the scenarios which do not have much business impact. For such scenarios, exploratory testing is recommended. Also when the software versions are in the early stage and tester is required to be familiar with the application, exploratory testing is recommended.
  • Since much logical thinking is required for Exploratory testing, the outcome is the best quality, while in cases of scripted testing objective is to pass/fail the test without brainstorming on test scenarios.
  • Exploratory can be done at any test level and type. Either performance, security, integration etc. In general, it can be included in all the testing practices.
  • Uncommon integration issues can be uncovered by rigorous exploratory testing, which could result in detecting more possible ways to break the application.

However, exploratory testing does not guarantee the quality of the application but it is advantageous especially during time crunch or shoe-string budgets.

Disadvantages of Exploratory Testing

  • Not able to showcase the efforts inputted for exploratory testing, since most activity is not documented.
  • Not possible to showcase the quality of exploratory testing if bugs weren’t found.
  • The outcome of exploratory usually comes up with some unanswered queries which need to be clarified by clients. Taking assumption mostly of them degrade the objective & effectiveness of Exploratory Testing.
  • Exploratory testing is only effective if done for some hour’s duration. It is never a day-long activity.
  • Mostly small tight schedules projects perform exploratory in that test case expected results for exploratory remains unreviewed.
  • If much critical thinking is not done during exploratory testing, the outcome could be much more disappointing than scripted testing.
  • Finalizing the test coverage is challenging, as there is no formal mapping between the tests performed and requirements. Until the end of User Acceptance Testing, the results of exploratory testing is not much taken into account for quality.
  • Since this is an unplanned activity, it is very difficult to convince customer about the expected efforts and time.
  • Sometimes, this testing is taken lightly and areas are left untested. Also, chances are there to leave the defects as there is no documentation involved in testing.
  • This does not help in root cause analysis in case of missed defects to production.

Challenges faced in Exploratory Testing

As in any other testing type, even exploratory testing have challenges:

  • The entire application and the purpose of each and every feature have to be learned in short span of time.
  • The term “unplanned activity” rises lot many questions on effort, time, and efficiency that the tester has put. Difficult to convince customer and management about the actual accounts.
  • There will be no proof to show the effort put unless defects are logged. In this case, it is challenging to defend the actual tests performed.
  • Since test cases and test scripts are not involved in exploratory testing, the tester has to continuously keep thinking about the possible tests to execute, which may become stressful.
  • Difficult to reproduce the defects in case of corner scenarios
  • Test Coverage is difficult to prove if it is clubbed with system/regression testing, as most of the coverage is shown by their results, and exploratory has nothing much to show the coverage.
  • Difficult to record the each and every test performed.
  • Tedious when exploratory testing results have to be reported.

Chartered Exploratory Testing

There are many ways to perform Exploratory testing, but one of the most commonly used methods is known as Chartered exploratory testing. In this method, the tester is required to make one charter, to guide how to test within a defined timeline. So first we need to prepare Charter and then time box should be defined, so as to finish the task within the defined timeline.

Mission

  • Defines the objective we need to achieve from exploratory tests.
  • What features we need to test and how to test
  • Testers may focus on specific complex functionalities for a particular application in one session.
  • Testers may choose one of the system workflows and test it in a particular session.

Charter

  • What we want to achieve
  • What tactics to use
  • What risks are involved
  • What kinds of defects to look for
  • Any high-level test plan
  • Who will be involved, would be good either to get test lead involved or perform pair to pair testing.

Session

  • Generally varies from 2-3 hours.
  • Breaks while performed exploratory testing isn’t recommended.
  • Don’t break current scenarios in-between if any other scenario is clicked.

Session Report

  • Testers can prepare test notes while exploring the application and prepare notes for their own learning.
  • If possible notes on all the scenario covered.
  • Note if clarification required with the client for some scenarios.

Bugs

  • List of bugs identified.
  • Raising them on Defect Management Tool with proper classification

Discussion

  • Schedule meeting with the manager to provide information regarding the different activities.
  • Prepare report and share with the team.
  • Keep documents prepared in Session Report for future reference.

Even though documentation is not mandatory for exploratory testing, it is advisable to create a one-page document at the end of exploratory testing.  This documentation can be submitted to the manager which would showcase the status of testing and the current state of the application.

Where to use exploratory testing

  • Exploratory testing may be useful in situations where testing needs to be completed without any proper requirement document.
  • It is also recommended when time available for testing is slim.
  • When the project has budget-related constraints.
  • If the modules are simple and don’t require elaborate test documentation in the form of test cases.
  • You need to add Critical thinking to find more defects.

Misconceptions Regarding Exploratory Testing

Lots of misconceptions are there regarding exploratory testing. Some of them are as follows:

  • Unstructured: Many people think that it is an unstructured form of testing which is not correct. It is structured and only done by testers who have ample product knowledge.
  • No need to plan or document test: Many people think that there is no need to plan or document test. But it’s a complete myth. The plan is mentioned in charter though it is very short and may be contained in a paragraph.
  • It is new and fast: Many people believe that Exploratory testing is new and can help in test completion at a fast pace. But the fact is, testers have been using exploratory testing for a very long time. Even though it is a swift alternative it is a time-bound affair and can take up quite a lot of time depending on the complexity of the flow or module under test.
  • It is more effective at finding bugs: Again it is not true. Traditional testing or scripting testing are also quite effective, and maybe even more effective at finding bugs than exploratory testing.
  • No need to do is scripted testing: It is also not true. Scripted testing helps in sequentially covering all elements of the software, to keep track of the progress and be confident of the quality of the software. Scripted testing or traditional testing needs to be applied along with Exploratory testing to get the best effect.

How to Be a Good Exploratory Tester

  • Be aware of the domain you are performing testing.
  • Gone through all the scripted test cases so that you can think scenarios beyond that.
  • Logical thinking is the key. Don’t confuse exploratory with monkey testing.
  • Get an idea, talk to the client, developers to be aware of the app complete functionality.
  • Don’t follow assumption while doing exploratory, note them down for clarifications.
  • Plan your tests i.e. objectives you want to achieve with current exploratory.
  • Don’t perform regression testing, exploratory test have the objective to find issues where critical thinking is required.
  • Don’t be dependent on exploratory testing tools, this type of testing is best with individual till the time AI testing tools are available for testing.

Conclusion

Most of the experienced testers while performing exploratory testing leverage their product knowledge to test the application intuitively. That is why exploratory testing is done by the experienced tester.

For this reason, fresh testers should follow traditional scripted testing techniques and go through the test documentations while testing the application.

Overall we can say that exploratory testing is an intelligent way of doing testing especially when the need of the hour is to deliver quality application within a limited time frame. Also, it helps in improving the design of the application in case of integration issues that are rarely encountered by end-users.

Even though documentation is not mandatory, good exploratory tests are planned, engaging and creative in nature.

This article is the outcome of my thoughts added to exploratory testing we have been performing in daily testing. If you want to read more about Exploratory please follow articles mentioned in the reference section.

References:

https://en.wikipedia.org/wiki/Exploratory_testing

http://www.kaner.com/pdfs/QAIExploring.pdf

http://www.satisfice.com/blog/archives/1509

http://www.testingeducation.org/a/nature.pdf

About the author

RamPrakash Singh author

Ram Prakash has worked in various domains of testing including security, performance, security testing and automation testing. Including several tools like QTP, selenium, LoadRunner,JMeter, VSTS Coded UI, soap UI, Burp Suite etc.

Leave a Reply

Your email address will not be published.