• info@helpingtesters.com
  • helpingtesters

Is your Bug Fixed? Retest or Regression??

August 27, 2017
Regression Testing, Types of Regression testing, how to do regression testing, ReTesting

This post covered various points related to ReTesting, Regression testing, Bug Estimation etc. It’s quite common in the software testing industry that the fixed bug is just tested and closed. But Check, Check, Check… Is your bug really fixed? Is it not impacting on any other features? Is it not introducing the new bug to the system? Is it fixed only for the given input, or it covers the entire input range? Straight forwardly, of course, we follow the steps to reproduce and find out whether the bug is corrected. But what next? Is it just moving the bug status to Verified and Closed? A big NO… 

When the developer makes corrections to the piece of code where the bug was identified, the bug status will be moved to Fixed. As a responsibility of tester, he/she should always be prepared to retest the fixed bug and perform regression test on it. So what does this actually mean? Let’s see…

Retesting: Test the bug as mentioned in the description following its steps to reproduce and check whether expected result is achieved as per requirement

Regression: Performing tests in the impacted areas that the bug fix could have caused. This is to check whether the already working features are still working as expected even after bug fix

 

Tester’s Checklist when the Bug is fixed

⇒ Test the actual bug

Always test the bug firstly in the environment where it was found and on all the OS/browser mentioned. As a first attempt, just test the bug as it is mentioned in the steps to reproduce section. Then do the testing around the bug depending on its type:

  • Data bug: Check for the specific input which made you identify the bug. This should then be followed by the same input range (here apply equivalence partitioning method) and at the boundary of the range. Check for all possible wrong inputs (Say if something is not working for positive values for which bug is logged, then upon fixing it check for positives, negatives, 0, alphabets, special characters, international characters, other language characters).
  • Text types: If the bug is of text type like error messages, information messages, pop-up messages, field names, it’s always better to list them for localization testing as well. Here you will have to retest the bug in an application for all languages that it supports.
  • Field level: If the bug is related to fields, test for all the same type fields in the page, followed by same type field in the feature where it was found.
  • Server-side: If the bug is related to the server side, i.e., search functionality, batch processing (triggering emails at the specific point of time), DB checks – then test with enormous data, metadata. (If at all you need to switch ON / OFF the server, keep the entire team informed)

⇒ Test the impacts

When the retesting has passed successfully, analyze the areas of impact that the bug fix could have caused and perform a quick test on it. Any issues found on this level should be logged as a new bug and the possible cause should be clearly mentioned. Here, the tester should make sure that the new bug caused should not be present in the previous builds. So the best thing to do here is, while logging the bug itself tester should do a thorough analysis and provide impacts so that it is a help to the developer and you as well (as you will already be having a handful of information on what all to test).

⇒ Retest failed cases

Whenever there is a bug logged during regression cycles, it is the common practice to link all the test cases that fail because of the encountered bug. So when it is fixed, have a look at the test cases that are linked and see re-execute them for failed scenario/steps. If the test case passes and no more bugs encountered from the mentioned scenarios, PASS it. If it fails due to other bugs, FAIL it and link it to the bug logged for the new issue.

⇒ Smoke Test Bugs

If the bug fixed was logged during Smoke testing, based on the severity, the level of testing varies. In this case, actual bug and the feature at a very high level should be tested and confirmed.

⇒ Keep an eye on the bug in future builds (1 or 2 at least)

Always mention the build on which the bug is being verified. Make some time and test the bug or its possible impacts in upcoming builds (hardly 5-10 minutes check). Or make sure your ad-hoc cycle covers it

⇒ Check the bug in other environments

It’s always a good idea to make a checklist of the bugs to test it on other environments (Staging / UAT / Pre-prod / Prod (if critical)). Here you can always make a choice with the severity of bugs, i.e., select Critical / Very High / High severity bugs can be chosen to be tested.

What if the Bug Reappears in later Stages(builds)

If the bug is Verified and Closed in the current build and the same bug is reappears in future builds, then tester will have to answer for:

  • Was the bug retested properly?
  • Were the impacts tested properly?
  • The possible cause for the bug to re-appear?
  • Is there a necessity to rerun the regression suite because of this bug?
  • The effort to re-run the regression suite?
  • And the list goes on.

 

What if the Bug that is not Logged by you comes as fixed

Scenario 1: Sometimes, bugs logged by one tester will be assigned to another tester (reasons may vary for such assignments – may be the tester who logged the bug is unavailable / is moved to other project / tied up with other priority tasks). In such cases, after retesting the bug just look around for associated features and analyze the impact and then test it, or perform a short ad-hoc around the bug. If there is a test case linked to the bug, assign it back to the one who executed it and keeps him/her informed, or informs lead/manager on how to go about it. Because this call depends on the effort, status reporting changes which are usually handled by lead/manager, or sometimes customers themselves don’t want to put extra efforts on executing the linked test case.

 

Scenario 2: Sometimes, the even customer reported bugs also get assigned to the testing team which then gets assigned to a tester. Before closing the bugs, it always a good thing to check with leads/managers before closing. This is because sometimes leads/managers will be having more details or references on the bug which are not documented while reporting or information might have passed on later in calls/mails where may be the tester is not in a loop.

 

Note: No matter on who has logged the bug, it’s always tester’s responsibility to give proper judgment to the fixed bugs either to Close or Re-open.

 

Bug Testing Estimation

Whenever there is an estimation to be provided for bug validation, it should always be kept in mind that effort to be put is the total of below points:

  • Actual bug testing
  • Impact analysis
  • Impact testing (Regression)

Changing Bug Status effects Reporting in Regression

Changing the bug status to “Verified and closed” also changes the report values on Open bugs, Closed bugs, Test case Pass / Fail / % of success (if it is failing due to test case step). So it is very important to make sure bug is properly tested before closing. Or else, it will mislead all the stakeholders on the status of the project.

Final word on this: It is very important to Close the bug only after thorough testing. Sometimes, it is OK to ask for more time to test the bug depending on all the above factors

About the author

Vaibhav Singhal author

Vaibhav is in software testing since a decade. Worked with different testing techniques along with exploratory testing. Performed different regression, performance, api etc testing with tools and libraries like selenium, jmeter, loadrunner, protractor, rest assured, Katalon Appium and others.

Leave a Reply

Your email address will not be published.