• info@helpingtesters.com
  • helpingtesters

SUBSCRIBE TO OUR NEWSLATER





What is Black Box Testing – HelpingTesters

September 18, 2017
Black Box Testing, Black Box Testing meaning, Black Box Testing techniques, Black Box Testing types

As the software tester, it is the common thing to know that we have two classifications for Software Testing: Static testing and Dynamic Testing. If Static testing is the Verification activity which is all about reviews, then Dynamic testing is the Validation activity which is about actually testing whether the application/product is working as per the business requirements. Again the dynamic testing is further classified into Functional and Non-functional Testing. And Functional testing is where the concept of Black Box Testing usually comes into account.

Synonyms for Black Box Testing: Functional Testing, Behavioral Testing, Closed-box Testing, Opaque-box Testing

Black Box Testing

Black Box Testing is the testing method where the internal structure/design/implementation of the application is not known to the tester, and only the functionality of the application is expected to be assessed based on the requirement specifications. The application is considered as the black box and the testing is performed by providing inputs to it, and output is analyzed. Here the internal working on what happens when the input is provided to the application is not known to the tester. Testing is performed focusing on the functionality of the application as a whole.

Actual output from the application when the variety of input is provided is analyzed by the tester to confirm that the functionality abides by the requirements. The input may be in the form of keystrokes and clicks, to verify what happens after performing these operations. Technical features are totally hidden from the testers. When an undesirable output follows the input, after the detailed analysis, the bug is reported for the same. Test cases are developed according to software specifications like functionality, usability, and performance.

Note: It is not that tester is forbidden from knowing the internal structure of the application, but it is highly not recommended or discouraged. The tester can still get to know about the internal structure of the application with his / her interest.

Black Box Testing can be performed only from the later phase of development. And this testing is applicable to different types of testing: Integration, System, Regression, and Acceptance Testing.

Advantages and Disadvantages of Black Box Testing

Black Box Testing has its own set of advantages and disadvantages, which we will hover below:

Advantages of Black Box Testing

  • No need for implementation knowledge, programming knowledge – Tester can be non-technical person
  • Tester and developer are independent and there are no dependencies between them
  • Assess the application quality by analyzing the actual outcome with expected outcome
  • Helps in developing software in a more user-friendly way than mentioned in requirement document as a tester is supposed to test the application from user’s point of view.
  • Testing can be done once the application is released by the development team and does not have any dependency on the development team for testing.
  • Once the functional specification is complete, test cases can be designed.
  • More effective when the application is large and complex
  • Ambiguities, inconsistencies, and incompleteness can be identified at the early stage of testing

Disadvantages of Black Box Testing

  • There are chances of leaving unidentified conditions in absence of knowledge of programming structure.
  • In short span of time, it is quite difficult for the testers to identify all possible inputs to write test cases for a particular scenario and test it.
  • Practically, only limited number of possible inputs can be tested through manual testing
  • Not all the paths/logics are tested
  • Results in repeated tests if the developer has already tested the feature, and tester is not aware of it
  • Majority of the bugs are expected to be covered from test cases, which is quite difficult of the application is large and complex
  • Difficult to achieve 100% Test coverage if the application is complex

Black Box Testing Steps

Black Box Testing can be carried out by following the below steps:

  • Analyze and examine the requirement specifications at the early stage of STLC
  • Analyze and determine the inputs (valid and invalid) for the application
  • Determine the expected outcome for all possible input categories
  • Design test cases with all possible types of inputs
  • Execute test cases on the application and record actual outcomes
  • Compare actual outcome with expected outcome for all the tests
  • Report defects for unusual behavior of the application
  • Retest and regression test the defect and application once the defect is fixed

Types of Black Box Testing

The common Black Box Testing types are as follows:

  • Functional Testing: Testing the application based on functional and business requirements of the application
  • Non-Functional Testing: Testing the application based on functional and business requirements of the application – Load, Performance, Stability, Reliability, Volume, Stress, etc
  • Regression Testing: Testing the impacts of the application due to bug fixes, new feature addition, existing feature modification, and existing feature deletion.

Black Box Testing Levels

Black Box Testing can be performed at different levels throughout the testing phase. Below are the different levels where Black Box Testing is conducted:

  • Functional Testing
  • Regression Testing
  • Scenario Testing
  • User Acceptance Testing
  • Alpha Testing
  • Beta Testing
  • Usability Testing
  • Smoke Testing
  • Exploratory Testing
  • Adhoc Testing
  • Load Testing
  • Stress Testing
  • Volume Testing

Black Box Testing Tools

Manual Testers do not need any tools as such to perform Black Box Testing on the application. But for automation testing, Black Box Testing tool depends on the type of testing being performed.

  • Functional or Regression Testing – QTP, Selenium
  • Non-Functional Testing – Jmeter, LoadRunner

Most of the Black Box Testing tools are of record and playback type, which records the tests conducted in the form of Scripts using JavaScript, VB Script, Perl, etc.

Black Box Testing Techniques

There are some techniques used in Black Box Testing to validate the any given functionality. These techniques are mainly to determine the possible inputs that can be provided to the application’s feature and to analyze the outcome of it. Below are the techniques used in Black Box Testing to design test cases as well as for execution:

1. Equivalence Partitioning

Equivalence Partitioning is also termed as Equivalence Class Partitioning (ECP). This technique classifies the whole input range into different partitions, called class. The idea behind this technique is that, if the test passes for one of the inputs within the class, then the same test passes for all the inputs within the same class. If the test fails for one of the inputs within the class, then the same test fails for all the inputs within the same class. From the same class, the test can either pass or fail, but not both for different inputs within the same class.

Let’s understand this technique with the simple example:

Functionality: Text field accepts only numbers from 1 to 99

Possible classes: Class A: {1-99}, Class B: {negative numbers}, Class C: {>100}, Class D: {0}, Class E: {a-z, A-Z}, Class F: {Special Characters}

Now, test the functionality as below:

  1. Test the functionality for one of the inputs in Class A – Test Passes. Here Test should pass for all the inputs in Class A
  2. Test the functionality for one of the inputs in Class B – Test Fails. Here Test should fail for all the inputs in Class B
  3. Test the functionality for one of the inputs in Class C – Test Fails. Here Test should fail for all the inputs in Class C
  4. Test the functionality for the input in Class D – Test Fails
  5. Test the functionality for one of the inputs in Class E – Test Fails. Here Test should fail for all the inputs in Class E
  6. Test the functionality for one of the inputs in Class F – Test Fails. Here Test should fail for all the inputs in Class F

This is how ECP technique works.

2. Boundary Value Analysis

Boundary Value Analysis is also termed as Boundary Testing or Boundary Value Testing (BVT). This technique involves testing of input within the data range and at the adjacent sides of the boundary range. In other words, we can say that this design technique involves testing of data on the boundary as well as the adjacent value inside/ outside of the boundaries. Boundary value is basically the test condition on either side of the boundary. It ensures that the application behaves differently on either side of a boundary, and consistently within the boundary.

Let’s understand this technique with the simple example:

Functionality: Text field accepts only numbers from 1 to 99

Now, test the functionality for the inputs: 0, 1, 2, 98, 99, 100, and any number from 3 to 97

Here test should pass for boundary value, within the range, and adjacent value inside the boundary. i.e., 1, 2, 98, 99, and any number from 3 to 97 should pass the test. And the test should fail for the adjacent value outside the boundary i.e., 0, and 100.

This is how Boundary Value Analysis works.

3. Cause-Effect Graphing

Cause-Effect Graphing is also known as Decision Table or Ishikawa Diagram or Fish Bone Diagram. This shows the relationship between the output and the factors that influence the output graphically. Cause stands for input condition that brings an internal change in the system. Effect represents an output condition, system state resulting from a combination of causes. The cause is the condition/input and the effect is the action/output, both together turns into the test case. Combination of inputs is more focused to analyze the output.

Let’s understand this technique with the simple example:

Functionality: Login to the account

Cause Effect Graph representation here will be:

Cause Effect Graphing

4. Error Guessing

Error Guessing technique is one such technique which comes with growing experience with respect to domain, type of application, etc. Testing the particular type of application for long-term helps in increasing the intuition towards the application type, and it would be easier for such testers to attack the faults more quickly. Tester applies his / her experience in guessing the areas where the defects could be and design test cases around it. This technique is sometimes called Fault attack.

Let’s understand this technique with the simple example:

Suppose tester has good experience in testing e-Commerce applications. Now when the Cart functionality of the application is assigned to the tester, he / she would definitely play around the feature based on the previous experience, like: increasing, decreasing the quantity, entering text (alpha, numbers, and special characters) in the quantity field if it is the text box, navigating to product through cart items, removing the product, etc.. All these points strike as soon as the tester is asked to test the feature. Here tester does not need any test cases, requirement specifications to validate the basic functionality of the feature. Whereas, the tester with no or less experience in this area might need test cases or requirement specifications to test the feature.

5. State Transition Testing

State Transition testing is the technique followed to validate all the possible states in the application. The application is tested for its transition to different states based on the given input. Here the input provided triggers the application to move to its next state, and the test cases are designed around this transition. It will be more complicated if the application accepts the number of inputs and has different state transition for each.

For each possible input, the state which results should be captured in the test case, failing which will result in major defects in the later stages. Any missed transition will be more expensive and high operational cost to correct it.

Let’s understand this technique with the simple example:

When the user registers to the application, starting from registration to login, there will be several validations to perform. Here, we will see all about registering to the application and logging in after account verification.

State Transition Technique

6. Use Case Testing

Use Case Testing is the technique where the all the real-time scenarios/business flows are validated. This is mainly to simulate the real-time usage of the application based on business requirements and the same is given more importance. Use Case Testing technique will be more difficult to follow when the application has complex flows and is larger. This is more suitable for System testing and Acceptance Testing phases, as the end-to-end scenario is considered as the use case and is assessed.

Critical business flows are considered as Business Cases, and others are considered as Use Cases. Test Cases are usually written in-depth (step-by-step), instead, they are designed as Test Scenarios. Testers here are expected to be highly skilled, domain testers, or subject matter expertise in the project.

Let’s understand this technique with the simple example (in continuation with the example in State Transition State):

When the user registers to the application, starting from registration to login, there will be several validations to perform. Here, we will see all about registering for application and logging in through verification process, followed by a regular login. Also, account block scenario is added in this example.

Use Case Testing

7. Exploratory and Adhoc Testing

This the technique which is followed when the Regression testing is at the end of its stage and/or testers do not have enough time to test the application with the help of test cases. This is done with guess and intuition where the application may fail with specific inputs. Exploratory testing is performed by any tester who has no or less domain knowledge and does not know the application much. Tester here tries to explore the application with curiosity on knowing what happens next. Whereas Adhoc testing is performed by the testers who are Subject Matter Expertise with the application and/or having a good hold on the domain of the application. Here testers try to break the application at business logic levels, and unusual flows are given more importance to discover corner areas of the application.

Black Box Testing Strategy

So far, we have seen how Black Box Testing and its techniques are used in functional testing of the application. But again, for Black Box Testing to become successful, there are some strategies to be followed.

  • Make sure that the requirements are clear and are covered in test cases
  • Test with the determined inputs at every level of the feature
  • Do not assume anything. For example: assuming the input A will result in the correct results and leaving it untested may result in huge operational cost at the later stages in the project
  • Always test with the huge amount of inputs and data to ensure the robustness of the application under any conditions.
  • Data range has to be strictly tested with BVT
  • 0 – the must test input if the text field accepts only numbers
  • Logics, transactions, state transitions, use cases – has to be validated several times with different inputs and all possible alternate flows
  • Recommend automation for wide-range of inputs and repeated tests
  • Cover all positive and negative tests for each cycle
  • Do not break the testing flow when Use Case / State Transition testing is being carried out. Complete the test in one go.

Conclusion

Black Box Testing is one of the most powerful methodologies that every project follows and recommends. The final goal of this testing is to make the application error-free and enhance the quality as this tends to figure out:

  • Missing requirements
  • Ambiguity in requirements
  • Usability issues
  • Concurrent and session issues
  • Complex Data logics

I have been in software testing for a decade. Maintaining HelpingTesters.com website to make a common platform to share my knowledge with everyone or where others can share to help out other testers. If you are willing to help than let me know at info@HelpingTesters.com

About the author

HelpingTestersTeam administrator

I have been in software testing for a decade. Maintaining HelpingTesters.com website to make a common platform to share my knowledge with everyone or where others can share to help out other testers. If you are willing to help than let me know at info@HelpingTesters.com

Leave a Reply

Your email address will not be published.