• info@helpingtesters.com
  • helpingtesters

8 Reasons For Automation failure

March 20, 2017
automation failure, automation fail

Automation seems like a magic word to testers, something which will do wonders to those looming testing deadlines, and make the management happy with skyrocketing ROIs.  But some automation projects become a big success, but many fail miserably. Some of the common mistakes found in all these failed projects are as follows

Tightly Coupled Script with the Tool

Testers, usually make this mistake where the script they create are tightly integrated with the test automation tool they have used. And the if ever the management decides to change the automation tool, let us say shift from open source solution to a commercial solution because the client asked, the cost of this cross-over is as high as creating a new automation suite or maybe even more, considering the skill set of resources needed for it.

So the best way to avoid it is ensuring that have a modular approach towards programming, and build your scripts independent of the underlying test automation tool. Use the concept of a driver script, which sends commands to the test tool and gets the result back. So in all, you will require changing will be the function library which is tool dependent, for the tool specific function calls. Rest most of your implementation remains unaffected by this.

Test Data

Test data is a crucial input for automation runs.  We need it before actual test cases are put to execution so that they can be consumed during the run. Generally, after a single run, the test data is exhausted, and we require to be generated again for another run. So this is a continuous activity for every test automation run.

Using the UI of the application to generate the test data as a prerequisite is another mistake testers do. They don’t realize that this causes an unnecessary prolong test automation execution time, which can be avoided if possible to generate data without using the front end UI actions, for example, if they could call a web service, do that or run some SQL queries, then write code to call such things directly.

A lot of applications are now developed where after an UI action a web service is called which then talks to the database, so for generating test data, we can bypass the UI and call the service directly which will be faster, and independent of the bugs of UI with every release. The same holds true for writing code to execute the database queries directly to generate test data. We need to understand our objective is to create test data and not to test the scenarios.

Increasing the test automation run, by these unnecessary baggage delays the test cycle and thus impacting the ROI negatively. So such areas shouldn’t be overlooked.

Overlooking Object Definition Change

Until and unless you are working with a commercial tool, which provides a feature for Object management, which takes care of the objects of your application, and ensures that the impact of the change is minimal in your test scripts, the testers usually overlook this when working with open source tools.  And then they wonder,  why their test automation suite failed when the new software build came.  So we need to ensure that the scripts are created in such a manner that the changes in the application UI causing the object information to change has minimal impact on the modification needed to be done at the script level.

Informing Stakeholders

There are instances when the automation is implemented in the project without proper approval from management. We need to understand that it is a time, resource investment activity and not something which can be started on a whim without proper buy-in from the management. So it is vital that proper authority has been informed about this activity and their approval is available to initiate it.

We should also understand that test automation is an activity which requires a deeper involvement of the development team, unlike in manual testing. The change in the application due to development work has a direct impact on the test automation suite. So they become direct stakeholders in this task, and keeping them in a loop for this activity and requesting the same from them is mandatory.

Wrong tool

We don’t realize that a proper selection of a tool for test automation is vital. It cannot be overlooked, and some quality is required to be spent on it. We need to carefully analyze and filter out important scenarios for the proof of concept with the tool. The application compatibility with the tool should be noted, how objects are being recognized, how much time it is taking for the execution, which should be significantly less than the time taken by the manual effort. Only by with proper and careful consideration and keeping in view long term the tool should be chosen.

Inexperienced team

We cannot just ask one fine day our manual tester to learn code and become a test automation engineer. It is the sure shot recipe for disaster, in terms of losing a great manual testing resource by scaring them away and if they don’t run away converting him to be a poor automation engineer.  Automation testing team should be a team dedicated to this activity. They should understand and have good knowledge of coding at the same time understand the testing perspective. Rather we should see automation engineers as coders who have domain experience in testing, instead of searching for testers who know coding.

 Bad planning

Automation is not an adventure activity where you are seeking an adrenaline rush by going without a concrete plan. Also, it is not a stand-alone activity. Many times people fail to recognize automation as an ongoing effort. They fail to understand the planning required for proper maintenance, the tools required for it as well. We need to understand that automation cannot exist alone. There has to be integration with an ALM technology, with a bug management tool, with a CI tool.

High expectations

This is a classic error. We all need results and we need them fast. We fail to realize that test automation ROI can be expected to show a higher curve only by 3-4 test cycle runs, by the time when everything stabilizes. It cannot produce positive results overnight, and expecting magic from it, is a sure shot way to disaster.  We need to be careful and understand we are generating code to test another code which is ever changing. We need to ensure that the automation code we are writing is robust and maintenance cost of change with every test run is not high. We do not want to spend the time for testing on automation suite correction but to actually test the application.

Above are few of the common mistakes people make while implementing automation in their projects. We should understand that test automation activity requires proper planning, execution, dedicated resource team, time and buy-in from management. Only then it can give us those magical ROIs, will save time, cost and improve test quality and faster go to market. All we need to do is be careful and avoid such common mistakes and make automation productive for us.

About the author

Leave a Reply

Your email address will not be published.