• info@helpingtesters.com
  • helpingtesters

Complete Guide to Test Cloud based SAAS Application

January 2, 2018
SaaS Applications, SaaS Application, Cloud based SaaS Application, Challenges in testing SaaS Applications, Automated testing tools for SaaS Application

We use online services on a daily basis. It can be as small as sending an MMS, reading an email, listening to music, playing a game etc. All these online services are possible because of cloud computing services. Today’s topic is all about knowing SaaS applications and how we can test these type of applications. But before learning about SaaS application, we need to know what cloud computing is. Let’s get started.

What is a Cloud Service?

Cloud service or cloud computing means all the services i.e. the servers, storage, databases, network, application, reporting etc. of an application are hosted over the internet. The companies which offer cloud computing are referred to as cloud providers.

What can be done by Cloud computing?

Following tasks can be accomplished via cloud computing:

  • Create new apps & services
  • Store/back up data
  • In case of disaster, recovering the data
  • Deploy applications
  • Video, audio streaming
  • Analyze data

Type of Cloud Computing services

There are 3 types of cloud services available.

  • Infrastructure as a service (IaaS),
  • Platform as a service (PaaS)
  • Software as a service (SaaS).

Out of these three, a software as a service (SAAS) model is the most talked about.

What is a SaaS application?

In simple terms, any application that is hosted centrally over the internet (as a service) is a SaaS application. A third party usually hosts the application and thus consumers can access them easily via their web browsers which they might be running either on a computer, Mobile or any device that’s connected to the internet.

SaaS customer really likes these sort of applications as they don’t need to buy expensive hardware or software for buying, installing, maintaining or upgrading the application. Also, there are no worries about data loss since the application is hosted in the cloud.

How SaaS model helped business vendors

Before SaaS applications existed, most of the applications were hosted on a client-server infrastructure or at clients/customer end. If the application needed to be upgraded, it would be upgraded on all the servers where the software was already present.

SaaS eliminates the installation cost and made everything centralized. The need for additional hardware is also reduced as all infrastructure is managed by the SaaS service provider.

How can SaaS applications be accessed

The SaaS applications are provided using a SaaS application server. SaaS services are offered on a monthly subscriptions package by the provider. They are also known as an application on demand when you will request the connection will be established.

Testing SaaS application

A SaaS application testing demands functional, non-functional and multi-tenancy testing with additional cloud infrastructure testing using the cloud components. The core multi-tenancy components are an application, infrastructure, and network.

In addition to this, user experience and customer experience both need to be tested. By this, I mean testing the ease of access towards the consistency of the application. Ease of access will be, a user able to successfully navigate within the application. Consistency will be a user successfully completing a transaction. SaaS applications testing is more complex and harder than the traditional testing approach as continuous rigorous testing cycles are performed.

How Agile methods benefits in SaaS testing

In order to achieve fast results, the agile testing methodology is used in SaaS applications which means a use of automated scripts is a must. For the regression testing, you can choose and create test scripts that will validate and verify the data on frequent deployments.

Challenges for QA in SaaS Applications

There are various challenges which should be addressed during testing as one change might be affecting hundreds or even thousands of users that will be using the application. Following challenges are faced by the QA teams while testing SaaS applications

1. Frequent Release deployment

This is a very complicated challenge as users from anywhere in the world might be using the application in any time frame. To cater this requirement being a tester you need to use automated test scripts so you don’t have to entirely test the application manually.

Prior release deployment it’s always best to inform user’s that the system will undergo a maintenance upgrade and you might feel some inconvenience. Or you can totally make the application offline for defined hours and after full upgrade and stability checking, make it online. The second option is no recommended and an application must never go offline.

2. Security testing

Security monitoring is one of the major concerns organizations feel uncomfortable with while opting for cloud computing. As the entire data will be hosted on a third-party data center. The security of a SaaS app is very much affected by the deployment model which can be:

A Public cloud, a Private cloud, a hybrid cloud and a Community cloud

  • The first step in security monitoring is adopting a risk-based cloud model which will help in identifying the most appropriate model.
  • The second step is to negotiate with your provider to implement security audits, security certification, leverage standards

Being a tester you need to make sure the application has strong algorithms implemented for user authorization and each transaction is fully encrypted.

A simple example will be checking the view source of the web page, search for username and password field. Is it encrypted? Or it’s shown in plain English. If the application has different roles, check each user is shown the assigned privilege and the access rights are not crossed by the users. Test for typical web application security attacks like SQL injection, HTTP header injection, cross-site scripting.

3. Network Security testing

All data over the network needs to be secured. Test, whether a strong encryption is applied to the connectivity protocol i.e. SSL (secure socket layer) and TLS (transport layer security), is applied over the network and no sensitive information is breached. Test for MITM (man-in-the-middle attack), phishing, and packet sniffing attacks.

Firewalls and antivirus can be installed to fight against any unwanted security threats, spyware, and malware. If a firewall is applied, you need to make sure that packets are not getting dropped or lost by being blocked by the firewall.

All these needs to be tested on the network layer, you can take help of a network engineer and diagnose all the breaches. Use automated tools like Acunetix , Wireshark, Nessus, John in Ripper, OpenVas, OpenSSH and so many others which can help you scan all sort of vulnerabilities.

4. Integration testing

Whenever the new modules are integrated with the previous modules, an entire integration testing is required to be executed. You need to think out of the box and perform boundary value analysis to identify each existing and new modules are stable.

5. Data migration testing

If your cloud provider is changed from provider 1 to provider 2 for whatever reasons you have to test data migration activity. Being a tester you have to identify all the URLs are correctly mapped and the users are not shown 403, 405 or 404 redirection errors. Once the new feature is rolled out its time to migrate the old data on the updated infrastructure.

6. Performance, Load, Stress testing

This is the backbone of SaaS applications and needs extensive testing. Being a tester you need to make sure the application behaves and performs same on multiple devices and a large number of users. First, test the application front end/client side end and optimize as much as you can. There are many online tools available to optimize the application, for example, GTMETRIX.com, ySlow etc. which analyzes your website and suggest all the errors and how they can be fixed.

After the application layer performance is optimized you need to make sure the backend does not have any memory leaks, open database connections etc. For this, you can load test the application using the available load testing tools. For example LOADRUNNER. This tool provides the most powerful reporting and diagnostics and has the most user-friendly environment / IDE.

After load test is passed, stress test your application i.e. pass additional load then the application can bear and analyze how it is responding.

Once you are done with load and stress testing, you need to volume test your application. By that I mean you need to pass extra load towards your database. It will help analyze your application request processing time and stability on system degradation. Test if the data is getting stored in high volume and is not overwritten or the referential integrity is not disturbed. Test the error messages and warning information is shown once heavy load is pro

7. Privacy testing

The data privacy policy for each country varies and each country follows their own regulations, if the application is hosted on different countries cloud, you need to make sure the data integrity is not compromised. For this standard are made, make sure all the standards for the defined country is followed.

8. Interoperability testing

Every SaaS application must operate seamlessly in any type of environment. May the user be from a mobile device, a windows machine, a MAC machine, a Linux machine or from any browser. You need to make sure that each component can communicate with other devices without any compatibility issues. Being a tester you have to verify on each device, machine which seems to be quite impossible. For this, you can use many automated testing tools.

9. Backup

Due to frequent release deployments backing up of data becomes a necessity. For this cause, we need to make sure versioning is done and a copy is placed on a separate server so you can always roll back to the previous version, a stable state in case of application flows break down.

A cloud to cloud backup solution is provided by the SaaS provider. You can also be in contact with a backup provider who can suggest you more ways to restore and backup data.

How to overcome SaaS apps testing challenges

  • For the continuous release upgrades, automated test scripts can be written and used as regression.
  • Test the most used functionality, area of the application for e.g. composing an email will be less likely in comparison with reading an email. You can test whether all the simultaneous users are able to retrieve data from their email inbox or not.
  • Prepare a testing plan so it can be followed in each iteration and none of the requirements are missed out after the upgrade. The complete scope should be mentioned.
  • Define performance benchmark. It will benefit the team to stabilize the application at least on the listed number of users.

Top known SaaS applications

Let’s have a look at the existing top SaaS applications. I am sure you must have used at least one of the following:

1. Google Apps

I can guarantee all of you must have used at least one service from Google whether it’s your email account, Google maps, Google hangout, downloading applications from the play store or any other. We have been using Google for a long time. All the apps offered by Google are the best example of a SaaS application.

You can use these applications on the go, the data is auto backup. You can also save the data and use it offline. If you accidentally lose your Android device, all the data is synced up and can be easily restored with a single click.

2. Microsoft Office 365

As Microsoft state, this particular suite is built to help you achieve more. MS Office 365 is an on the go solution. Whether you are in your office or outside anywhere you can share, edit, and create your files from your PC, MAC, Android, iOS or Windows device with anyone in real time.

3. Zendesk

Zendesk is a customer service platform. It helps by providing the organizations create a relationship to support and manage their clientele. One of the biggest features is the ticketing system and the chat application.

4. Dropbox

Drop box provides you free storage and brings all your files together in one place. You can add any type of file and share it with users across the globe. It provides an automated sync feature too keeping all your data safe and usable from anywhere.

5. Slack

Slack is a digital workspace used to communicate with each other. In easy words, it’s a mixture of Skype and team foundation (well that’s my personal opinion, yours may vary).

Cloud-based Automated Testing Tools

As we have automated testing test tools for functional testing, UI testing, performance testing etc., there are special cloud-based SaaS applications testing tools too. Cloud-based automated testing tools are considered far productive than the automated tools. Let’s see how?

Cost effective – We don’t need to buy expensive per seat licenses hence decreasing the capital investment. It does not require additional hardware so hardware cost is minimized too.

Quick Testing – We don’t need to invest more time in installing and configuration of the tool. The cloud-based tools are easily deployed and accessible anywhere from the world.

Test collaboration – as everything is in the cloud, you can easily collaborate with the geo-located teams. The test results, reports everything is accessible from anywhere so you don’t have to put in the efforts of uploading them to a location and sharing.

In comparison with the availability of open source automated testing tools, SaaS automated testing tools are less. You have to use a paid tool.

Few of the automated SaaS testing tools are:

  1. PractiTest
  2. TestingWhiz
  3. PushToTest
  5. Cloud testing
  6. uTest
  7. QMetry
  8. CloudQA
  9. Google Analytics

Disadvantages of using SaaS

While we have discussed so many advantages of using this workflow above, there is risk associated with it too. Let’s discuss the disadvantages or the options that can’t be tested in SaaS applications.

  1. White box testing is not possible.
  2. Verification of test results will be limited to front end only.
  3. Limited database access rights are provided.
  4. The problem in identifying whether it’s a code defect or a configuration defect.
  5. No or little control over the scheduling of jobs.
  6. The communication gap between the testing team and the service provider.
  7. Open compliance and standardization issues.


Concluding today’s topic, SaaS applications usage is trending day by day. Many businesses are opting it to enhance collaboration, scalability, availability and agility opportunities. This directly affects the existing QA teams as the extra responsibility to assure quality in terms of user experience and platform both are required. We being a tester should never stop learning and should explore all the new ways to manage, test and break the applications.

About the author

Aisha Mazhar author

Leave a Reply

Your email address will not be published.