Understanding Performance Testing, Load Testing and Stress Testing

It’s the end of the year, most of the times people will do these 2 things, shopping and travel. I like to explain things with real life examples. Set your thoughts on shopping and travel, let’s look at these scenarios.

Scenario 1:

You own an ecommerce/marketplace app, you are also doing crazy promotions and sales because you know people will go shopping for Christmas, Boxing Day, 12.12.17 sales, etc. You have massive traffic going to your app, more than your system can handle. Soon enough your app is down due to heavy traffic.

Scenario 2:

You own a travel/hotel booking app. People travel during end of the year, holiday bloom time. Your app is busier than ever with people searching for hotels and comparing prices. You have one user trying to make a booking, he did the normal business process for booking, selected a date, made the payment etc. At the moment he finished making the payment, upon redirecting to your app from the payment gateway, your app crashed. The user received the confirmation email etc, he thought that his booking went through without any issue. Vacation time, the user went to the hotel, showed the receipt etc, but unfortunately, the hotel said that there was no booking made. That booking was not even captured in the hotel booking list. The hotel will surely have said that it’s the app issue.

Impact

The scenarios above are real scenarios. First thing, your user will be very upset, he/she might leave a bad review and comments on your app and share it to social media. Worst is, it becomes viral! This will surely tarnish your reputation. The hotels and vendors on your app will upset as well and worst thing is, they don't want to use your app or be your partner anymore!

Your users and partners will go to your competitors and you will lose your market share! Due to bad reviews on the app marketplace, prospect users might get scared-off to use your app

As for you, the first thing that should cross your mind will be “this is a performance issue”.

Do you know that ‘performance issue’ is very broad and general subject and you might be doing the wrong testing and getting the wrong results. Your test might show that everything is fine.

Understanding Performance Testing.

Performance Testing

First, let’s understand what Performance Testing is. Performance testing validates the performance of an application/system and involves measuring the execution speed of the application. It’s a type of Non-Functional Testing. The objective of this test is to ensure that your app meets the required performance level. Most of the time people are confused between Performance Testing and Load Testing.

Types of Performance Testing

Diagram below shows the types of Performance Testing but in this article I’ll be covering Load Testing and a poking a bit of Stress Testing.

Types Of Performance Testing

Load Testing

As mentioned above, people are often confused between Performance Testing and Load Testing. Load Testing is not a performance testing but it’s a type of performance testing. It involves measuring the usage level that the application can sustain. In Load Testing, we’re trying to see whether the performance data we’ve collected during performance testing is still valid with an increasing number of the load (users). So, do you see the difference here? In performance testing, you validate the performance of the application regardless the number of the load while Load Testing is basically validating the performance of the application with the real-world load. This is where most of the application lack. They thought they have covered the performance testing but they left out the most important part of it, which is the real-world load!

Stress Testing

The easiest way to say this is basically Stress Testing is similar to Load Testing only that for Load Testing, you test your performance with the number of load, gather your data and stop. Stress Testing your app means you keep doing it, adding loads on your app until you break your application. You want to understand the limitation or the breaking point of your application.

Few things to consider for Performance Testing

I can say that Performance Testing is quiet a complex testing. It’s requires careful research to be done before you can start testing (including the type of performance testing to execute).

Precise and concrete goal/objective

When considering performance testing, you need to know precisely what you want to measure and how you want your app to be. Normally, people will say things like this:

  1. Room search should be fast.
  2. During the peak season (e.g. Christmas - 15th-23rd Dec) and peak time (e.g. 12pm - based on your information gathering), add to cart should be working as usual.

Above objective is too subjective and it’s impossible for you to measure it. My fast might not be fast enough for you. When you say heavy load, do you mean 10, 100, 1000, 10000?

A clearer objective/goal should be something like:

  1. Room search functionality should respond in X.XXs upon user clicking submit on XXXX page.
  2. 10,000 concurrent users on the application should be able to use the add to cart function without any issue during peak time (12pm)

Monitor setup

To get precise results, you need to know what you want to measure and how/where to setup the monitor. Incorrect monitor setup will get you wrong results. E.g. setup over firewall or etc.

Test Environment

This is very hard to get. Based on my experiences, this is a very tough request for you to get. Your test environment must mirror live environment as close as possible. If it is not same or at least as close as possible, your test result will be misleading and you can’t precisely identify the bottleneck and check where the issue is.

Hopefully, you get the idea now. What is Performance Testing, Load Testing and Stress Testing. Above are only a few things that you need to consider before the execution of Performance Testing. There are much more to it such as tools and resources etc. Performance Testing is a complex type of software testing and requires thorough research and the right people before you start the testing. The correct planning and objective will give you the right results for you to tweak your application/system while the wrong planning and execution can turn out to be a disaster.

Happy New Year and Happy Holiday!

Kamil Khaidir Roslan
Author: Kamil Khaidir RoslanWebsite: http://www.bugsdetective.com
Test Enthusiast & Co-Founder of Bugs Detective
A passionate software tester with extensive experience in software testing. Involved in various type of testing with various type of clients, locally and globally and also involved in giving training on software testing methodology and enterprise tools.

Leave a comment

Comments

  • No comments found