I can't login to my app after the new update!

John is a frequent user of app ABC. He uses this app whenever he wants to do his online shopping. Now, he wanted to buy some stuff. As usual, he will open app ABC and make his purchase there. Unfortunately, after the new update, he can’t even login to that app. After trying for the 3rd time, he just gave up, he left some bad review and comment on the playstore due to his dissatisfaction. Since there are a few similar app out there, John decided to download a similar app, which is app XYZ and make his purchase there.

Now imagine you’re the founder/owner/CTO of app ABC. You check the review, there’s like 20 similar comments like John and gazillion of email to your customer support due to this issue. You wonder why, because this new update is supposed to fix previous reported bugs, not to add more bugs and due to this issue, you might:

  1. Get a lot of bad comments and ratings that will decrease your app rating.
  2. Lose a lot of revenue for that day - until your team manage to fix it.
  3. Lose customers retention and acquisition. Jeopardize trust and reliability.

Imagine at that very moment there’s a prospect investor evaluating your app. After seeing a lot of negative feedbacks about your mobile app, what would they might think?

What works before must work now!

First of all, we need to understand that there are various type of testing and one of the most important one is Regression Testing! So, what is regression testing?

Regression testing is re-running functional and non-functional tests to ensure that previously developed and tested software still performs after a change. Changes that may require regression testing include bug fixes, software enhancements, configuration changes, and even substitution of electronic components.(wikipedia)

The easier way to understand this is, What works before must work now! Whenever there’s a new update to your app, either a new functionality, bug fixes, enhancement, etc, you need to go through a series of test to ensure that it’s doesn’t break the previous functionality or flow.

My team has done the mobile app testing prior to releasing it to user!

Let’s take a look at below figure:

Regression Testing

Let's say that for this new release, you’re releasing the bug fixes for the green box in module 3. Most of the time:

  1. You will only do testing on that bug fixes on the green box. OR
  2. You will do testing on everything in box 3, hence you missed the overall end to end flow of your mobile app user journey.

Without having a thorough regression testing on you mobile app, you won’t know that by fixing that green box bug, does it introduce new bugs somewhere along the end to end process as per figure below?

Regression Testing 2

But, we’ve done the full regression testing for our mobile app prior to the release, but we didn’t find those issues reported by our users.

There are a lot of possibilities to justify the above statement, some of it are:

  1. Does your test environment mirror your production environment? (E.g. Hardware, Code, 3rd party integration, etc)
  2. How did you do your regression testing? (Simulator/Emulator? Real devices?)
  3. Did you do your regression testing only on one device? Is that device the same device your users are using when they encounter the issue?

The list goes on.

I think it’s OK. We can release it first and let the users report the bug to us. We can manage the damage later.

I believe I’ve mentioned on most of my previous posts, the cost of fixing bugs after it goes live can go as high as 30x more expensive than the bugs found during development*. As you can see on the figure above, the complexity increases when it’s already live. Not to mention, the impact on your business as well as reputation.

 Regression Issue comment

Let’s say that your user gives a feedback like this. What can you do with it? If you give back to your tech team, what can they do with that? In order for you to fix a bug on your mobile app, you need to understand various information and data that I believe your users won’t provide, but an experienced tester will provide that to you. E.g.

  1. How to reproduce the bug? How many time it occurs?
  2. What type of device that you use? OS version?
  3. Network type?
  4. The state of your mobile device prior to opening the mobile app?

Again, the list goes on and on. As I mentioned above, normal users or even your beta testers might not be able to provide this kind of report to you. This needs to be done by a professional tester.

Still think that mobile app testing is just a matter of tapping screens? Regression Testing, Mobile App Testing or App Testing in general requires a lot of planning, strategy and the right approach. Without the right expertise, you might potentially missed a lot of bugs and release a buggy app which eventually will make you lose your users, and your revenue as well! If you would like to understand further on how Bugs Detective can help you on Regression Testing for your mobile app, please do not hesitate to contact us We would be very happy to assist you in any way possible. 

*IBM Development Solutions Whitepaper

Free basic web app testing tool

Kamil Khaidir Roslan
Author: Kamil Khaidir RoslanWebsite: http://www.bugsdetective.com
Test Enthusiast & Co-Founder of Bugs Detective
A passionate software tester with extensive experience in software testing. Involved in various type of testing with various type of clients, locally and globally and also involved in giving training on software testing methodology and enterprise tools.

Leave a comment

Comments

  • No comments found