4 Cases When A/B App Testing Is Beside The Point

A/B testing (in other words split testing) is one of the most popular marketing tools. Yandex, Google experts advise to use it in order to increase the functionality of your website or mobile application, to make it more understandable and interesting to users. Nevertheless, there are some cases the using of this type of testing is inappropriate and beside the point. Let’s talk about them.

Case #1. When you don’t have enough time for planning.

When you are going to perform A/B application testing, you should bear in mind that it is not the simplest and the most trivial thing.

First of all, it requires a considerable amount of time. It needs the planning of the test procedure, programming various variants and pushing your application through a play/app store. Then you should wait for the customers to start using your application so that you could get proper results. Of course, if your A/B testing platform is equipped with visual editor, it can save you some time, but still you will not be able to avoid completely some planning and waiting.

There are some situations when you don’t have enough time for these things (it happens when you have some significant changes in the design or you are going to launch a new feature). In these cases the using of A/B testing will not be appropriate and it simply will not pay off.

Case #2. When you don’t have a proper goal or just don’t know what to test.

Very often A/B tests fail because there is no clear goal behind them. That is why you always need to know what you are testing. For example, try to use this test to check the following hypotheses:

• Will this picture help to increase the conversion, if you add it to the landing page?
• Are people more likely to click on the blue or the red button?
• What happens if you change the name, stressing that the offer period is limited?

The effect of all these changes is possible to measure. You can face great challenges while running A / B testing, when your goal is too vague – for example, when you are testing two design options with a variety of differences. This can last for a long time until you find the clear winner. Besides, conclusions can be inaccurate and you will still be uncertain as for the cause of an increase of conversion.

Case #3. When the number of your users is not enough.

The carrying out some significant experiment and as a result getting some crucial data points requires a large amount of customers, participating in your experiment. Depending on the structure of the text (how many variants does it have) and the results you are expecting to get, you may need approximately a thousand of customers. If you don’t have enough users, try to pay your attention on the activities that may help you to bring users, for example on building up some new features that your customers will definitely use. When you have enough users, you will be able to run A/B test without any difficulties.

Case #4. When the object of your A/B testing conflicts with your brand.

In these cases we are talking about the process of testing some design elements. For example, if you are going to test the new logo of the company that had established for many years can confuse your customers. The conversion can be increased in the short period of time, since a new logo attracts customers’ attention but it can be catastrophic in long term to test some significant or crucial changes of your brand.

Since the A/B testing (split testing) involves sequential study of the influence of various elements on the users’ behavior, you may require a lot of time for running it. Try not to use it when you don’t have a proper goal or enough users, since in these cases the using of A/B testing will not run off.