When we talk about media, there are a lot of things that come to our mind, especially how much will be invested in a digital campaign to be successful. We think about the target audience, the channel, the language, the strategy, the tactic, and more. A lot of marketers, clients, or creative teams have a clear idea of which creative they are going to use, but others have more than one creative and want to do an A/B testing. Normally I try to recommend to not run completely different creatives if we are going to do an A/B testing. If we run to different creatives then it is just to different videos, ad banners, or audio ads in rotation. Yes, you can compare which one did better between those two, but you will never know a clear answer of why it did better since both have different elements across the whole ad. We cannot only do A/B testing by different ads but also test the same ad and serve two different audiences. In this case, I want to focus on the ads side of things.
I’ve been running a lot of A/B testing where the client wants to test two similar ads where it only changes one element. For example, I have a client who is a restaurant, and they have a happy hour, but the only way to enjoy it, is by making a reservation. This client wants to do video ads across the web to advertise the happy hour. We started discussing how the ad should be and how the customers should make the reservation. We brainstormed for a couple of minutes until we agreed that we should try to get those reservations by calling the business and/or filling out an online form. What did we do? We created two different video ads where one is going to have the restaurant phone number big enough for the customer so they can call to make the reservation. The other ad will have a QR code that the customer can scan and will direct you to a form fill that needs to be completed to make the reservation. After the campaign ran for a month, we wanted to see performance and compare between those two ads, which one did better. We started analyzing the data and we found out that the Ad with the QR code drove more reservations, but the ads that had the call tracking number had reservations with bigger groups of people. We started questioning ourselves if the ad with the QR code worked better than the ad with the call tracking number. You will think, yes, since it drove more reservations. The problem is that at the end of the day, we are looking at money generated. Because of this, I started looking for a solution or explanation of the data recompiled, and a colleague recommended a website called SocSciStatistics.com. This website contains a lot of different tools and calculators that help you analyze data in the social sciences. I started looking at the calculator and my colleague explained to me how I can use the Chi-Square Calculator for 2×2 to analyze the association or differences between ad A and B since the data is presented in numbers. In other words, if the data recompiled tells us something or there is not much difference to think of a clear conclusion. SocSciStatistics.com explained it as an association between two categorical variables. The equation or formula you must do to find out that is the following:
SocSciStatistics.com contains the calculator to make this formula based on your data.
Follow me to learn more!
Leave a Reply