The retail world is decidedly digital. Our clients have shared a challenge it is to manage product data content at e-commerce retailers to ensure that it is compliant, effective, and helps drive sales. We hear you! And we have chronicled your challenges in our e-book that addresses Ten Pain Points in E-commerce to Overcome.
Challenge #8 is A/B is not A – Z
Quant A/B testing is limited and inadequate. Retailers dictate what you test. It takes too long and leads to lost revenue. Isn’t there another way to define product image potential?
The concept behind A/B testing, or split testing, is relatively simple: If you want to test multiple versions of messages for a webpage, e-mail, or other digital marketing, you can post in real-time more than one live manifestation, exposing half of your audience to one and a half to another, and decide on the “right one” based on which delivers better performance based on the desired outcome. On the digital e-commerce shelf, the best performance would be, of course, a purchase.
A/B testing can be used to test price, copy. And images, which are the topic of discussion in this article. But there are things you should know before embarking on the A/B testing journey to optimize conversion rates. A/B testing is often compared to “multivariate testing,” which means you can test multiple variations of an offer, not just “A” versus “B,” but that is when things start to get super complicated. To do this requires A LOT of traffic on an e-commerce site to test and take lots (comparatively speaking) of time to reach any conclusion.
There is a pretty extensive guide to all things A/B in a blog post by retail commerce platform Shopify. However, as with any data analytics, A/B testing can be cumbersome, time-consuming, limited, and risky.
The key takeaway? A/B testing has had its place in measuring certain aspects of digital effectiveness. However, there is a myriad of variables in A/B testing that require an accomplished statistician. At the end of the day, in the fast-paced world of e-commerce data management, where e-marketers are not necessarily data scientists, A/B testing for decision making for optimizing deficient content is anything but agile.
To unpack this further, let’s take the details of this challenge one by one:
Retailers dictate what you can test
Let’s take Amazon as an example. To create tools that help manufacturers manage PDPs, self-proclaimed experts like Page.One offers platform solutions for managing A/B testing experimentation. But they warn that “Amazon limits your ability to run experiments by only allowing you to run them on ‘high-traffic ASINs.’” (As a reminder: Amazon Standard Identification Number (or ASIN) is a ten-digit code that identifies products on Amazon. It’s unique for each product and is assigned when you create a new product in Amazon’s catalog.)
Page.One warns that you must have other prerequisites in place to determine your ASIN’s eligibility based on traffic, which is somewhat opaque and variable.
“When you have A+ content up and running for a while, that’s how Amazon will determine if your ASIN has a high enough traffic volume to run a statistically significant experiment. They don’t tell you what their specific standards are, but apparently, they vary by category.”
And that is just Amazon!
It takes too long
The point about critical mass / high traffic is addressed in the section above already. Also, Shopify powers smaller e-commerce sellers, not big retailers like Amazon or Walmart. The point stated above about High Traffic ASINS is more relevant. Instead would focus this section on the points mentioned in the second paragraph- it takes 8-10 weeks on average, which is a long time.
The Shopify guide tells us if the PDPs you wish to measure receive the volume of traffic required to run a statistically reliable A/B test, it will take weeks, if not months, to capture the data. In the other e-commerce challenges, we revealed that the average e-commerce marketer is fixing 600+ pieces of deficient content per day on average. So, decision support ideally needs to reflect the velocity of change inherent in the marketers’ challenging day-to-day responsibilities.
“You can run experiments for 4, 6, 8, or 10 weeks. Amazon recommends 8 – 10 weeks. But don’t worry: You can adjust the schedule or turn off the test while it’s running. After about 1 – 2 weeks, you’ll start to see data.” Says the folks at Page. One.
AND it takes as long as 7 business days to see if you even qualify before you can get your test up and running!
So as seductive as A/B testing might be, it will not address the speed marketers need to react.
It leads to lost revenue:
Over the life of an e-commerce A/B test (and we now know it is measured in “weeks and months” – not days), it is easy to envision that one or more of the variations you test will lead to a shopper de-selecting your product. It might be something as concrete and easy to identify as price. Still, it could be the imagery on the PDP, which we have already determined is much harder to test for the ability to drive conversion, which means you will lose that sale and the associated revenue – and you won’t know why! (More on THAT in next week’s post.)
So, if A/B testing doesn’t give e-marketers sufficient data to identify deficient images and the ways to correct them at the speed at which optimization will make a difference in shopper growth, is there an alternative?
The Solution? Flash.PDP™!
Behaviorally is the leading digital partner to help brands drive shopper growth. Knowing these challenges exist for our clients on e-commerce teams who want to win in digital retail, we developed a solution that leverages visual recognition AI, our extensive database of shopper marketing content, our unique behavioral framework, and decades of category expertise—introducing Flash.PDP – an always-on alert system to identify and optimize product images on the PDP that will convert to sales and drive shopper growth. It addresses category and retailer-specific metrics that provide easy, efficient ways to monitor and optimize images that will lead to increased sales.
To learn more, contact a Behaviorally digital retail expert today here!
