“Today, A/B testing is thriving — it’s been a huge improvement from non-A/B testing,” said George Khachatryan, CEO of AI company OfferFit, in a recent webinar. “At the same time, the people performing these tests every day recognize that’s it’s a lot more difficult than it may seem.”
Designing A/B tests, determining samples sizes, and deploying them takes up a lot of time and resources, and analyzing the findings requires high levels of precision. All in all, the manual tasks required by A/B testing can place a heavy burden on marketers.
“When you’re running a full experimentation program, it’s never enough to run one A/B test,” Khachatryan said. “When you run one, you gain valuable insights, and inevitably want to gain more. So you end up running more.”
He added, “Those who are doing this hands-on realize it just becomes a rapid explosion in the number of tests they need – it becomes infeasible very early on this exponential curve.”
Marketers need a solution that allows them to test a growing number of campaign variables while simultaneously giving them enough time to analyze the data. Fortunately, A/B testing is evolving.
Expanding the power of A/B testing
In the webinar, Khachatryan highlighted the “multi-armed bandit problem” that’s affecting modern A/B testing. In the traditional version of this scenario, a person at a casino must determine which slot machines (the “one-armed bandits” that steal your money) are going to have the best payouts, then figure out which order will be optimal. With A/B testing, the variables are the multi-armed bandits, and the marketer must discover which are most effective so they can allocate more resources to the areas performing well.
“You can think of [a multi-armed bandit] like a smart A/B test,” he said. “It’ll navigate the exploration-exploitation tradeoff — it’ll start randomly pulling those ten handles, but as it goes, it’ll dynamically reallocate resources so that if something looks bad it’ll stop pulling.”
He added, “These multi-arm bandits are designed to experiment just the right amount so you’re learning but also taking advantage of what you’ve already learned.”
While these multi-armed, or A/B, models have served marketers well over the years, there’s a new iteration of the framework that is more accurate and effective. According to Khachatryan, these are “contextual bandits.”
“It does what a multi-armed bandit does, but it takes into account different contexts,” he said. “So, if you have two different customers, with different characteristics, it’ll know to pull different levers.”
Contextual bandit frameworks are essentially automated experimentation and personalization at scale. It’s a model that can completely automate the process, and it’s what every marketer should be moving toward to improve campaign effectiveness at scale.
Get the daily newsletter digital marketers rely on.
Automated experimentation is the future
Many of the tech giants have already adopted contextual bandit frameworks, but marketers should note that this technology is still incredibly new. Brands should allocate enough time and resources to make the transition process easier, because, according to Khachatryan, it’s the “future of experimentation.”
“In the past, manual A/B testing works with one mention at a time,” he said. “With these contextual bandits, you can set it up to simlutantous test multiple dimensions.”
Whether it’s testing email subject line efficacy, call-to-action click-through rates, or optimal article posting times, marketers have a lot of experimental data to keep up with. Automated testing solutions can make these processes more manageable by decreasing the time spent on manual tasks, replacing them with continuous automated experimentation.
“You can think of this as the next iteration of experimentation, or test-and-learn programs,” Khachatryan said. “When a marketer sets up this system of continuous automated experimentation, it creates this interplay where you can see what happens, gain insights, and then use those insights to get new ideas.”
He added, “So you still have the agile test-and-learn cycle, but it’s accelerated.”
Time will tell how quickly marketers adopt these automated experimentation technologies. But, with the high level of marketing technology replacements that took place over the past year, there’s a good chance more brands will sign on sooner rather than later.
What they are. For today’s marketers, automation platforms are often the center of the marketing stack. They aren’t shiny new technologies, but rather dependable stalwarts that marketers can rely upon to help them stand out in a crowded inbox and on the web amidst a deluge of content.
How they’ve changed. To help marketers win the attention battle, marketing automation vendors have expanded from dependence on static email campaigns to offering dynamic content deployment for email, landing pages, mobile and social. They’ve also incorporated features that rely on machine learning and artificial intelligence for functions such as lead scoring, in addition to investing in the user interface and scalability.
Why we care. The growing popularity of account-based marketing has also been a force influencing vendors’ roadmaps, as marketers seek to serve the buying group in a holistic manner — speaking to all of its members and their different priorities. And, ideally, these tools let marketers send buyer information through their tight integrations with CRMs, giving the sales team a leg up when it comes to closing the deal.
The author’s views are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.
There are plenty of impressive tactics or metrics that aren’t often discussed, not necessarily because they aren’t important, but because it’s easy to get locked into the rhythm of simply reporting on traffic and sales.
To change things up, let’s look into some other areas we can optimize to improve the organic performance of e-commerce websites, and some underrated but useful metrics that can help you report on that performance.
Tactics to optimize and measure your e-commerce website performance
Data scraping for SEO and analytics
Data scraping is very useful when you want to retrieve, or scrape, elements from a page for further analysis or optimization.
Most people know that you can scrape common webpage elements such as publication date, author name, or price, but what about more specific aspects of e-commerce websites, and what can we use them for? Product pages have unique attributes that you can scrape, such as “add to basket” type buttons or even product schema; below, I’ll talk about how you can scrape breadcrumb data.
Scraping the breadcrumbs
In short, breadcrumbs are a trail that shows users where they are in the structure of a website, and they are especially useful for navigation and internal linking.
By using crawling tools to scrape data from the breadcrumbs, you can have a more complete view of the site as a whole, and it allows you to identify any trends.
Below, you can see that it’s possible to extract breadcrumb data as a series of values by using XPath, and setting this up as a custom field. This allows you to see the data as a separate field once a crawl is finished.
Evaluating your page templates
The typical page templates that you’d expect to see on an e-commerce site include:
Information pages (e.g. about us, delivery information, terms and conditions)
Navigational landing pages
Blogs / guides
Payment / cart pages
A large e-commerce website may have a significant number of product and category pages. These are the pages that generate the most conversions and transactions, so it is tremendously helpful to know how you can break these down into more manageable chunks.
For a website with millions of pages, it is practically impossible to crawl the whole site; your crawler will run out of memory and space, or it could take weeks to finish, and that’s just not feasible for most of us. This is where segmentation comes in. Segmenting your website also allows you to focus on one area of the site before moving on to another.
A common tactic for websites the size of Target or Tesco is to focus on one category per quarter, and then move on to another area of the site. It’s through segmentation that they’re able to do this.
Segmenting product pages
There are many different ways you can segment a website, and focusing on your products can help you start seeing improvements in revenue sooner than if you were to focus on other areas of the site.
With product pages, a good tactic is to look for URL patterns, such as those that end in .html or contain /product/.
It’s also possible to get additional dimensions from your product pages by segmenting your products by their stock status. Separating pages by whether or not a product item is in stock or not can help you determine:
How much traffic is going to out-of-stock products.
Whether availability and out-of-stock products are affecting product conversion rates.
Get a granular view of what page engagement metrics are affected by stock availability.
When scraping this data, you can look for specific on-page elements such as missing prices or an Unavailable / Out of Stock message on your pages.
One method of doing this would be to extract the product availability property from a site’s schema markup. If you’re using Screaming Frog, you can access the Custom Extraction feature in the Configuration dropdown under Custom > Extraction,and then set up your extraction rules.
Segmenting category pages
Segmenting category pages allows you to find any categories that have hundreds of products and could benefit from being split into subcategories.
Category pages don’t always have specific URL patterns, and they differ from one CMS to another, but you can look out for those that contain /category/ or /shop/. Another good option is to look for unique attributes, such as those with text showing X of Y results or pages with options for sorting product results.
Structured data markup
We saw earlier that you could scrape pages for instances of product data to identify product pages. But before we move on, we need to ensure we understand what structured data or schema markup is and how it can benefit e-commerce websites.
Product markup provides more information about your products directly in the SERPs when your audience searches for them. Product markup can also mean your products are more eligible for rich results, such as carousels, images, and other non-textual elements.
The product schema might look something like this:
Once added, product schema allows your audience to see valuable information about your products before they even land on your page, improving your CTR! We can see Walmart has added product schema to their products in the two examples below:
Star ratings in search results
The more positive reviews your products have, the more likely customers will be to visit your website and buy your products, especially when compared to your competitors.
Star ratings can be pulled in from your product markup through third-party tools such as Trustpilot or Reevoo, or from on-page customer reviews.
We see this when looking at these searches for Dell laptops. Realistically, which links are you more likely to click on as a customer: those with high star ratings or those with seemingly no rating at all?
Optimizing crawl budget for e-commerce
There will likely be pages on your website that are useful to existing customers, such as thank you pages after placing an order, logged-in account pages, etc. However, these pages won’t be the most important for new users looking to find you or your products on search.
It costs Google time and money to crawl our sites, so they need to budget accordingly. By managing this crawl budget, we guide search engines toward our most valuable and essential pages.
We don’t need to index every page on our websites.
It’s entirely acceptable to meta-noindex or disallow certain pages in the robots.txt file — in fact, it’s expected. This is because indexing everything could mean that Google might not crawl all of our pages, so they might not index all of our content. This would be a problem, as it could mean some of our high-value, top-converting pages might not rank organically.
That said, we shouldn’t be noindexing vast chunks of an e-commerce website without proper research. By noindexing huge chunks, we’re missing out on the ranking potential for key search behavior, e.g. locations, product sizing, etc.
Use of URL parameters
As users or owners of e-commerce websites, we’re likely familiar with URL parameters. Common areas that we see these parameters include:
Faceted navigation pages and product sorting options are typically blocked in robots.txt files, but it’s a good idea to find out how many of those pages Google is still serving to searchers. We can do this in our chosen crawling tool by selecting the option to ignore robots.txt rules. Alternatively, you can segment landing page session data in Google Analytics by URLs with parameters to see how many of those parameter pages are being served to users. Then, the session data will be used to show how many visits those pages are getting.
It may seem counterintuitive to do this, but these pages tend not to have unique on-page content, as they will have duplicated titles, headings, or body content, which means you could be missing out on other, more essential pages ranking for relevant keywords.
Measuring site speed across templates
With large e-commerce websites, it doesn’t make sense to simply test one or two pages and take that as a site speed reading across the entire website. Each page template is built differently. One type of page can load faster than another — even if all other test parameters are the same.
Testing site speed across multiple page templates
As discussed earlier, there are many different template types that can make up a successful website. Testing a selection of pages from each of these templates is recommended to get the best picture of the load time performance of your site.
An excellent way to do this is through using the PageSpeed Insights API and connecting it to Screaming Frog or using cloud tools such as OnCrawl or Site Bulb, which will test the speed of each page on your website as it crawls.
To do this in Screaming Frog, go to “Configuration”. In “API Access”, select “PageSpeed Insights”, and there you will see fields to include the API key.
Once done, in the “Metrics” section, you can select both the device that you want to track and the reports, metrics, etc., that you are interested in extracting page speed information. In the example below, we have selected Crux Data and TTFB (Time to First Byte) and LCP and FCP data. Although the crawl may take longer to complete, this information should now appear alongside the URLs in the final crawl.
Choosing your testing location
There are various tools you can use to test your site speed, such as PageSpeed Insights, WebPageTest, and GTmetrix, and most of these do allow you to set your testing location.
It’s important to test your e-commerce site from a location close to where your data centre is located (where your website is hosted), as well as one that is further away. Doing this lets you get an idea of how your real customers are experiencing your store.
If you have a CDN installed, such as Cloudflare, this is also useful, as it allows you to see how much of an impact the CDN is having on your website and how it helps your site load more quickly.
Wherever you decide to test from, remember to keep these locations the same each time you test so you can get accurate results.
Understanding caching and how it influences site speed
If your e-commerce website has caching installed, it’s even more important to test your pages more than once. This is because, on the first test, your page may not have loaded over the cache yet. Once it does, your results will likely be much faster than what you saw on your first test.
With or without caching installed, I would recommend testing each page template around three times for both mobile and desktop devices to get a good measurement and then calculate the average..
Common e-commerce website mistakes
Understanding the common problems that e-commerce websites make is valuable for learning how to avoid them on your own website, as the reasons some tactics remain underused come down to these errors.
Faceted navigation for e-commerce
Whatever your e-commerce site sells, it should be easy to navigate, with sensible menus and navigation options that clearly tell visitors what they will see when they click.
You can see this on the Boohoo website, a prominent fashion retailer in the UK. This image shows the women’s dresses navigation, but you can see how it is broken down by type of dresses, dresses by occasion, colour, how they fit, and even by current fashion trends. Users are able to navigate directly to the subcategories they need.
Good website architecture matters
The importance of good architecture cannot be underestimated and should be centered around the core actions you want people to complete. Ideally, it would be best if you attempted to set up a site with the homepage, followed by the subsequent categories, subcategories, and then the products underneath.
Boohoo has followed this same ideology with their architecture — as trends change and new lines of dresses are added, they can quickly expand and edit the architecture as needed.
Keeping it simple and scalable is the key to setting up good architecture. As your store grows, you will likely add more categories and products, so you need to be able to do this efficiently. You should attempt to keep important pages less than three clicks from the homepage and implement keyword research processes to create highly relevant page URLs and subdirectories.
You want people to buy your products, so don’t make it difficult for them. You can then have other areas on the site for content silos and blogs that link to the various categories and products around the site.
Creating effective product pages
The product page design shouldn’t detract from the shopping experience, and the product information should be as “friendly” and accessible as possible.
Try to use the product information you have available in your Product Information Management (PIM) system. Ensure that your sizes, measurements, colors, prices, and other details are easy to find, read, and understand. These details are even more vital if you happen to sell products that others also offer. If you’re not including any sizes, but your competitors are, you’re increasing your chances that potential customers can choose to buy from them instead. If you’re targeting multiple countries, consider whether you need to include your measurements in imperial, metric, or both. Information should be localized where relevant.
Some top ways to ensure you always include enough information and avoid thin content on your product pages are to:
Start with a 50-100 word introduction: Think about what the product does and who needs it? One way to do this can be seen in the example from Apple below.
List the critical features and technical specifications in bullet format.
Include a “deep dive” section: Write a detailed product description with use cases, relevant awards the product may have won, benefits of the product, images of the product in use, and any FAQs.
Make use of user-generated content such as customer photos and reviews.
End with a 50-100 word conclusion: Summarize the product and use a call to action to encourage your customers to make the purchase.
Including enough information can be the difference between whether or not you make the sale or whether a customer purchases from a competitor.
Utilizing FAQ content to sell more products
People undoubtedly have questions about your products. If customers can’t find the answers they need on your website, they’ll search elsewhere. They’re likely to buy from that website when they find the answers.
You can rectify this by having a general FAQ section on your website. This is where you would answer questions about website security, shipping and return policies, etc. When it comes to product-specific questions, these should be answered on the product pages themselves.
The need to monitor out-of-stock products
There can be many reasons why a product is out of stock, yet the page is still live on an e-commerce site, including:
Ultimately, out-of-stock products can lead to customer frustration. Unsatisfied customers and a poor user experience — on top of the SEO implications of so many unuseful pages — result in fewer purchases and, ultimately, a poor-performing e-commerce store.
There are many ways that the performance of an e-commerce website can be optimized and analyzed, and these are just a few. While they may be less common, they can allow you to get additional data, which, once acted upon accordingly, can help you to outperform others in your market.