Connect with us

MARKETING

11 A/B Testing Examples From Real Businesses

Published

on

11 A/B Testing Examples From Real Businesses

Whether you’re looking to increase revenue, sign-ups, social shares, or engagement, A/B testing and optimization can help you get there.

But for many marketers out there, the tough part about A/B testing is often finding the right test to drive the biggest impact — especially when you’re just getting started. So, what’s the recipe for high-impact success?

Free Download: A/B Testing Guide and Kit

Truthfully, there is no one-size-fits-all recipe. What works for one business won’t work for another — and finding the right metrics and timing to test can be a tough problem to solve. That’s why you need inspiration from A/B testing examples.

In this post, let’s review how a hypothesis will get you started with your testing, and check out excellent examples from real businesses using A/B testing. While the same tests may not get you the same results, they can help you run creative tests of your own.

A/B Testing Hypothesis Examples

A hypothesis can make or break your experiment, especially when it comes to A/B testing. When creating your hypothesis, you want to make sure that it’s:

Advertisement
  1. Focused on one specific problem you want to solve or understand
  2. Able to be proven or disproven
  3. Focused on making an impact (bringing higher conversion rates, lower bounce rate, etc.)

When creating a hypothesis, following the “If, then” structure can be helpful, where if you changed a specific variable, then a particular result would happen.

Here are some examples of what that would look like in an A/B testing hypothesis:

  • Shortening contact submission forms to only contain required fields would increase the number of sign-ups.
  • Changing the call-to-action text from “Download now” to “Download this free guide” would increase the number of downloads.
  • Reducing the frequency of mobile app notifications from five times per day to two times per day will increase mobile app retention rates.
  • Using featured images that are more contextually related to our blog posts will contribute to a lower bounce rate.
  • Greeting customers by name in emails will increase the total number of clicks.

Let’s go over some real-life examples of A/B testing to prepare you for your own.

A/B Testing Examples

Website A/B Testing Examples

1. HubSpot Academy’s Homepage Hero Image

Most websites have a homepage hero image that inspires users to engage and spend more time on the site. This A/B testing example shows how hero image changes can impact user behavior and conversions.

Problem

Based on previous data, HubSpot Academy found that out of more than 55,000 page views, only .9% of those users were watching the video on the homepage. Of those viewers, almost 50% watched the full video.

Chat transcripts also highlighted the need for clearer messaging for this useful and free resource.

That’s why the HubSpot team decided to test how clear value propositions could improve user engagement and delight.

A/B Test Method

HubSpot used three variants for this test, using HubSpot Academy conversion rate (CVR) as the primary metric. Secondary metrics included CTA clicks and engagement.

Advertisement

Variant A was the control.

A/B testing examples: HubSpot Academy's Homepage Hero

For variant B, the team added more vibrant images and colorful text and shapes. It also included an animated “typing” headline.

A/B testing examples: HubSpot Academy's Homepage Hero

Variant C also added color and movement, as well as animated images on the right-hand side of the page.

A/B testing examples: HubSpot Academy's Homepage Hero

Results

As a result, HubSpot found that variant B outperformed the control by 6%. In contrast, variant C underperformed the control by 1%. From those numbers, HubSpot was able to project that using variant B would lead to about 375 more sign ups each month.

2. FSAstore.com’s Site Navigation

Every marketer will have to focus on conversion at some point. But building a website that converts is tough.

Problem

FSAstore.com is an ecommerce company supplying home goods for Americans with a flexible spending account.

Advertisement

This useful site could help the 35 million+ customers that have an FSA. But the website funnel was overwhelming. It had too many options, especially on category pages. The team felt that customers weren’t making purchases because of that issue.

A/B Test Method

To figure out how to appeal to its customers, this company tested a simplified version of its website. The current site included an information-packed subheader in the site navigation.

To test the hypothesis, this A/B testing example compared the current site to an update without the subheader.

A/B testing examples: FSAstore.com

Results

This update showed a clear boost in conversions and FSAstore.com saw a 53.8% increase in revenue per visitor.

3. Expoze’s Web Page Background

The visuals on your web page are important because they help users decide whether they want to spend more time on your site.

In this A/B testing example, Expoze.io decided to test the background on its homepage.

Advertisement
Problem

The website home page was difficult for some users to read because of low contrast. The team also needed to figure out how to improve page navigation while still representing the brand.

A/B Test Method

First, the team did some research and created several different designs. The goals of the redesign were to improve the visuals and increase attention to specific sections of the home page, like the video thumbnail.

A/B testing examples: Expoze.io

They used AI-generated eye tracking as they designed to find the best designs before A/B testing. Then they ran an A/B heatmap test to see whether the new or current design got the most attention from visitors.

A/B testing examples: Expoze.io heatmaps

Results

The new design showed a big increase in attention, with version B bringing over 40% more attention to the desired sections of the home page.

This design change also brought a 25% increase in CTA clicks. The team believes this is due to the added contrast on the page bringing more attention to the CTA button, which was not changed.

4. Thrive Themes’ Sales Page Optimization

Many landing pages showcase testimonials. That’s valuable content and it can boost conversion.

Advertisement

That’s why Thrive Themes decided to test a new feature on its landing pages — customer testimonials.

Problem

In the control, Thrive Themes had been using a banner that highlighted product features, but not how customers felt about the product.

The team decided to test whether adding testimonials to a sales landing page could improve conversion rates.

A/B Test Method

In this A/B test example, the team ran a 6-week test with the control against an updated landing page with testimonials.

A/B testing examples: Thrive Themes

Results

This change netted a 13% increase in sales. The control page had a 2.2% conversion rate, but the new variant showed a 2.75% conversion rate.

Email A/B Testing Examples

5. HubSpot’s Email Subscriber Experience

Getting users to engage with email isn’t an easy task. That’s why HubSpot decided to A/B test how alignment impacts CTA clicks.

Advertisement
Problem

HubSpot decided to change text alignment in the weekly emails for subscribers to improve the user experience. Ideally, this improved experience would result in a higher click rate.

A/B Test Method

For the control, HubSpot sent centered email text to users.

A/B test examples: HubSpot, centered text alignment

For variant B, HubSpot sent emails with left-justified text.

A/B test examples: HubSpot, left-justified text alignment

Results

HubSpot found that emails with left-aligned text got fewer clicks than the control. And of the total left-justified emails sent, less than 25% got more clicks than the control.

6. Neurogan’s Deal Promotion

Making the most of email promotion is important for any company, especially those in competitive industries.

This example uses the power of current customers for increasing email engagement.

Advertisement
Problem

Neurogan wasn’t always offering the right content to its audience and it was having a hard time competing with a flood of other new brands.

A/B Test Method

An email agency audited this brand’s email marketing, then focused efforts on segmentation. This A/B testing example starts with creating product-specific offers. Then, this team used testing to figure out which deals were best for each audience.

Results

These changes brought higher revenue for promotions and higher click rates. It also led to a new workflow with a 37% average open rate and a click rate of 3.85%.

For more on how to run A/B testing for your campaigns, check out this free A/B testing kit.

Social Media A/B Testing Examples

7. Vestiaire’s TikTok Awareness Campaign

A/B testing examples like the one below can help you think creatively about what to test and when. This is extra helpful if your business is working with influencers and doesn’t want to impact their process while working toward business goals.

Problem

Fashion brand Vestaire wanted help growing the brand on TikTok. It was also hoping to increase awareness with Gen Z audiences for its new direct shopping feature.

Advertisement
A/B Test Method

Vestaire’s influencer marketing agency asked eight influencers to create content with specific CTAs to meet the brand’s goals. Each influencer had extensive creative freedom and created a range of different social media posts.

Then, the agency used A/B testing to choose the best-performing content and promoted this content with paid advertising.

A/B testing examples: Vestaire

Results

This testing example generated over 4,000 installs. It also decreased the cost per install by 50% compared to the brand’s existing presence on Instagram and YouTube.

8. Underoutfit’s Promotion of User-Generated Content on Facebook

Paid advertising is getting more expensive, and clickthrough rates decreased through the end of 2022.

To make the most of social ad spend, marketers are using A/B testing to improve ad performance. This approach helps them test creative content before launching paid ad campaigns, like in the examples below.

Problem

Underoutfit wanted to increase brand awareness on Facebook.

Advertisement
A/B Test Method

To meet this goal, it decided to try adding branded user-generated content. This brand worked with an agency and several creators to create branded content to drive conversion.

Then, Underoutfit ran split testing between product ads and the same ads combined with the new branded content ads. Both groups in the split test contained key marketing messages and clear CTA copy.

The brand and agency also worked with Meta Creative Shop to make sure the videos met best practice standards.

A/B testing examples: Underoutfit

Results

The test showed impressive results for the branded content variant, including a 47% higher clickthrough rate and 28% higher return on ad spend.

9. Databricks’ Ad Performance on LinkedIn

Pivoting to a new strategy quickly can be difficult for organizations. This A/B testing example shows how you can use split testing to figure out the best new approach to a problem.

Problem

Databricks, a cloud software tool, needed to raise awareness for an event that was shifting from in-person to online.

Advertisement
A/B Test Method

To connect with a large group of new people in a personalized way, the team decided to create a LinkedIn Message Ads campaign. To make sure the messages were effective, it used A/B testing to tweak the subject line and message copy.

Results

A/B testing examples: Databricks

The third variant of the copy featured a hyperlink in the first sentence of the invitation. Compared to the other two variants, this version got nearly twice as many clicks and conversions.

Mobile A/B Testing Example

7. HubSpot’s Mobile Calls-to-Action

On this blog, you’ll notice anchor text in the introduction, a graphic CTA at the bottom, and a slide-in CTA when you scroll through the post. Once you click on one of these offers, you’ll land on a content offer page.

While many users access these offers from a desktop or laptop computer, many others plan to download these offers to mobile devices.

Problem

But on mobile, users weren’t finding the CTA buttons as quickly as they could on a computer. That’s why HubSpot tested mobile design changes to improve the user experience.

Previous A/B tests revealed that HubSpot’s mobile audience was 27% less likely to click through to download an offer. Also, less than 75% of mobile users were scrolling down far enough to see the CTA button.

Advertisement
A/B Test Method

So, HubSpot decided to test different versions of the offer page CTA, using conversion rate (CVR) as the primary metric. For secondary metrics, the team measured CTA clicks for each CTA, as well as engagement.

HubSpot used four variants for this test.

For variant A, the control, the traditional placement of CTAs remained unchanged.

For variant B, the team redesigned the hero image and added a sticky CTA bar.

A/B testing examples: HubSpot mobile, A & B

For variant C, the redesigned hero was the only change.

For variant D, the team redesigned the hero image and repositioned the slider.

Advertisement

A/B testing examples: HubSpot mobile, C & D

Results

All variants outperformed the control for the primary metric, CVR. Variant C saw a 10% increase, variant B saw a 9% increase, and variant D saw an 8% increase.

From those numbers, HubSpot was able to project that using variant C on mobile would lead to about 1,400 more content leads and almost 5,700 more form submissions each month.

11. Hospitality.net’s Mobile Booking

Businesses need to keep up with quick shifts in mobile devices to create a consistently strong customer experience.

A/B testing examples like the one below can help your business streamline this process.

Problem

Hospitality.net offered both simplified and dynamic mobile booking experiences. The simplified experience showed a limited number of available dates and the design is for smaller screens. The dynamic experience is for the larger mobile device screens. It shows a wider range of dates and prices.

But the brand wasn’t sure which mobile optimization strategy would be better for conversion.

Advertisement
A/B Test Method

This brand believed that customers would prefer the dynamic experience and that it would get more conversions. But it chose to test these ideas with a simple A/B test. Over 34 days, it sent half of the mobile visitors to the simplified mobile experience, and half to the dynamic experience, with over 100,000 visitors total.

A/B testing examples: Hospitality.net

Results

This A/B testing example showed a 33% improvement in conversion. It also helped confirm the brand’s educated guesses about mobile booking preferences.

A/B Testing Takeaways for Marketers

A lot of different factors can go into A/B testing, depending on your business needs. But there are a few key things to keep in mind:

  • Every A/B test should start with a hypothesis focused on one specific problem that you can test.
  • Make sure you’re testing a control variable (your original version) and a treatment variable (a new version that you think will perform better).
  • You can test various things, like landing pages, CTAs, emails, or mobile app designs.
  • The best way to understand if your results mean something is to figure out the statistical significance of your test.
  • There are a variety of goals to focus on for A/B testing (increased site traffic, lower bounce rates, etc.), but you should be able to test, support, prove, and disprove your hypothesis.
  • When testing, make sure you’re splitting your sample groups equally and randomly, so your data is viable and not due to chance.
  • Take action based on the results you observe.

Start Your Next A/B Test Today

You can see amazing results from the A/B testing examples above. These businesses were able to take action on goals because they started testing. If you want to get great results, you’ve got to get started, too.

Editor’s note: This post was originally published in October 2014 and has been updated for comprehensiveness.

The Ultimate A/B Testing Kit

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

MARKETING

How To Combine PR and Content Marketing Superpowers To Achieve Business Goals

Published

on

A figure pulls open a dress shirt to reveal the term PR on a Superman-like costume, reflecting the superpower resulting from combining content and PR.

A transformative shift is happening, and it’s not AI.

The aisle between public relations and content marketing is rapidly narrowing. If you’re smart about the convergence, you can forever enhance your brand’s storytelling.

The goals and roles of content marketing and PR overlap more and more. The job descriptions look awfully similar. Shrinking budgets and a shrewd eye for efficiency mean you and your PR pals could face the chopping block if you don’t streamline operations and deliver on the company’s goals (because marketing communications is always first to be axed, right?).

Yikes. Let’s take a big, deep breath. This is not a threat. It’s an opportunity.

Advertisement

Reach across the aisle to PR and streamline content creation, improve distribution strategies, and get back to the heart of what you both are meant to do: Build strong relationships and tell impactful stories.

So, before you panic-post that open-to-work banner on LinkedIn, consider these tips from content marketing, PR, and journalism pros who’ve figured out how to thrive in an increasingly narrowing content ecosystem.

1. See journalists as your audience

Savvy pros know the ability to tell an impactful story — and support it with publish-ready collateral — grounds successful media relationships. And as a content marketer, your skills in storytelling and connecting with audiences, including journalists, naturally support your PR pals’ media outreach.

Strategic storytelling creates content focused on what the audience needs and wants. Sharing content on your blog or social media builds relationships with journalists who source those channels for story ideas, event updates, and subject matter experts.

“Embedding PR strategies in your content marketing pieces informs your audience and can easily be picked up by media,” says Alex Sanchez, chief experience officer at BeWell, New Mexico’s Health Insurance Marketplace. “We have seen reporters do this many times, pulling stories from our blogs and putting them in the nightly news — most of the time without even reaching out to us.”

Acacia James, weekend producer/morning associate producer at WTOP radio in Washington, D.C., says blogs and social media posts are helpful to her work. “If I see a story idea, and I see that they’re willing to share information, it’s easier to contact them — and we can also backlink their content. It’s huge for us to be able to use every avenue.” 

Advertisement

Kirby Winn, manager of PR at ImpactLife, says reporters and assignment editors are key consumers of their content. “And I don’t mean a news release that just hit their inbox. They’re going to our blog and consuming our stories, just like any other audience member,” he says. “Our organization has put more focus into content marketing in the past few years — it supports a media pitch so well and highlights the stories we have to tell.”

Storytelling attracts earned media that might not pick up the generic news topic. “It’s one thing to pitch a general story about how we help consumers sign up for low-cost health insurance,” Alex says. “Now, imagine a single mom who just got a plan after years of thinking it was too expensive. She had a terrible car accident, and the $60,000 ER bill that would have ruined her financially was covered. Now that’s a story journalists will want to cover, and that will be relatable to their audience and ours.” 

2. Learn the media outlet’s audience

Seventy-three percent of reporters say one-fourth or less of the stories pitched are relevant to their audiences, according to Cision’s 2023 State of the Media Report (registration required).

PR pros are known for building relationships with journalists, while content marketers thrive in building communities around content. Merge these best practices to build desirable content that works for your target audience and the media’s audiences simultaneously.

WTOP’s Acacia James says sources who show they’re ready to share helpful, relevant content often win pitches for coverage. “In radio, we do a lot of research on who is listening to us, and we’re focused on a prototype called ‘Mike and Jen’ — normal, everyday people in Generation X … So when we get press releases and pitches, we ask, ‘How interested will Mike and Jen be in this story?’” 

3. Deliver the full content package (and make journalists’ jobs easier)

Cranking out content to their media outlet’s standards has never been tougher for journalists. Newsrooms are significantly understaffed, and anything you can do to make their lives easier will be appreciated and potentially rewarded with coverage. Content marketers are built to think about all the elements to tell the story through multiple mediums and channels.

Advertisement

“Today’s content marketing pretty much provides a package to the media outlet,” says So Young Pak, director of media relations at MedStar Washington Hospital Center. “PR is doing a lot of storytelling work in advance of media publication. We (and content marketing) work together to provide the elements to go with each story — photos, subject matter experts, patients, videos, and data points, if needed.”   

At WTOP, the successful content package includes audio. “As a radio station, we are focused on high-quality sound,” Acacia James says. “Savvy sources know to record and send us voice memos, and then we pull cuts from the audio … You will naturally want to do someone a favor if they did you one — like providing helpful soundbites, audio, and newsworthy stories.”  

While production value matters to some media, you shouldn’t stress about it. “In the past decade, how we work with reporters has changed. Back in the day, if they couldn’t be there in person, they weren’t going to interview your expert,” says Jason Carlton, an accredited PR professional and manager of marketing and communications at Intermountain Health. “During COVID, we had to switch to virtual interviewing. Now, many journalists are OK with running a Teams or Zoom interview they’ve done with an expert on the news.”

BeWell’s Alex Sanchez agrees. “I’ve heard old school PR folks cringe at the idea of putting up a Zoom video instead of getting traditional video interviews. It doesn’t really matter to consumers. Focus on the story, on the timeliness, and the relevance. Consumers want authenticity, not super stylized, stiff content.”

4. Unite great minds to maximize efficiency

Everyone needs to set aside the debate about which team — PR or content marketing — gets credit for the resulting media coverage.

At MedStar Washington Hospital Center, So Young and colleagues adopt a collaborative mindset on multichannel stories. “We can get the interview and gather information for all the different pieces — blog, audio, video, press release, internal newsletter, or magazine. That way, we’re not trying to figure things out individually, and the subject matter experts only have to have that conversation once,” she says.

Advertisement

Regular, cross-team meetings are essential to understand the best channels for reaching key audiences, including the media. A story that began life as a press release might reap SEO and earned media gold if it’s strategized as a blog, video, and media pitch.

“At Intermountain Health, we have individual teams for media relations, marketing, social media, and hospital communications. That setup works well because it allows us to bring in the people who are the given experts in those areas,” says Intermountain’s Jason Carlton. “Together, we decide if a story is best for the blog, a media pitch, or a mix of channels — that way, we avoid duplicating work and the risk of diluting the story’s impact.”

5. Measure what matters

Cutting through the noise to earn media mentions requires keen attention to metrics. Since content marketing and PR metrics overlap, synthesizing the data in your team meetings can save time while streamlining your storytelling efforts.

“For content marketers, using analytical tools such as GA4 can help measure the effectiveness of their content campaigns and landing pages to determine meaningful KPIs such as organic traffic, keyword rankings, lead generation, and conversion rates,” says John Martino, director of digital marketing for Visiting Angels. “PR teams can use media coverage and social interactions to assess user engagement and brand awareness. A unified and omnichannel approach can help both teams demonstrate their value in enhancing brand visibility, engagement, and overall business success.”

To track your shared goals, launch a shared dashboard that helps tell the combined “story of your stories” to internal and executive teams. Among the metrics to monitor:

  • Page views: Obviously, this queen of metrics continues to be important across PR and content marketing. Take your analysis to the next level by evaluating which niche audiences are contributing to these views to further hone your storytelling targets, including media outlets.
  • Earned media mentions: Through a media tracker service or good old Google Alerts, you can tally the echo of your content marketing and PR. Look at your site’s referral traffic report to identify media outlets that send traffic to your blog or other web pages.
  • Organic search queries: Dive into your analytics platform to surface organic search queries that lead to visitors. Build from those questions to develop stories that further resonate with your audience and your targeted media.
  • On-page actions: When visitors show up on your content, what are they doing? What do they click? Where do they go next? Building next-step pathways is your bread and butter in content marketing — and PR can use them as a natural pipeline for media to pick up more stories, angles, and quotes.

But perhaps the biggest metric to track is team satisfaction. Who on the collaborative team had the most fun writing blogs, producing videos, or calling the news stations? Lean into the natural skills and passions of your team members to distribute work properly, maximize the team output, and improve relationships with the media, your audience, and internal teams.

“It’s really trying to understand the problem to solve — the needle to move — and determining a plan that will help them achieve their goal,” Jason says. “If you don’t have those measurable objectives, you’re not going to know whether you made a difference.”

Advertisement

Don’t fear the merger

Whether you deliberately work together or not, content marketing and public relations are tied together. ImpactLife’s Kirby Winn explains, “As soon as we begin to talk about (ourselves) to a reporter who doesn’t know us, they are certainly going to check out our stories.”

But consciously uniting PR and content marketing will ease the challenges you both face. Working together allows you to save time, eliminate duplicate work, and gain free time to tell more stories and drive them into impactful media placements.

Register to attend Content Marketing World in San Diego. Use the code BLOG100 to save $100. Can’t attend in person this year? Check out the Digital Pass for access to on-demand session recordings from the live event through the end of the year.

HANDPICKED RELATED CONTENT:

Cover image by Joseph Kalinowski/Content Marketing Institute

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

Trends in Content Localization – Moz

Published

on

Trends in Content Localization - Moz

Multinational fast food chains are one of the best-known examples of recognizing that product menus may sometimes have to change significantly to serve distinct audiences. The above video is just a short run-through of the same business selling smokehouse burgers, kofta, paneer, and rice bowls in an effort to appeal to people in a variety of places. I can’t personally judge the validity of these representations, but what I can see is that, in such cases, you don’t merely localize your content but the products on which your content is founded.

Sometimes, even the branding of businesses is different around the world; what we call Burger King in America is Hungry Jack’s in Australia, Lays potato chips here are Sabritas in Mexico, and DiGiorno frozen pizza is familiar in the US, but Canada knows it as Delissio.

Tales of product tailoring failures often become famous, likely because some of them may seem humorous from a distance, but cultural sensitivity should always be taken seriously. If a brand you are marketing is on its way to becoming a large global seller, the best insurance against reputation damage and revenue loss as a result of cultural insensitivity is to employ regional and cultural experts whose first-hand and lived experiences can steward the organization in acting with awareness and respect.

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

How AI Is Redefining Startup GTM Strategy

Published

on

How AI Is Redefining Startup GTM Strategy

AI and startups? It just makes sense.

(more…)

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS