Connect with us

MARKETING

Metrics: How to supercharge your experimentation program

Published

on

Metrics: How to supercharge your experimentation program


Experimentation can be pretty…well, testing at times. Downsized teams, restricted budgets, lack of buy-in, or a misguided focus on win rates (which, by the way, are always low – read on to see why that shouldn’t concern you). 

So, how do you actually go about scaling an experimentation program? The process starts with choosing the right experimentation metrics.  

Metrics are everywhere, but do they really impact business outcomes?  

Yes – because metrics are all about measuring success. As a practitioner, you need to decide what really matters, what you’re trying to achieve, and how well you’re doing. 

  

Establishing your key metrics also sets a baseline for comparisons, which you can use to chart progress over time. They also make your results easier to share across the organization by setting the benchmark for what ‘good’ looks like. As such, metrics play a key role in communicating the value of experimentation, achieving buy-in, and ultimately attracting the resources you need to scale up.   

Image: Metrics by impact share

Measuring the impact of programs is a common challenge – and key metrics are a great way to answer that. Eric pointed out that setting the right metrics is perhaps the most important thing to get right, not just when launching an experimentation program but also in terms of moving it to the next level.   

But… Only 12% of experiments actually win??? 

One of the most eye-catching findings was the fact that only 12% of experiments won. Yet you’d think win rate must be the most important factor in experimentation, right?   

Wrong.

 

Our team discovered that win rates are more a vanity metric than a key indicator of performance.  Eric highlighted the fact that impact is a far more reliable indicator of success. It not only covers your win rate, but also the uplift delivered – because at the end of the day, the monetary impact is what businesses really care about.   

For example, would you rather have tests that win 10% of the time but deliver a million-dollar uplift? Or tests that win 50% of the time but only deliver $100 in additional revenues? (You don’t really have to answer that one.)  

Look at the win rate alone and sure, it can be psychologically challenging to see just 12%. It’s not hard to imagine an underwhelmed management ask: why bother in the first place? Well, first of all, that’s where your metrics come into play, so you can evaluate what success really looks like.   

At the same time, the need to flip things around and see yourself as learning 100% of the time. For example: the figures showed that just 12% of experiments also lose, which means you get to eliminate features that can have a negative effect. The 76% proving to be inconclusive means you can stop investing time and resources into irrelevant areas.  

So sure, win rate is important – especially if you’re trying to get buy-in at the beginning of your program. But we saw you need to move past that and start framing the value of experimentation in terms of uplift, and translating win rates into expected impact per test.  

 

So… what other metrics should you be looking at to get up to speed 

The report also identified the most common metrics used to gauge the overall success of experimentation programs. The team found that velocity was far and away the most widely used metric for experimentation programs. The number of tests performed matters more than anything else – with the proviso, they are underpinned by quality. By that, we mean experiments with some thought behind them.   

number of experiments run by a median company per year

Image: Median companies run 34 tests a year.

The impact is another key KPI. It may be a lagging metric but, as we’ve already seen, uplift is what the top dog, the big cheese, the head honcho, really cares about.  

A third significant metric that came up is the percentage of an organization contributing to the program. This is critical when you’re on the road to maturity, creating momentum, and measuring how well you’re doing. Gaining buy-in from across the organization also plays a big part in extending the pipeline of testing ideas that will allow you to keep growing and improving.    

More tests = more value. Even data says that’s not true.

Both I and Eric also discussed the importance of testing velocity: is it really as simple as more tests = more value? As a rule, yes: the evidence shows that more tests = more wins = more value. After all, however much research you do, whatever kind of preparations you make, you never know for sure what’s going to work. It stands to reason the more experiments you run, the greater your chance of hitting a winner.  

When you’re getting your program up and running, say the first 12-18 months, yes – run as many tests as possible. That’ll help you build a data bank of successful stories with the aim of winning more resources and establishing a culture of experimentation. 

But we also saw that moving to the next level is not necessarily about increasing velocity. It’s about focusing on complexity and moving beyond cosmetic changes. Minute tweaks tend to result in minute uplifts. Our research showed us that the highest uplift experiments share two things in common:  

  1. They make larger code changes with more effect on the user experience.   
  2. They test a higher number of variations simultaneously.   

More complex experiments that make major changes to the user experience e.g. pricing, discounts, checkout flow, data collection, etc. are more likely to generate higher uplifts.   

 

Solely focusing on revenue can be a roadblock

Revenue is another key metric that teams report on when highlighting the value of their experimentation program. Not surprising since ultimately revenue keeps a business… in business. Most companies are focused on making money, so that’s where you’re going to want to focus to gain support from the execs. Having said that, Mark and Eric discussed the flip side of the coin.   

Uplift in revenue can be hard to track – say if you don’t have an ecommerce website, or if your business necessitates super complex buying cycles that last two or three years. In that case, if you can’t directly track revenue you want to focus your efforts as low down in the funnel as you can.  

Or say your site or app doesn’t actually aim to generate revenue. They gave the example of a company that makes big sales directly to a small group of customers, and use their site to educate the public.   

The key here is to understand the ultimate purpose of the channel and build your strategy around that. Rather than boosting conversions, you may instead be more interested in click-throughs to certain pages, maximizing time on site, encouraging sign-ups for an event or downloads for a paper.  

It’s also worth considering a few other metrics that we found to be undervalued. Take search rate, for example. As the most undervalued experiment goal, it is only tested 1% of the time, yet it has the highest expected impact at 2.3%. Customers who actively search for a product or service are likely to convert at two to three times the rate of all other users.  

Metrics impact on website visitors

Remember, no journey means no conversions

Three at-a-glance takeaways   

To wrap up, here are three takeaways to help you build and scale a successful experimentation program. 

  1. Think ABCD instead of simply AB. Experiments that test multiple treatments are three times more successful than standard A/B tests.     
  2. Conduct complex experiments. Tests that make major changes to the user experience (pricing, discounts, checkout flow, data collection, etc.) are more likely to win and with higher uplifts.   
  3. Choose the right metrics that match the overall purpose of your site or app’s objectives – and don’t get too hung up on win rates!  

All this is just a taster. 

Why not see the full report? 

Our Evolution of Experimentation report is packed with data from 127,000 experiments, revealing insights, techniques, and examples for scaling up a successful experimentation program. Read the report.


Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

MARKETING

Boost Your Traffic in Google Discover

Published

on

Boost Your Traffic in Google Discover

2. Understand topical authority: Keywords vs. entities

Google has been talking about topical authority for a long time, and in Discover, it is completely relevant. Traditional SEO includes the use of keywords to position your web pages for a specific search, but the content strategy in Discover should be based on entities, i.e., concepts, characters, places, topics… everything that a Knowledge Panel can have. It is necessary to know in which topics Google considers we have more authority and relevance in order to talk about them.

3. Avoid clickbait in titles

“Use page titles that capture the essence of the content, but in a non-clickbait fashion.” This is the opening sentence that describes how headlines should be in Google’s documentation. I always say that it is not about using clickbait but a bit of creativity from the journalist. Generating a good H1 is also part of the job of content creation.

Google also adds:

“Avoid tactics to artificially inflate engagement by using misleading or exaggerated details in preview content (title, snippets, or images) to increase appeal, or by withholding crucial information required to understand what the content is about.”

“Avoid tactics that manipulate appeal by catering to morbid curiosity, titillation, or outrage.

Provide content that’s timely for current interests, tells a story well, or provides unique insights.”

Do you think this information fits with what you see every day on Google Discover? I would reckon there were many sites that did not comply with this and received a lot of traffic from Discover.

With the last core updates in 2023, Google was extremely hard on news sites and some niches with content focused on Discover, directly affecting E-E-A-T. The impact was so severe that many publishers shared drastic drops in Search Console with expert Lily Ray, who wrote an article with data from more than 150 publishers.

4. Images are important

They say that a picture is worth a thousand words. If you look at your Discover feed, you’ll see most of the images catch your attention. They are detailed shots of delicious food, close-ups of a person’s face showing emotions, or even images where the character in question does not appear, such as “the new manicure that will be a trend in 2024,” persuading you to click.

Google’s documentation recommends adding “high-quality images in your content, especially large images that are more likely to generate visits from Discover” and notes important technical requirements such as images needing to be “at least 1200 px wide and enabled by the max-image-preview:large setting.” You may also have found that media outlets create their own collages in order to have images that stand out from competitors.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

Everything You Need to Know About Google Search Essentials (formerly Google Webmaster Guidelines)

Published

on

Everything You Need to Know About Google Search Essentials (formerly Google Webmaster Guidelines)

One of the most important parts of having a website is making sure your audience can find your site (and find what they’re looking for).

The good news is that Google Search Essentials, formerly called Google Webmaster Guidelines, simplifies the process of optimizing your site for search performance.

(more…)

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

Salesforce rolls out new edition of Marketing Cloud for small businesses

Published

on

Salesforce summer 2023 release: The business executive's guide

Today Salesforce announced Marketing Cloud Growth, an edition of Marketing Cloud designed specifically for small businesses.

With help from AI, this edition makes it easy for marketers to segment audiences, create and execute email campaigns from text to image, optimize campaign performance and create unified customer profiles. It also has a prompt builder that can store and manage known reliable prompts for organizations.

Dig deeper: 70% of SMB marketers willing to pay more for tools with AI or automation

Salesforce developed the new edition by looking at the most common use cases for which small businesses frequenty don’t have the people or resources. This includes things like personalizing campaigns and advanced testing.

The company is also letting small businesses (those with fewer than 200 employees) that have Sales or Service Enterprise Edition “get started with Data Cloud at no cost.” Marketing Cloud Growth will initially be available in the U.S. and Canada and is expected to roll out to Europe, the Middle East and Asia by the end of the year.

Why we care. First of all, small businesses need all the help they can get. This creates an opportunity to start using AI within a centralized marketing workflow rather than importing content from independent generative AI tools. Perhaps it’s also a sign of Salesforce moving to compete with platforms (can we say HubSpot?) that more overtly court SMB clients.

Get MarTech! Daily. Free. In your inbox.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS