Connect with us

MARKETING

Is A/B testing dead?

Published

on

Is A/B testing dead?

“Today, A/B testing is thriving — it’s been a huge improvement from non-A/B testing,” said George Khachatryan, CEO of AI company OfferFit, in a recent webinar. “At the same time, the people performing these tests every day recognize that’s it’s a lot more difficult than it may seem.”

Designing A/B tests, determining samples sizes, and deploying them takes up a lot of time and resources, and analyzing the findings requires high levels of precision. All in all, the manual tasks required by A/B testing can place a heavy burden on marketers.

“When you’re running a full experimentation program, it’s never enough to run one A/B test,” Khachatryan said. “When you run one, you gain valuable insights, and inevitably want to gain more. So you end up running more.”

He added, “Those who are doing this hands-on realize it just becomes a rapid explosion in the number of tests they need – it becomes infeasible very early on this exponential curve.”

graph showing how A/B testing fails to scale
Source: George Khachatryan

Marketers need a solution that allows them to test a growing number of campaign variables while simultaneously giving them enough time to analyze the data. Fortunately, A/B testing is evolving.

Expanding the power of A/B testing

In the webinar, Khachatryan highlighted the “multi-armed bandit problem” that’s affecting modern A/B testing. In the traditional version of this scenario, a person at a casino must determine which slot machines (the “one-armed bandits” that steal your money) are going to have the best payouts, then figure out which order will be optimal. With A/B testing, the variables are the multi-armed bandits, and the marketer must discover which are most effective so they can allocate more resources to the areas performing well.

Advertisement

“You can think of [a multi-armed bandit] like a smart A/B test,” he said. “It’ll navigate the exploration-exploitation tradeoff — it’ll start randomly pulling those ten handles, but as it goes, it’ll dynamically reallocate resources so that if something looks bad it’ll stop pulling.”

mult-armed bandits vs contextual bandits in A/B testing
Source: George Khachatryan

He added, “These multi-arm bandits are designed to experiment just the right amount so you’re learning but also taking advantage of what you’ve already learned.”

While these multi-armed, or A/B, models have served marketers well over the years, there’s a new iteration of the framework that is more accurate and effective. According to Khachatryan, these are “contextual bandits.”

“It does what a multi-armed bandit does, but it takes into account different contexts,” he said. “So, if you have two different customers, with different characteristics, it’ll know to pull different levers.”

Contextual bandit frameworks are essentially automated experimentation and personalization at scale. It’s a model that can completely automate the process, and it’s what every marketer should be moving toward to improve campaign effectiveness at scale.


Get the daily newsletter digital marketers rely on.


Automated experimentation is the future

Many of the tech giants have already adopted contextual bandit frameworks, but marketers should note that this technology is still incredibly new. Brands should allocate enough time and resources to make the transition process easier, because, according to Khachatryan, it’s the “future of experimentation.”

Advertisement

“In the past, manual A/B testing works with one mention at a time,” he said. “With these contextual bandits, you can set it up to simlutantous test multiple dimensions.”

Whether it’s testing email subject line efficacy, call-to-action click-through rates, or optimal article posting times, marketers have a lot of experimental data to keep up with. Automated testing solutions can make these processes more manageable by decreasing the time spent on manual tasks, replacing them with continuous automated experimentation.

automated campaign testing
Source: George Khachatryan

“You can think of this as the next iteration of experimentation, or test-and-learn programs,” Khachatryan said. “When a marketer sets up this system of continuous automated experimentation, it creates this interplay where you can see what happens, gain insights, and then use those insights to get new ideas.”

He added, “So you still have the agile test-and-learn cycle, but it’s accelerated.”

Time will tell how quickly marketers adopt these automated experimentation technologies. But, with the high level of marketing technology replacements that took place over the past year, there’s a good chance more brands will sign on sooner rather than later.

Watch this webinar presentation at Digital Marketing Depot.


Marketing automation: A snapshot

What they are. For today’s marketers, automation platforms are often the center of the marketing stack. They aren’t shiny new technologies, but rather dependable stalwarts that marketers can rely upon to help them stand out in a crowded inbox and on the web amidst a deluge of content.

Advertisement

How they’ve changed. To help marketers win the attention battle, marketing automation vendors have expanded from dependence on static email campaigns to offering dynamic content deployment for email, landing pages, mobile and social. They’ve also incorporated features that rely on machine learning and artificial intelligence for functions such as lead scoring, in addition to investing in the user interface and scalability.

Why we care. The growing popularity of account-based marketing has also been a force influencing vendors’ roadmaps, as marketers seek to serve the buying group in a holistic manner — speaking to all of its members and their different priorities. And, ideally, these tools let marketers send buyer information through their tight integrations with CRMs, giving the sales team a leg up when it comes to closing the deal.

Read next: What is marketing automation?


About The Author

4 ways to build a successful ABM strategy

Corey Patterson is an Editor for MarTech and Search Engine Land. With a background in SEO, content marketing, and journalism, he covers SEO and PPC to help marketers improve their campaigns.


Source link
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

MARKETING

Trends in Content Localization – Moz

Published

on

Trends in Content Localization - Moz

Multinational fast food chains are one of the best-known examples of recognizing that product menus may sometimes have to change significantly to serve distinct audiences. The above video is just a short run-through of the same business selling smokehouse burgers, kofta, paneer, and rice bowls in an effort to appeal to people in a variety of places. I can’t personally judge the validity of these representations, but what I can see is that, in such cases, you don’t merely localize your content but the products on which your content is founded.

Sometimes, even the branding of businesses is different around the world; what we call Burger King in America is Hungry Jack’s in Australia, Lays potato chips here are Sabritas in Mexico, and DiGiorno frozen pizza is familiar in the US, but Canada knows it as Delissio.

Tales of product tailoring failures often become famous, likely because some of them may seem humorous from a distance, but cultural sensitivity should always be taken seriously. If a brand you are marketing is on its way to becoming a large global seller, the best insurance against reputation damage and revenue loss as a result of cultural insensitivity is to employ regional and cultural experts whose first-hand and lived experiences can steward the organization in acting with awareness and respect.

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

How AI Is Redefining Startup GTM Strategy

Published

on

How AI Is Redefining Startup GTM Strategy

AI and startups? It just makes sense.

(more…)

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

More promotions and more layoffs

Published

on

More promotions and more layoffs

For martech professionals salaries are good and promotions are coming faster, unfortunately, layoffs are coming faster, too. That’s according to the just-released 2024 Martech Salary and Career Survey. Another very unfortunate finding: The median salary of women below the C-suite level is 35% less than what men earn.

The last year saw many different economic trends, some at odds with each other. Although unemployment remained very low overall and the economy grew, some businesses — especially those in technology and media — cut both jobs and spending. Reasons cited for the cuts include during the early years of the pandemic, higher interest rates and corporate greed.

Dig deeper: How to overcome marketing budget cuts and hiring freezes

Be that as it may, for the employed it remains a good time to be a martech professional. Salaries remain lucrative compared to many other professions, with an overall median salary of $128,643. 

Advertisement

Here are the median salaries by role:

  • Senior management $199,653
  • Director $157,776
  • Manager $99,510
  • Staff $89,126

Senior managers make more than twice what staff make. Directors and up had a $163,395 median salary compared to manager/staff roles, where the median was $94,818.

One-third of those surveyed said they were promoted in the last 12 months, a finding that was nearly equal among director+ (32%) and managers and staff (30%). 

PX3zocqNZfzMbWNEZhW9dZnAgkdPrLW8fjkrbVrcEkrNJpJiXrVKkjlQ0Tzuj8YKh Ht9HTEvmxDDt0ZsntfYiZHS0NJ7zEZ 6yMT3OjZajbaXBFV1D2Pk5euJeHKdRuzOzM5ZUxwNtsVNaiIbNrd Q

Extend the time frame to two years, and nearly three-quarters of director+ respondents say they received a promotion, while the same can be said for two-thirds of manager and staff respondents.

Dig deeper: Skills-based hiring for modern marketing teams

Employee turnover 

In 2023, we asked survey respondents if they noticed an increase in employee churn and whether they would classify that churn as a “moderate” or “significant” increase. For 2024, given the attention on cost reductions and layoffs, we asked if the churn they witnessed was “voluntary” (e.g., people leaving for another role) or “involuntary” (e.g., a layoff or dismissal). More than half of the marketing technology professionals said churn increased in the last year. Nearly one-third classified most of the churn as “involuntary.”

FIHUBtZJfK3IzbyZl C6WXBPTE64Gzg1URDzQUXCrD8YkAPZS7mmjpmAAiuhhheJUE4dGVcn6e9XW87ogLVz0Ya4rqHwB8WfXTHS W0hRW7yEdr2bQNjlTwnXvNhMv9NZ092pq1ws7lu DYqLV8i6fcFIHUBtZJfK3IzbyZl C6WXBPTE64Gzg1URDzQUXCrD8YkAPZS7mmjpmAAiuhhheJUE4dGVcn6e9XW87ogLVz0Ya4rqHwB8WfXTHS W0hRW7yEdr2bQNjlTwnXvNhMv9NZ092pq1ws7lu DYqLV8i6fc

Men and Women

Screenshot 2024 03 21 124540Screenshot 2024 03 21 124540

This year, instead of using average salary figures, we used the median figures to lessen the impact of outliers in the salary data. As a result, the gap between salaries for men and women is even more glaring than it was previously.

In last year’s report, men earned an average of 24% more than women. This year the median salary of men is 35% more than the median salary of women. That is until you get to the upper echelons. Women at director and up earned 5% more than men.

Methodology

The 2024 MarTech Salary and Career Survey is a joint project of MarTech.org and chiefmartec.com. We surveyed 305 marketers between December 2023 and February 2024; 297 of those provided salary information. Nearly 63% (191) of respondents live in North America; 16% (50) live in Western Europe. The conclusions in this report are limited to responses from those individuals only. Other regions were excluded due to the limited number of respondents. 

Advertisement

Download your copy of the 2024 MarTech Salary and Career Survey here. No registration is required.

Get MarTech! Daily. Free. In your inbox.

Source link

Advertisement
Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS