Connect with us

MARKETING

How to Build a Customer-Centric Business

Published

on

How to Build a Customer-Centric Business

Most businesses out there are product-focused, i.e. they have a product to sell.

This is how it works:

  • Offer a service or create a product
  • Find people to sell it to
  • Make a lot of money
  • Quit selling “offline” or quit their day job

It sounds good (in theory). The problem is that this is like pushing a huge rock uphill. Trying to force a product on a market that hasn’t asked for it can be a long, long process – which nonetheless can be successful, of course. The world hadn’t known they needed computers when they got into the market. But let’s face it. Cases like that are very few. 

There’s another model that a lot of companies are using very successfully, but you just might not know it. It’s called building a “customer-centric” business.

An customer-centric business includes the following typical steps:

  • Determine what a market needs
  • Build a product that covers that need (or adjust your own product to better fit that need)

These are two fundamental steps but there are many routes you can take to build it up:

  • Build a true bond with your “audience”
  • Learn what they really need
  • Sell products/services that they’ve asked for
  • Continue delivering way more value than you “take”
  • Provide high value education to a small set of people
  • Delight those people and grow your platform (website/blog, youtube channel, email list, podcast subscribers)
1666805406 695 Heres Why CRM For Digital Marketing Helps Boost ROI

Get the Digital Marketing Blueprint

With a Customer Value Journey that strategically builds a relationship with new prospects and converts them into loyal, repeat customers. Click here

So how to understand what your target audience needs?

Market research can be as expensive and extensive as your budget allows. There are lots of companies offering you a detailed niche analysis, surveying, numbers and stats. This is what corporations are investing into on a regular basis.

The good news is, the Internet offers lots of opportunities to research your target niche for free or on a tight budget and keep it in-house.

Advertisement

The Internet made customer centricity possible for any small business. Here are a few ideas:

On-site analytics

On-site analytics is a traditional way for website owners to evaluate their traffic, its sources and entry pages. But it is also a great way to understand your target audience better. Google Analytics provides detailed demographic information – from gender and age to your customers’ locations (like popular cities people tend to access your site from).

As I am writing this, I am surprised my readers tend to access my site from London – sounds like I need to be doing a better job addressing UK audience’s needs:

How to Build a Customer Centric Business

Surveying your current (or potential) customers

Asking your customers what they need from your product or service, which features they like best and which of those are not set up the way they can be useful is a great way to align your business to your target audience needs.

When surveying your customers:

  • Mind that there will always be people who just don’t like everything. 
  • Don’t let yourself get discouraged by negative comments. Remember, it’s the route, not the destination
  • Don’t rush to implement changes or add features right away: Make sure to take time to collect feedback, there will likely be conflicting comments: Some people will love the features others hate.
  • Don’t just look at stats: Make sure there’s a field for your customers or readers to have an ability to add their own comments. Some of those ideas will turn invaluable and spark your inspiration for future plans.

As to how you create that survey and introduce it to your customers and/or site users, you can use Google Forms (which are free). There are also quite a few surveying plugins allowing you to embed your surveys to your site.

You can also set up a newsletter to reach out to your customers to invite them to help you out. 

1667205016 485 How to Build a Customer Centric Business

This will also let you identify your most active and loyal customers: Make sure to create a segment for those to reach out more, give away special offers and turn them into brand advocates.

It is also a good idea to time your newsletters the way it is more engaging and delivers better results.

Advertisement

Semantic research

When you are just entering a niche, semantic research is one of the best ways to quickly understand underlying concepts and entities. Text Optimizer is the best tool for the job. Run your established competitor’s name through the tool to understand what your future customers associate that brand with:

1667205016 408 How to Build a Customer Centric Business

Keyword research

Traditional SEO-driven keyword research offers a solid insight into what people are struggling with and even the demand (i.e. search volume) for a particular feature or problem solution.

SE Ranking keyword research tool allows marketers to extend their search queries, find related queries, evaluate keyword difficulty and search volume. You can also cluster your keywords and turn them into mindmaps to better understand your target topic.

Just like with Text Optimizer, for market research, consider using your competitor’s product name as a keyword to better understand what their customers are searching for:

1667205016 985 How to Build a Customer Centric Business

Question research

Knowing what your target customers are asking around the web is a great way to create a product and content development strategy that solves their problems.

Buzzsumo is the best way to research niche questions because it pulls data from multiple sources allowing you to get lots of insights at a glance:

1667205016 170 How to Build a Customer Centric Business

Social media listening

Social media listening is a great way to understand your target audience and relate to it. By listening to and participating in relevant discussions you get to build connections, identify who influences them and what they don’t like about the existing products and competitors.

As you may have guessed, listening to your competitors’ customers is just as a good idea as listening to your own. 

Awario offers a great social media listening solution that helps you create alerts that include several of your competitors by using boolean search. For small and local business, the feature allows one to hyper-target their alerts to a certain location:

Advertisement
1667205016 313 How to Build a Customer Centric Business

Social media listening is not just about monitoring brand mentions though. It is a good idea to monitor related questions, discussions and trends. This will allow you to align all your marketing efforts – including blogger outreach, email marketing and seasonal content creation to niche trends and activity.

Customer centricity at every stage of product development

Creating a customer-centric marketing and product development strategy doesn’t have to happen at the launch.

In fact, it is never too late or too early to make your business customer-focused:

  • Ask for your customers’ (or social media followers’)  input on your new product or brand name. Namify is a cool tool that can suggest those names and provide an easy way for you to choose the best one.
  • Evaluate your customer support activity to better understand your customers’ struggles and develop your content strategy around those problems.
  • Use social media listening and keyword research to understand what is missing from your competitors’ offerings and develop a product that covers that need. Keyword gaps can help you in understanding customers’ demand that is not yet satisfied.

Why should you consider building an audience-based business?

So should you consider turning your business from product-based into audience-based? Here’s why you might want to consider it:

  • It’s hard at the start, but if you persist you’ll have an loyal audience and a solid platform
  • Your product ideas can come from the audience (vs from your imagination)
  • It beats chasing transactional sales. You’re making sales based on relationship
  • You own the platform – you don’t have to continually beg to “borrow” other people’s audiences

Customer-centricity makes your business safer from reputation crises or changing digital marketing landscape because it fosters brand loyalty.


What Does it Take to be the Head of Marketing

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

MARKETING

Crafting Effortless Sales Through ‘Wow’ Moments in Experience Marketing

Published

on

Crafting Effortless Sales Through 'Wow' Moments in Experience Marketing

Crafting Effortless Sales Through Wow Moments in Experience Marketing

In an era where consumers are bombarded with endless choices and digital noise, standing out as a brand is more challenging than ever. Enter experience marketing – a strategy that transcends traditional advertising by focusing on creating immersive, memorable interactions. This innovative approach leverages the elements of surprise, delight, and reciprocity to forge strong emotional connections with customers, making the sale of your core product feel effortless. But how can businesses implement this strategy effectively? This guide delves into the art of crafting ‘wow’ moments that captivate audiences and transform customer engagement.

The Basics of Experience Marketing

Experience marketing is an evolved form of marketing that focuses on creating meaningful interactions with customers, aiming to elicit strong emotional responses that lead to brand loyalty and advocacy. Unlike conventional marketing, which often prioritizes product promotion, experience marketing centers on the customer’s holistic journey with the brand, creating a narrative that resonates on a personal level.

In today’s competitive market, experience marketing is not just beneficial; it’s essential. It differentiates your brand in a crowded marketplace, elevating your offerings beyond mere commodities to become integral parts of your customers’ lives. Through memorable experiences, you not only attract attention but also foster a community of loyal customers who are more likely to return and recommend your brand to others.

Principles of Experience Marketing

At the heart of experience marketing lie several key principles:

  • Emotional Connection: Crafting campaigns that touch on human emotions, from joy to surprise, creating memorable moments that customers are eager to share.
  • Customer-Centricity: Putting the customer’s needs and desires at the forefront of every marketing strategy, ensuring that each interaction adds value and enhances their experience with the brand.
  • Immersive Experiences: Utilizing technology and storytelling to create immersive experiences that captivate customers, making your brand a living part of their world.
  • Engagement Across Touchpoints: Ensuring consistent, engaging experiences across all customer touchpoints, from digital platforms to physical stores.

Understanding Your Audience

Before diving into the intricacies of crafting ‘wow’ moments, it’s crucial to understand who you’re creating these moments for. Identifying your audience’s pain points and desires is the first step in tailoring experiences that truly resonate.

1709033181 544 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 544 Crafting Effortless Sales Through Wow Moments in Experience Marketing

This involves deep market research, customer interviews, and leveraging data analytics to paint a comprehensive picture of your target demographic. By understanding the journey your customers are on, you can design touchpoints that not only meet but exceed their expectations.

  • Identifying Pain Points and Desires: Use surveys, social media listening, and customer feedback to gather insights. What frustrates your customers about your industry? What do they wish for more than anything else? These insights will guide your efforts to create experiences that truly resonate.
  • Mapping the Customer Journey: Visualize every step a customer takes from discovering your brand to making a purchase and beyond. This map will highlight critical touchpoints where you can introduce ‘wow’ moments that transform the customer experience.

Developing Your Experience Marketing Strategy

With a clear understanding of your audience, it’s time to build the framework of your experience marketing strategy. This involves setting clear objectives, identifying key customer touchpoints, and conceptualizing the experiences you want to create.

  • Setting Objectives: Define what you aim to achieve with your experience marketing efforts. Whether it’s increasing brand awareness, boosting sales, or improving customer retention, having clear goals will shape your approach and help measure success.
  • Strategic Touchpoint Identification: List all the potential touchpoints where customers interact with your brand, from social media to in-store experiences. Consider every stage of the customer journey and look for opportunities to enhance these interactions.

Enhancing Customer Experiences with Surprise, Delight, and Reciprocity

This section is where the magic happens. By integrating the elements of surprise, delight, and reciprocity, you can elevate ordinary customer interactions into unforgettable experiences.

1709033181 790 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 790 Crafting Effortless Sales Through Wow Moments in Experience Marketing
  • Incorporating Surprise and Delight: Go beyond what’s expected. This could be as simple as a personalized thank-you note with each purchase or as elaborate as a surprise gift for loyal customers. The key is to create moments that feel special and unexpected.
  • Applying the Principle of Reciprocity: When customers receive something of value, they’re naturally inclined to give something back. This can be leveraged by offering helpful resources, exceptional service, or customer appreciation events. Such gestures encourage loyalty and positive word-of-mouth.
  • Examples and Case Studies: Highlight real-world examples of brands that have successfully implemented these strategies. Analyze what they did, why it worked, and how it impacted their relationship with customers.

Best Practices for Experience Marketing

To ensure your experience marketing strategy is as effective as possible, it’s important to adhere to some best practices.

  • Personalization at Scale: Leverage data and technology to personalize experiences without losing efficiency. Tailored experiences make customers feel valued and understood.
  • Using Technology to Enhance Experiences: From augmented reality (AR) to mobile apps, technology offers myriad ways to create immersive experiences that surprise and engage customers.
  • Measuring Success: Utilize analytics tools to track the success of your experience marketing initiatives. Key performance indicators (KPIs) could include engagement rates, conversion rates, and customer satisfaction scores.

Section 5: Overcoming Common Challenges

Even the best-laid plans can encounter obstacles. This section addresses common challenges in experience marketing and how to overcome them.

1709033181 656 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 656 Crafting Effortless Sales Through Wow Moments in Experience Marketing
  • Budget Constraints: Learn how to create impactful experiences without breaking the bank. It’s about creativity, not just expenditure.
  • Maintaining Consistency: Ensuring a consistent brand experience across all touchpoints can be daunting. Develop a comprehensive brand guideline and train your team accordingly.
  • Staying Ahead of Trends: The digital landscape is ever-changing. Stay informed about the latest trends in experience marketing and be ready to adapt your strategy as necessary.

The Path to Effortless Sales

By creating memorable experiences that resonate on a personal level, you make the path to purchase not just easy but natural. When customers feel connected to your brand, appreciated, and valued, making a sale becomes a byproduct of your relationship with them. Experience marketing, when done right, transforms transactions into interactions, customers into advocates, and products into passions.

Now is the time to reassess your marketing strategy. Are you just selling a product, or are you providing an unforgettable experience? Dive into the world of experience marketing and start creating those ‘wow’ moments that will not only distinguish your brand but also make sales feel effortless.


Disruptive Design Raising the Bar of Content Marketing with Graphic

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

The Current State of Google’s Search Generative Experience [What It Means for SEO in 2024]

Published

on

The Current State of Google's Search Generative Experience [What It Means for SEO in 2024]

SEO enthusiasts, known for naming algorithm updates after animals and embracing melodrama, find themselves in a landscape where the “adapt or die” mantra prevails. So when Google announced the launch of its Search Generative Experience (SGE) in May of 2023 at Google/IO, you can imagine the reaction was immense.

Although SGE has the potential to be a truly transformative force in the landscape, we’re still waiting for SGE to move out of the Google Labs Sandbox and integrate into standard search results. 

Curious about our current take on SGE and its potential impact on SEO in the future? Read on for more.

Decoding Google’s Defensive Move

In response to potential threats from competitors like ChatGPT, Bing, TikTok, Reddit, and Amazon, Google introduced SGE as a defensive maneuver. However, its initial beta release raised questions about its readiness and global deployment.

ChatGPT provided an existential threat that had the potential to eat into Google’s market share. When Bing started incorporating it into its search results, it was one of the most significant wins for Bing in a decade. In combination with threats from TikTok, Reddit, and Amazon, we see a more fractured search landscape less dominated by Google. Upon its launch, the expectation was that Google would push its SGE solution globally, impact most queries, and massively shake up organic search results and strategies to improve organic visibility.

Advertisement

Now, industry leaders are starting to question if Google is better off leaving SGE in the testing ground in Google labs. According to Google’s recent update, it appears that SGE will remain an opt-in experience in Google Labs (for at least the short term). If SGE was released, there could be a fundamental reset in understanding SEO. Everything from organic traffic to optimization tactics to tracking tools would need adjustments for the new experience. Therefore, the prospect of SGE staying in Google Labs is comforting if not entirely reliable. 

The ever-present option is that Google can change its mind at any point and push SGE out broadly as part of its standard search experience. For this reason, we see value in learning from our observations with SGE and continuing to stay on top of the experience.

SGE User Experience and Operational Challenges

If you’ve signed up for search labs and have been experimenting with SGE for a while, you know firsthand there are various issues that Google should address before rolling it out broadly to the public.

At a high level, these issues fall into two broad categories including user experience issues and operational issues.

Below are some significant issues we’ve come across, with Google making notable progress in addressing certain ones, while others still require improvement:

  • Load time – Too many AI-generated answers take longer to load than a user is willing to wait. Google recommends less than a 3-second load time to meet expectations. They’ll need to figure out how to consistently return results quickly if they want to see a higher adoption rate.
  • Layout – The SGE layout is massive. We believe any major rollout will be more streamlined to make it a less intrusive experience for users and allow more visibility for ads, and if we’re lucky, organic results. Unfortunately, there is still a decent chance that organic results will move below the fold, especially on mobile devices. Recently, Google has incorporated more results where users are prompted to generate the AI result if they’d like to see it. The hope is Google makes this the default in the event of a broad rollout where users can generate an AI result if they want one instead of assuming that’s what a user would like to see. 
  • Redundancy – The AI result duplicates features from the map pack and quick answer results. 
  • Attribution – Due to user feedback, Google includes sources on several of their AI-powered overviews where you can see relevant web pages if there is an arrow next to the result. Currently, the best way to appear as one of these relevant pages is to be one of the top-ranked results, which is convenient from an optimization standpoint. Changes to how attribution and sourcing are handled could heavily impact organic strategies. 

 

On the operational side, Google also faces significant hurdles to making SGE a viable product for its traditional search product. The biggest obstacle appears to be making the cost associated with the technology worth the business outcomes it provides. If this was a necessary investment to maintain market share, Google might be willing to eat the cost, but if their current position is relatively stable, Google doesn’t have much of an incentive to take on the additional cost burden of heavily leveraging generative AI while also presumably taking a hit to their ad revenue. Especially since slow user adoption doesn’t indicate this is something users are demanding at the moment.

Advertisement

While the current experience of SGE is including ads above the generative results now, the earliest iterations didn’t heavily feature sponsored ads. While they are now included, the current SGE layout would still significantly disrupt the ad experience we’re used to. During the Google I/O announcement, they made a statement to reassure advertisers they would be mindful of maintaining a distinct ad experience in search.  

“In this new generative experience, Search ads will continue to appear in dedicated ad slots throughout the page. And we’ll continue to uphold our commitment to ads transparency and making sure ads are distinguishable from organic search results” – Elizabeth Reid, VP, Search at Google

Google is trying to thread a delicate needle here of staying on the cutting edge with their search features, while trying not to upset their advertisers and needlessly hinder their own revenue stream. Roger Montti details more of the operational issues in a recent article digging into the surprising reasons SGE is stuck in Google Labs.

He lists three big problems that need to be solved before SGE will be integrated into the foreground of search:

  1. Large Language Models being inadequate as an information retrieval system
  2. The inefficiency and cost of transformer architecture
  3. Hallucinating (providing inaccurate answers)

 

Until SGE provides more user value and checks more boxes on the business sense side, the traditional search experience is here to stay. Unfortunately, we don’t know when or if Google will ever feel confident they’ve addressed all of these concerns, so we’ll need to stay prepared for change.

Experts Chime in on Search Generative Experience

Our team has been actively engaging with SGE, here’s a closer look at their thoughts and opinions on the experience so far:

Advertisement

“With SGE still in its early stages, I’ve noticed consistent changes in how the generative results are produced and weaved naturally into the SERPs. Because of this, I feel it is imperative to stay on top of these on-going changes to ensure we can continue to educate our clients on what to expect when SGE is officially incorporated into our everyday lives. Although an official launch date is currently unknown, I believe proactively testing various prompt types and recording our learnings is important to prepare our clients for this next evolution of Google search.” – Jon Pagano, SEO Sr. Specialist at Tinuiti

“It’s been exciting to watch SGE grow through different variations over the last year, but like other AI solutions its potential still outweighs its functionality and usefulness. What’s interesting to see is that SGE doesn’t just cite its sources of information, but also provides an enhanced preview of each webpage referenced. This presents a unique organic opportunity where previously untouchable top 10 rankings are far more accessible to the average website. Time will tell what the top ranking factors for SGE are, but verifiable content with strong E-E-A-T signals will be imperative. –Kate Fischer, SEO Specialist at Tinuiti

“Traditionally, AI tools were very good at analytical tasks. With the rise of ChatGPT, users can have long-form, multi-question conversations not yet available in search results. When, not if, released, Google’s Generative Experience will transform how we view AI and search. Because there are so many unknowns, some of the most impactful ways we prepare our clients are to discover and develop SEO strategies that AI tools can’t directly disrupt, like mid to low funnel content.” – Brandon Miller, SEO Specialist at Tinuiti

“SGE is going to make a huge impact on the ecommerce industry by changing the way users interact with the search results. Improved shopping experience will allow users to compare products, price match, and read reviews in order to make it quicker and easier for a user to find the best deals and purchase. Although this leads to more competitive results, it also improves organic visibility and expands our product reach. It is more important than ever to ensure all elements of a page are uniquely and specifically optimized for search. With the SGE updates expected to continue to impact search results, the best way to stay ahead is by focusing on strong user focused content and detailed product page optimizations.”  – Kellie Daley, SEO Sr. Specialist at Tinuiti

Navigating the Clash of Trends

One of the most interesting aspects of the generative AI trend in search is that it appears to be in direct opposition to other recent trends.

Advertisement

One of the ways Google has historically evaluated the efficacy of its search ranking systems is through the manual review of quality raters. In their quality rater guidelines, raters were instructed to review for things like expertise, authority, and trustworthiness (EAT) in results to determine if Google results are providing users the information they deserve. 

In 2022, Google updated their search guidelines to include another ‘e’ in the form of experience (EEAT). In their words, Google wanted to better assess if the content a user was consuming was created by someone with, “a degree of experience, such as with actual use of a product, having actually visited a place or communicating what a person has experienced. There are some situations where really what you value most is content produced by someone who has firsthand, life experience on the topic at hand.” 

Generative AI results, while cutting-edge technology and wildly impressive in some cases, stand in direct opposition to the principles of E-E-A-T. That’s not to say that there’s no room for both in search, but Google will have to determine what it thinks users value more between these competing trends. The slow adoption of SGE could be an indication that a preference for human experience, expertise, authority, and trust is winning round one in this fight. 

Along these lines, Google is also diversifying its search results to cater to the format in which users get their information. This takes the form of their Perspectives Filter. Also announced at Google I/O 2023, the perspectives filter incorporates more video, image, and discussion board posts from places like TikTok, YouTube, Reddit, and Quora. Once again, this trend shows the emphasis and value searchers place on experience and perspective. Users value individual experience over the impersonal conveyance of information. AI will never have these two things, even if it can provide a convincing imitation.

The current iteration of SGE seems to go too far in dismissing these trends in favor of generative AI. It’s an interesting challenge Google faces. If they don’t determine the prevailing trend correctly, veering too far in one direction can push more market share to ChatGPT or platforms like YouTube and TikTok.

Final Thoughts

The range of outcomes remains broad and fascinating for SGE. We can see this developing in different ways, and prognostication offers little value, but it’s invaluable to know the potential outcomes and prepare for as many of them as possible.

Advertisement

It’s critical that you or your search agency be interacting and experimenting with SGE because:

  • The format and results will most likely continue to see significant changes
  • This space moves quickly and it’s easy to fall behind
  • Google may fix all of the issues with SGE and decide to push it live, changing the landscape of search overnight
  • SGE experiments could inform other AI elements incorporated into the search experience

 

Ultimately, optimizing for the specific SGE experience we see now is less important because we know it will inevitably continue changing. We see more value in recognizing the trends and problems Google is trying to solve with this technology. With how quickly this space moves, any specifics mentioned in this article could be outdated in a week. That’s why focusing on intention and process is important at this stage of the game.

By understanding the future needs and wants SGE is attempting to address, we can help you future-proof your search strategies as much as possible. To some extent we’re always at the whims of the algorithm, but by maintaining a user-centric approach, you can make your customers happy, regardless of how they find you.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

Here’s Optimizely’s Automatic Sample Ratio Mismatch Detection

Published

on

Here's Optimizely’s Automatic Sample Ratio Mismatch Detection

Optimizely Experiment’s automatic sample ratio mismatch (SRM) detection delivers peace of mind to experimenters. It reduces a user’s exposure time to bad experiences by rapidly detecting any experiment deterioration.

This deterioration is caused by unexpected imbalances of visitors to a variation in an experiment. Most importantly, this auto SRM detection empowers product managers, marketers, engineers, and experimentation teams to confidently launch more experiments. 

How Optimizely Experiment’s stats engine and automatic sample rate mismatch detection work together

The sample ratio mismatch actslike the bouncer at the door who has a mechanical counter, checking guests’ tickets (users) and telling them which room they get to party in.

Stats engine is like the party host who is always checking the vibes (behavior) of the guests as people come into the room.

Advertisement

If SRM does its job right, then stats engine can confidently tell which party room is better and direct more traffic to the winning variation (the better party) sooner.

Why would I want Optimizely Experiment’s SRM detection?

It’s equally important to ensure Optimizely Experiment users know their experiment results are trustworthy and have the tools to understand what an imbalance can mean for their results and how to prevent it.

Uniquely, Optimizely Experiment goes further by combining the power of automatic visitor imbalance detection with an insightful experiment health indicator. This experiment health indicator plays double duty by letting our customers know when all is well and there is no imbalance present.

Then, when just-in-time insight is needed to protect your business decisions, Optimizely also delivers just-in-time alerts that help our customers recognize the severity of, diagnose, and recover from errors.

Why should I care about sample ratio mismatch (SRM)?

Just like a fever is a symptom of many illnesses, a SRM is a symptom of a variety of data quality issues. Ignoring a SRM without knowing the root cause may result in a bad feature appearing to be good and being shipped out to users, or vice versa. Finding an experiment with an unknown source of traffic imbalance lets you turn it off quickly and reduce the blast radius.

Then what is the connection between a “mismatch” and “sample ratio”?

When we get ready to launch an experiment, we assign a traffic split of users for Optimizely Experiment to distribute to each variation. We expect the assigned traffic split to reasonably match up with the actual traffic split in a live experiment. An experiment is exposed to an SRM imbalance when there is a statistically significant difference between the expected and the actual assigned traffic splits of visitors to an experiment’s variations.

Advertisement
1. A mismatch doesn’t mean an imperfect match

Remember: A bonified imbalance requires a statistically significant result of the difference in visitors. Don’t expect a picture-perfect, identical, exact match of the launch-day traffic split to your in-production traffic split. There will always be some ever-so-slight deviation.

Not every traffic disparity automatically signifies that an experiment is useless. Because Optimizely deeply values our customers’ time and energy, we developed a new statistical test that continuously monitors experiment results and detects harmful SRMs as early as possible. All while still controlling for crying wolf over false positives (AKA when we conclude there is a surprising difference between a test variation and the baseline when there is no real difference). 

2. Going under the hood of Optimizely Experiment’s SRM detection algorithm

Optimizely Experiment’s automatic SRM detection feature employs a sequential Bayesian multinomial test (say that 5 times fast!), named sequential sample ratio mismatch. Optimizely statisticians Michael Lindon and Alen Malek pioneered this method, and it is a new contribution to the field of Sequential Statistics. Optimizely Experiment’s sample ratio mismatch detection harmonizes sequential and Bayesian methodologies by continuously checking traffic counts and testing for any significant imbalance in a variation’s visitor counts. The algorithm’s construction is Bayesian inspired to account for an experiment’s optional stopping and continuation while delivering sequential guarantees of Type-I error probabilities.

3. Beware of chi-eap alternatives!

The most popular freely available SRM calculators employ the chi-square test. We highly recommend a careful review of the mechanics of chi-square testing. The main issue with the chi-squared method is that problems are discovered only after collecting all the data. This is arguably far too late and goes against why most clients want SRM detention in the first place. In our blog post “A better way to test for sample ratio mismatches (or why I don’t use a chi-squared test)”, we go deeper into chi-square mechanics and how what we built accounts for the gaps left behind by the alternatives.

Common causes of an SRM  

1. Redirects & Delays

A SRM usually results from some visitors closing out and leaving the page before the redirect finishes executing. Because we only send the decision events once they arrive on the page and Optimizely Experiment loads, we can’t count these visitors in our results page unless they return at some point and send an event to Optimizely Experiment.

A SRM can emerge in the case of anything that would cause Optimizely Experiment’s event calls to delay or not fire, such as variation code changes. It also occurs when redirect experiments shuttle visitors to a different domain. This occurrence is exacerbated by slow connection times.

Advertisement
2. Force-bucketing

If a user first gets bucketed in the experiment and then that decision is used to force-bucket them in a subsequent experiment, then the results of that subsequent experiment will become imbalanced.

Here’s an example:

Variation A provides a wildly different user experience than Variation B.

Visitors bucketed into Variation A have a great experience, and many of them continue to log in and land into the subsequent experiment where they’re force-bucketed into Variation A.

But, visitors who were bucketed into Variation B aren’t having a good experience. Only a few users log in and land into a subsequent experiment where they will be force-bucketed into Variation B.

Well, now you have many more visitors in Variation A than in Variation B.

Advertisement
3. Site has its own redirects

Some sites have their own redirects (for example, 301s) that, combined with our redirects, can result in a visitor landing on a page without the snippet. This causes pending decision events to get locked in localStorage and Optimizely Experiment never receives or counts them.

4. Hold/send events API calls are housed outside of the snippet

Some users include hold/send events in project JS. However, others include it in other scripts on the page, such as in vendor bundles or analytics tracking scripts. This represents another script that must be properly loaded for the decisions to fire appropriately. Implementation or loading rates may differ across variations, particularly in the case of redirects.

Interested?  

If you’re already an Optimizely Experiment customer and you’d like to learn more about how automatic SRM detection benefits your A/B tests, check out our knowledge base documentation:

For further details you can always reach out to your customer success manager but do take a moment to review our documentation first!

If you’re not a customer, get started with us here! 

And if you’d like to dig deeper into the engine that powers Optimizely experimentation, you can check out our page faster decisions you can trust for digital experimentation. 

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS