Connect with us

MARKETING

The Professional’s Guide to SEO: Advanced SEO Strategy Sneak Preview

Published

on

The Professional’s Guide to SEO: Advanced SEO Strategy Sneak Preview

If you know Moz, you know the Beginner’s Guide to SEO. It’s the resource marketers the world over have used to learn SEO and get a taste for its potential and power. And while we offer a delectable buffet of guides in our content smörgåsbord, there hasn’t been one comprehensive resource to serve as a follow-up for those who’ve mastered the beginner level. That’s why we’ve developed the Professional’s Guide to SEO: a guide that will help folks take the next step, preparing them with all the baseline knowledge they need to practice SEO in a professional capacity.

Over the coming weeks, we’ll be sharing chapters and/or chapter excerpts here on the blog, with the full guide releasing at the end. Yes, we want to whet your appetite, but we’d also love to hear your feedback — if there’s something you know in your heart of hearts we should cover, get at us on Twitter (@moz) and let us know!

First up, we’re sharing a portion of our chapter on advanced SEO strategy. Brought to you by the inimitable Kavi Kardos, Moz alumni and SEO Manager at Automox, this chapter looks at getting started with a next-step strategy, tactics to implement, and resources for leveling up.

Bon appétit!

Advanced SEO Strategy

Getting started: SEO priorities & plausibility

In the beginning stages, it’s easy to audit a site and come up with long lists of pie-in-the-sky ideas for content, link building, technical, and so on. Most sites, especially those that have never been handled by an advanced SEO, need a lot of work, and the new strategist arriving on the scene often gets pulled in several directions by various teams seeking their expertise.

Advertisement

Prioritization of the tasks you’ll undertake and the tactics you’ll employ is a vital first step in developing an advanced SEO strategy. And it’s important to work on this step thoughtfully — ask questions, be realistic, and involve as many stakeholders as you’re able to meet with. A misstep in the prioritization stage can throw off your schedule for the whole quarter and cause important tasks to fall through the cracks.

Try a SWOT analysis

It may feel old-fashioned, but the classic SWOT analysis (identifying strengths, weaknesses, opportunities, and threats) is a great way to frame your initial site audit because it will familiarize you with both the website itself and the competitive landscape in which it lives. As you explore both, jot down your thoughts in a Google Doc that you can return to whenever you discover something new.

  • Strengths: What is already working well? What high-value terms does the site already rank for? What high-authority sites already link to you? Does the site already score well in page speed and performance tests or avoid other common technical snafus?

  • Weaknesses: What is the site lacking? Is it difficult to navigate? Are its sitemaps and robots.txt file messy? Is the organization lacking insight because it doesn’t make use of basic reporting tools like Google Analytics? Does it have a lackluster content strategy?

  • Opportunities: What’s on the horizon that could be capitalized on as part of your strategy? Is there a highly valuable asset that’s already been created and is now begging for distribution? Is a tough competitor lagging behind in a certain content area?

  • Threats: What’s on the horizon that could be harmful to your search visibility? Is there an up-and-coming competitor with an obvious wealth of SEO resources? Is there a platform migration looming? Is the site likely to fall victim to the next algorithm update?

Being conscious of the site’s current standings, both in the SERPs and in terms of its overall health, will help you prioritize tactics based on urgency. The most dire threats should usually be addressed first, while minor weaknesses can often be moved to your “nice to have” list.

Assess the organization’s search maturity

Regardless of how urgent the need is or how simple a task seems to you, the difficulty of getting your SEO recommendations implemented will vary from organization to organization. The plausibility of executing your strategy depends largely on the organization’s search maturity, or how fully they understand and integrate SEO at all levels of the business.

The concept of search maturity was developed by Heather Physioc of VMLY&R, and her guidance on diagnosing where your organization falls along the maturity spectrum is an absolute must-read at this stage in the strategic planning process. Not only does using this model help you solidify your recommendations; it also makes it more likely that those recommendations will see the light of day because it allows you to communicate with stakeholders on their level.

How much buy-in can you expect from your department, your direct manager or client contact, and the rest of the larger team all the way up to the C-suite? If SEO has been socialized across the organization and is already a part of the company culture, you can probably expect your recommendations to be met with excitement. If not, you may experience some pushback when asking for necessary resources. At an agency, you’ll be dealing with the confines of existing SEO packages as well as the amount of time you’re expected to spend on each client each month. As an in-house SEO, you may have more autonomy but must often answer to more stakeholders and navigate more red tape.

Advertisement

How difficult will it be to get recommended changes implemented? If the content team has an existing calendar that tends to be jam-packed, new assets may not get slotted in as quickly as you’d like. If the web devs are slammed, working back-end fixes into their sprint cycle can be challenging.

What resources will be available for SEO? Resources come in many forms, and the most scarce of them tend to be headcount and tools. Are there writers on staff who are capable of creating best-in-class content? Does the marketing team have dedicated developers, or are the folks with access to the site’s code in a totally separate department? What tool subscriptions already exist, and how much budget is available to add to your tool kit?

Create an impact vs. effort matrix

Once you know which areas of the site need the most help the fastest, it’s time to make a list of recommended tactics and further prioritize that list by likely impact weighed against required effort, based on what you learned in the previous step.

Create a matrix like the one above, perhaps in a meeting with relevant stakeholders. The likely impact of a tactic could be small, medium, or large, and the same scale will apply to the level of effort required to complete it. Plot each planned tactic into its own cell. Your list of tactics for the quarter, the year, or whatever time frame is dictated by your organization can include granular tasks as well as larger-scale projects — just make sure you’ve broken down any bigger ideas into pieces that make sense within the plot.

Taking urgency into account, tackle the tactics that will have the highest impact and require the lowest effort first. You may also want to set in motion some more demanding, high-impact tactics at kickoff if they can be chipped away at simultaneously. Low-impact, high-effort tactics can often be reevaluated.


Want more news about the Professional’s Guide to SEO? Don’t miss any of our future sneak peeks — make sure you’re signed up for Moz Blog email updates!

Advertisement

Keep me updated!




Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address

MARKETING

Crafting Effortless Sales Through ‘Wow’ Moments in Experience Marketing

Published

on

Crafting Effortless Sales Through 'Wow' Moments in Experience Marketing

Crafting Effortless Sales Through Wow Moments in Experience Marketing

In an era where consumers are bombarded with endless choices and digital noise, standing out as a brand is more challenging than ever. Enter experience marketing – a strategy that transcends traditional advertising by focusing on creating immersive, memorable interactions. This innovative approach leverages the elements of surprise, delight, and reciprocity to forge strong emotional connections with customers, making the sale of your core product feel effortless. But how can businesses implement this strategy effectively? This guide delves into the art of crafting ‘wow’ moments that captivate audiences and transform customer engagement.

The Basics of Experience Marketing

Experience marketing is an evolved form of marketing that focuses on creating meaningful interactions with customers, aiming to elicit strong emotional responses that lead to brand loyalty and advocacy. Unlike conventional marketing, which often prioritizes product promotion, experience marketing centers on the customer’s holistic journey with the brand, creating a narrative that resonates on a personal level.

In today’s competitive market, experience marketing is not just beneficial; it’s essential. It differentiates your brand in a crowded marketplace, elevating your offerings beyond mere commodities to become integral parts of your customers’ lives. Through memorable experiences, you not only attract attention but also foster a community of loyal customers who are more likely to return and recommend your brand to others.

Principles of Experience Marketing

At the heart of experience marketing lie several key principles:

  • Emotional Connection: Crafting campaigns that touch on human emotions, from joy to surprise, creating memorable moments that customers are eager to share.
  • Customer-Centricity: Putting the customer’s needs and desires at the forefront of every marketing strategy, ensuring that each interaction adds value and enhances their experience with the brand.
  • Immersive Experiences: Utilizing technology and storytelling to create immersive experiences that captivate customers, making your brand a living part of their world.
  • Engagement Across Touchpoints: Ensuring consistent, engaging experiences across all customer touchpoints, from digital platforms to physical stores.

Understanding Your Audience

Before diving into the intricacies of crafting ‘wow’ moments, it’s crucial to understand who you’re creating these moments for. Identifying your audience’s pain points and desires is the first step in tailoring experiences that truly resonate.

1709033181 544 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 544 Crafting Effortless Sales Through Wow Moments in Experience Marketing

This involves deep market research, customer interviews, and leveraging data analytics to paint a comprehensive picture of your target demographic. By understanding the journey your customers are on, you can design touchpoints that not only meet but exceed their expectations.

  • Identifying Pain Points and Desires: Use surveys, social media listening, and customer feedback to gather insights. What frustrates your customers about your industry? What do they wish for more than anything else? These insights will guide your efforts to create experiences that truly resonate.
  • Mapping the Customer Journey: Visualize every step a customer takes from discovering your brand to making a purchase and beyond. This map will highlight critical touchpoints where you can introduce ‘wow’ moments that transform the customer experience.

Developing Your Experience Marketing Strategy

With a clear understanding of your audience, it’s time to build the framework of your experience marketing strategy. This involves setting clear objectives, identifying key customer touchpoints, and conceptualizing the experiences you want to create.

  • Setting Objectives: Define what you aim to achieve with your experience marketing efforts. Whether it’s increasing brand awareness, boosting sales, or improving customer retention, having clear goals will shape your approach and help measure success.
  • Strategic Touchpoint Identification: List all the potential touchpoints where customers interact with your brand, from social media to in-store experiences. Consider every stage of the customer journey and look for opportunities to enhance these interactions.

Enhancing Customer Experiences with Surprise, Delight, and Reciprocity

This section is where the magic happens. By integrating the elements of surprise, delight, and reciprocity, you can elevate ordinary customer interactions into unforgettable experiences.

1709033181 790 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 790 Crafting Effortless Sales Through Wow Moments in Experience Marketing
  • Incorporating Surprise and Delight: Go beyond what’s expected. This could be as simple as a personalized thank-you note with each purchase or as elaborate as a surprise gift for loyal customers. The key is to create moments that feel special and unexpected.
  • Applying the Principle of Reciprocity: When customers receive something of value, they’re naturally inclined to give something back. This can be leveraged by offering helpful resources, exceptional service, or customer appreciation events. Such gestures encourage loyalty and positive word-of-mouth.
  • Examples and Case Studies: Highlight real-world examples of brands that have successfully implemented these strategies. Analyze what they did, why it worked, and how it impacted their relationship with customers.

Best Practices for Experience Marketing

To ensure your experience marketing strategy is as effective as possible, it’s important to adhere to some best practices.

  • Personalization at Scale: Leverage data and technology to personalize experiences without losing efficiency. Tailored experiences make customers feel valued and understood.
  • Using Technology to Enhance Experiences: From augmented reality (AR) to mobile apps, technology offers myriad ways to create immersive experiences that surprise and engage customers.
  • Measuring Success: Utilize analytics tools to track the success of your experience marketing initiatives. Key performance indicators (KPIs) could include engagement rates, conversion rates, and customer satisfaction scores.

Section 5: Overcoming Common Challenges

Even the best-laid plans can encounter obstacles. This section addresses common challenges in experience marketing and how to overcome them.

1709033181 656 Crafting Effortless Sales Through Wow Moments in Experience Marketing1709033181 656 Crafting Effortless Sales Through Wow Moments in Experience Marketing
  • Budget Constraints: Learn how to create impactful experiences without breaking the bank. It’s about creativity, not just expenditure.
  • Maintaining Consistency: Ensuring a consistent brand experience across all touchpoints can be daunting. Develop a comprehensive brand guideline and train your team accordingly.
  • Staying Ahead of Trends: The digital landscape is ever-changing. Stay informed about the latest trends in experience marketing and be ready to adapt your strategy as necessary.

The Path to Effortless Sales

By creating memorable experiences that resonate on a personal level, you make the path to purchase not just easy but natural. When customers feel connected to your brand, appreciated, and valued, making a sale becomes a byproduct of your relationship with them. Experience marketing, when done right, transforms transactions into interactions, customers into advocates, and products into passions.

Now is the time to reassess your marketing strategy. Are you just selling a product, or are you providing an unforgettable experience? Dive into the world of experience marketing and start creating those ‘wow’ moments that will not only distinguish your brand but also make sales feel effortless.


Disruptive Design Raising the Bar of Content Marketing with Graphic

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

The Current State of Google’s Search Generative Experience [What It Means for SEO in 2024]

Published

on

The Current State of Google's Search Generative Experience [What It Means for SEO in 2024]

SEO enthusiasts, known for naming algorithm updates after animals and embracing melodrama, find themselves in a landscape where the “adapt or die” mantra prevails. So when Google announced the launch of its Search Generative Experience (SGE) in May of 2023 at Google/IO, you can imagine the reaction was immense.

Although SGE has the potential to be a truly transformative force in the landscape, we’re still waiting for SGE to move out of the Google Labs Sandbox and integrate into standard search results. 

Curious about our current take on SGE and its potential impact on SEO in the future? Read on for more.

Decoding Google’s Defensive Move

In response to potential threats from competitors like ChatGPT, Bing, TikTok, Reddit, and Amazon, Google introduced SGE as a defensive maneuver. However, its initial beta release raised questions about its readiness and global deployment.

ChatGPT provided an existential threat that had the potential to eat into Google’s market share. When Bing started incorporating it into its search results, it was one of the most significant wins for Bing in a decade. In combination with threats from TikTok, Reddit, and Amazon, we see a more fractured search landscape less dominated by Google. Upon its launch, the expectation was that Google would push its SGE solution globally, impact most queries, and massively shake up organic search results and strategies to improve organic visibility.

Advertisement

Now, industry leaders are starting to question if Google is better off leaving SGE in the testing ground in Google labs. According to Google’s recent update, it appears that SGE will remain an opt-in experience in Google Labs (for at least the short term). If SGE was released, there could be a fundamental reset in understanding SEO. Everything from organic traffic to optimization tactics to tracking tools would need adjustments for the new experience. Therefore, the prospect of SGE staying in Google Labs is comforting if not entirely reliable. 

The ever-present option is that Google can change its mind at any point and push SGE out broadly as part of its standard search experience. For this reason, we see value in learning from our observations with SGE and continuing to stay on top of the experience.

SGE User Experience and Operational Challenges

If you’ve signed up for search labs and have been experimenting with SGE for a while, you know firsthand there are various issues that Google should address before rolling it out broadly to the public.

At a high level, these issues fall into two broad categories including user experience issues and operational issues.

Below are some significant issues we’ve come across, with Google making notable progress in addressing certain ones, while others still require improvement:

  • Load time – Too many AI-generated answers take longer to load than a user is willing to wait. Google recommends less than a 3-second load time to meet expectations. They’ll need to figure out how to consistently return results quickly if they want to see a higher adoption rate.
  • Layout – The SGE layout is massive. We believe any major rollout will be more streamlined to make it a less intrusive experience for users and allow more visibility for ads, and if we’re lucky, organic results. Unfortunately, there is still a decent chance that organic results will move below the fold, especially on mobile devices. Recently, Google has incorporated more results where users are prompted to generate the AI result if they’d like to see it. The hope is Google makes this the default in the event of a broad rollout where users can generate an AI result if they want one instead of assuming that’s what a user would like to see. 
  • Redundancy – The AI result duplicates features from the map pack and quick answer results. 
  • Attribution – Due to user feedback, Google includes sources on several of their AI-powered overviews where you can see relevant web pages if there is an arrow next to the result. Currently, the best way to appear as one of these relevant pages is to be one of the top-ranked results, which is convenient from an optimization standpoint. Changes to how attribution and sourcing are handled could heavily impact organic strategies. 

 

On the operational side, Google also faces significant hurdles to making SGE a viable product for its traditional search product. The biggest obstacle appears to be making the cost associated with the technology worth the business outcomes it provides. If this was a necessary investment to maintain market share, Google might be willing to eat the cost, but if their current position is relatively stable, Google doesn’t have much of an incentive to take on the additional cost burden of heavily leveraging generative AI while also presumably taking a hit to their ad revenue. Especially since slow user adoption doesn’t indicate this is something users are demanding at the moment.

Advertisement

While the current experience of SGE is including ads above the generative results now, the earliest iterations didn’t heavily feature sponsored ads. While they are now included, the current SGE layout would still significantly disrupt the ad experience we’re used to. During the Google I/O announcement, they made a statement to reassure advertisers they would be mindful of maintaining a distinct ad experience in search.  

“In this new generative experience, Search ads will continue to appear in dedicated ad slots throughout the page. And we’ll continue to uphold our commitment to ads transparency and making sure ads are distinguishable from organic search results” – Elizabeth Reid, VP, Search at Google

Google is trying to thread a delicate needle here of staying on the cutting edge with their search features, while trying not to upset their advertisers and needlessly hinder their own revenue stream. Roger Montti details more of the operational issues in a recent article digging into the surprising reasons SGE is stuck in Google Labs.

He lists three big problems that need to be solved before SGE will be integrated into the foreground of search:

  1. Large Language Models being inadequate as an information retrieval system
  2. The inefficiency and cost of transformer architecture
  3. Hallucinating (providing inaccurate answers)

 

Until SGE provides more user value and checks more boxes on the business sense side, the traditional search experience is here to stay. Unfortunately, we don’t know when or if Google will ever feel confident they’ve addressed all of these concerns, so we’ll need to stay prepared for change.

Experts Chime in on Search Generative Experience

Our team has been actively engaging with SGE, here’s a closer look at their thoughts and opinions on the experience so far:

Advertisement

“With SGE still in its early stages, I’ve noticed consistent changes in how the generative results are produced and weaved naturally into the SERPs. Because of this, I feel it is imperative to stay on top of these on-going changes to ensure we can continue to educate our clients on what to expect when SGE is officially incorporated into our everyday lives. Although an official launch date is currently unknown, I believe proactively testing various prompt types and recording our learnings is important to prepare our clients for this next evolution of Google search.” – Jon Pagano, SEO Sr. Specialist at Tinuiti

“It’s been exciting to watch SGE grow through different variations over the last year, but like other AI solutions its potential still outweighs its functionality and usefulness. What’s interesting to see is that SGE doesn’t just cite its sources of information, but also provides an enhanced preview of each webpage referenced. This presents a unique organic opportunity where previously untouchable top 10 rankings are far more accessible to the average website. Time will tell what the top ranking factors for SGE are, but verifiable content with strong E-E-A-T signals will be imperative. –Kate Fischer, SEO Specialist at Tinuiti

“Traditionally, AI tools were very good at analytical tasks. With the rise of ChatGPT, users can have long-form, multi-question conversations not yet available in search results. When, not if, released, Google’s Generative Experience will transform how we view AI and search. Because there are so many unknowns, some of the most impactful ways we prepare our clients are to discover and develop SEO strategies that AI tools can’t directly disrupt, like mid to low funnel content.” – Brandon Miller, SEO Specialist at Tinuiti

“SGE is going to make a huge impact on the ecommerce industry by changing the way users interact with the search results. Improved shopping experience will allow users to compare products, price match, and read reviews in order to make it quicker and easier for a user to find the best deals and purchase. Although this leads to more competitive results, it also improves organic visibility and expands our product reach. It is more important than ever to ensure all elements of a page are uniquely and specifically optimized for search. With the SGE updates expected to continue to impact search results, the best way to stay ahead is by focusing on strong user focused content and detailed product page optimizations.”  – Kellie Daley, SEO Sr. Specialist at Tinuiti

Navigating the Clash of Trends

One of the most interesting aspects of the generative AI trend in search is that it appears to be in direct opposition to other recent trends.

Advertisement

One of the ways Google has historically evaluated the efficacy of its search ranking systems is through the manual review of quality raters. In their quality rater guidelines, raters were instructed to review for things like expertise, authority, and trustworthiness (EAT) in results to determine if Google results are providing users the information they deserve. 

In 2022, Google updated their search guidelines to include another ‘e’ in the form of experience (EEAT). In their words, Google wanted to better assess if the content a user was consuming was created by someone with, “a degree of experience, such as with actual use of a product, having actually visited a place or communicating what a person has experienced. There are some situations where really what you value most is content produced by someone who has firsthand, life experience on the topic at hand.” 

Generative AI results, while cutting-edge technology and wildly impressive in some cases, stand in direct opposition to the principles of E-E-A-T. That’s not to say that there’s no room for both in search, but Google will have to determine what it thinks users value more between these competing trends. The slow adoption of SGE could be an indication that a preference for human experience, expertise, authority, and trust is winning round one in this fight. 

Along these lines, Google is also diversifying its search results to cater to the format in which users get their information. This takes the form of their Perspectives Filter. Also announced at Google I/O 2023, the perspectives filter incorporates more video, image, and discussion board posts from places like TikTok, YouTube, Reddit, and Quora. Once again, this trend shows the emphasis and value searchers place on experience and perspective. Users value individual experience over the impersonal conveyance of information. AI will never have these two things, even if it can provide a convincing imitation.

The current iteration of SGE seems to go too far in dismissing these trends in favor of generative AI. It’s an interesting challenge Google faces. If they don’t determine the prevailing trend correctly, veering too far in one direction can push more market share to ChatGPT or platforms like YouTube and TikTok.

Final Thoughts

The range of outcomes remains broad and fascinating for SGE. We can see this developing in different ways, and prognostication offers little value, but it’s invaluable to know the potential outcomes and prepare for as many of them as possible.

Advertisement

It’s critical that you or your search agency be interacting and experimenting with SGE because:

  • The format and results will most likely continue to see significant changes
  • This space moves quickly and it’s easy to fall behind
  • Google may fix all of the issues with SGE and decide to push it live, changing the landscape of search overnight
  • SGE experiments could inform other AI elements incorporated into the search experience

 

Ultimately, optimizing for the specific SGE experience we see now is less important because we know it will inevitably continue changing. We see more value in recognizing the trends and problems Google is trying to solve with this technology. With how quickly this space moves, any specifics mentioned in this article could be outdated in a week. That’s why focusing on intention and process is important at this stage of the game.

By understanding the future needs and wants SGE is attempting to address, we can help you future-proof your search strategies as much as possible. To some extent we’re always at the whims of the algorithm, but by maintaining a user-centric approach, you can make your customers happy, regardless of how they find you.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

MARKETING

Here’s Optimizely’s Automatic Sample Ratio Mismatch Detection

Published

on

Here's Optimizely’s Automatic Sample Ratio Mismatch Detection

Optimizely Experiment’s automatic sample ratio mismatch (SRM) detection delivers peace of mind to experimenters. It reduces a user’s exposure time to bad experiences by rapidly detecting any experiment deterioration.

This deterioration is caused by unexpected imbalances of visitors to a variation in an experiment. Most importantly, this auto SRM detection empowers product managers, marketers, engineers, and experimentation teams to confidently launch more experiments. 

How Optimizely Experiment’s stats engine and automatic sample rate mismatch detection work together

The sample ratio mismatch actslike the bouncer at the door who has a mechanical counter, checking guests’ tickets (users) and telling them which room they get to party in.

Stats engine is like the party host who is always checking the vibes (behavior) of the guests as people come into the room.

Advertisement

If SRM does its job right, then stats engine can confidently tell which party room is better and direct more traffic to the winning variation (the better party) sooner.

Why would I want Optimizely Experiment’s SRM detection?

It’s equally important to ensure Optimizely Experiment users know their experiment results are trustworthy and have the tools to understand what an imbalance can mean for their results and how to prevent it.

Uniquely, Optimizely Experiment goes further by combining the power of automatic visitor imbalance detection with an insightful experiment health indicator. This experiment health indicator plays double duty by letting our customers know when all is well and there is no imbalance present.

Then, when just-in-time insight is needed to protect your business decisions, Optimizely also delivers just-in-time alerts that help our customers recognize the severity of, diagnose, and recover from errors.

Why should I care about sample ratio mismatch (SRM)?

Just like a fever is a symptom of many illnesses, a SRM is a symptom of a variety of data quality issues. Ignoring a SRM without knowing the root cause may result in a bad feature appearing to be good and being shipped out to users, or vice versa. Finding an experiment with an unknown source of traffic imbalance lets you turn it off quickly and reduce the blast radius.

Then what is the connection between a “mismatch” and “sample ratio”?

When we get ready to launch an experiment, we assign a traffic split of users for Optimizely Experiment to distribute to each variation. We expect the assigned traffic split to reasonably match up with the actual traffic split in a live experiment. An experiment is exposed to an SRM imbalance when there is a statistically significant difference between the expected and the actual assigned traffic splits of visitors to an experiment’s variations.

Advertisement
1. A mismatch doesn’t mean an imperfect match

Remember: A bonified imbalance requires a statistically significant result of the difference in visitors. Don’t expect a picture-perfect, identical, exact match of the launch-day traffic split to your in-production traffic split. There will always be some ever-so-slight deviation.

Not every traffic disparity automatically signifies that an experiment is useless. Because Optimizely deeply values our customers’ time and energy, we developed a new statistical test that continuously monitors experiment results and detects harmful SRMs as early as possible. All while still controlling for crying wolf over false positives (AKA when we conclude there is a surprising difference between a test variation and the baseline when there is no real difference). 

2. Going under the hood of Optimizely Experiment’s SRM detection algorithm

Optimizely Experiment’s automatic SRM detection feature employs a sequential Bayesian multinomial test (say that 5 times fast!), named sequential sample ratio mismatch. Optimizely statisticians Michael Lindon and Alen Malek pioneered this method, and it is a new contribution to the field of Sequential Statistics. Optimizely Experiment’s sample ratio mismatch detection harmonizes sequential and Bayesian methodologies by continuously checking traffic counts and testing for any significant imbalance in a variation’s visitor counts. The algorithm’s construction is Bayesian inspired to account for an experiment’s optional stopping and continuation while delivering sequential guarantees of Type-I error probabilities.

3. Beware of chi-eap alternatives!

The most popular freely available SRM calculators employ the chi-square test. We highly recommend a careful review of the mechanics of chi-square testing. The main issue with the chi-squared method is that problems are discovered only after collecting all the data. This is arguably far too late and goes against why most clients want SRM detention in the first place. In our blog post “A better way to test for sample ratio mismatches (or why I don’t use a chi-squared test)”, we go deeper into chi-square mechanics and how what we built accounts for the gaps left behind by the alternatives.

Common causes of an SRM  

1. Redirects & Delays

A SRM usually results from some visitors closing out and leaving the page before the redirect finishes executing. Because we only send the decision events once they arrive on the page and Optimizely Experiment loads, we can’t count these visitors in our results page unless they return at some point and send an event to Optimizely Experiment.

A SRM can emerge in the case of anything that would cause Optimizely Experiment’s event calls to delay or not fire, such as variation code changes. It also occurs when redirect experiments shuttle visitors to a different domain. This occurrence is exacerbated by slow connection times.

Advertisement
2. Force-bucketing

If a user first gets bucketed in the experiment and then that decision is used to force-bucket them in a subsequent experiment, then the results of that subsequent experiment will become imbalanced.

Here’s an example:

Variation A provides a wildly different user experience than Variation B.

Visitors bucketed into Variation A have a great experience, and many of them continue to log in and land into the subsequent experiment where they’re force-bucketed into Variation A.

But, visitors who were bucketed into Variation B aren’t having a good experience. Only a few users log in and land into a subsequent experiment where they will be force-bucketed into Variation B.

Well, now you have many more visitors in Variation A than in Variation B.

Advertisement
3. Site has its own redirects

Some sites have their own redirects (for example, 301s) that, combined with our redirects, can result in a visitor landing on a page without the snippet. This causes pending decision events to get locked in localStorage and Optimizely Experiment never receives or counts them.

4. Hold/send events API calls are housed outside of the snippet

Some users include hold/send events in project JS. However, others include it in other scripts on the page, such as in vendor bundles or analytics tracking scripts. This represents another script that must be properly loaded for the decisions to fire appropriately. Implementation or loading rates may differ across variations, particularly in the case of redirects.

Interested?  

If you’re already an Optimizely Experiment customer and you’d like to learn more about how automatic SRM detection benefits your A/B tests, check out our knowledge base documentation:

For further details you can always reach out to your customer success manager but do take a moment to review our documentation first!

If you’re not a customer, get started with us here! 

And if you’d like to dig deeper into the engine that powers Optimizely experimentation, you can check out our page faster decisions you can trust for digital experimentation. 

Advertisement

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading

Trending

Follow by Email
RSS