Connect with us


How to Hire Freelance Writers in 5 Steps (Ahrefs’ Process)



How to Hire Freelance Writers in 5 Steps (Ahrefs' Process)

If you’ve ever tried to hire freelance writers, you probably ended up with thousands of applications.

This usually leads to you asking yourself two questions: 

  1. How do I choose the best ones?
  2. How will I even have time to filter through that many applications?

In this post, you’ll learn how we set up a process to answer these questions.

Step 1. Create an application form

Most job listings tell applicants to email the hiring manager their cover letter and resume. That works well when looking for full-time hires but not for freelance positions because they often attract hundreds or thousands of applicants. 

There’s just no way you can review that many resumes without losing it. As a result, suitable applicants often slip through the net and you miss out on great writers. 

One way to solve this problem is to have all applicants apply through a Google Form. 

Here are two reasons why this makes sense: 

  1. It helps you vet applicants fast and efficiently – Google Forms funnels applicants into a Google Sheet, so you can review all applicants there instead of sifting through emails. 
  2. It allows you to test writers’ skills and knowledge at scale – You can easily ask questions designed to judge applicants’ knowledge, skills, and experience with the topic at hand.

In terms of what you should ask applicants, I recommend a mix of multiple-choice questions and ones with short answers. Multiple-choice questions help gauge knowledge and experience, whereas “short answer” questions allow you to judge writing ability. 

Here’s one of our multiple-choice questions:

Example of a multiple-choice question in our application process

The answer to this question is something that we expect most experienced SEOs to know. But it may trip up those with less experience. 


There’s still a bit of “it depends” with this question. But generally speaking, there’s a clear and correct answer that I think most knowledgeable SEOs will choose.

Here’s an example of a “short answer” question: 

Example of a "short answer" question in our application process

We designed this question to help us judge the applicant’s ability to explain things succinctly and accurately.

If you’re curious about our application form, here it is.

Only a handful of these questions are for vetting purposes; the rest are to get basic details like their names, email addresses, Twitter handles, etc. It’s essential to ask for these details, as it’s the only way to get them when applicants aren’t applying by email.

You now need to attract candidates to your application form, which you can do by creating a job listing and posting everywhere you can.

Here are the four things you need to explain:

  1. What you’re looking for – Keep it short. Tell applicants what the position entails.
  2. What skills you require – List everything you expect in applicants. 
  3. Who you are – Give some details about your company. (Keep it short; don’t bore people.) 
  4. How to apply – Send people to your application form. 

There are plenty of job boards where you can submit your listing. We posted to ProBlogger, Swipe Files Job Board (prev. Hey Marketers), our careers page, and a few others.

One of the job boards we posted on

We also posted in a few Slack and Facebook groups.

Given that every applicant goes through the same process, the more places you can post your job listing, the better. 

Every aspect of the vetting process takes place in the Google Sheet. This makes life super easy, as everything is in one place. I recommend splitting the process into automated and manual vetting to make things as quick and efficient as possible. 

Automated vetting

Here, the idea is to disqualify applicants who don’t fit the bill, give a preliminary score to the remaining ones, and clean up the sheet to make manual vetting as quick as possible. Here’s how to do it in three steps. 

A. Disqualify

There’s no point wasting time reviewing applications from folks who can’t follow basic rules or don’t meet basic criteria. So it pays to disqualify them automatically. The beauty of Google Sheets is that you can do this easily with filters and formulas.

You’ll recall that we asked all applicants to define two terms in under 50 words. As we wanted to disqualify those who exceeded the allowed word count, we added two additional columns with formulas to count the number of words in each definition. 

Columns calculating the word count of "short answer" responses

Then we filtered to exclude rows where the definitions were above 50 words.

Filtering out short answers that exceed the specified word count

We then filtered for a few more criteria: 

  • Rate per 1,000 words – We excluded anyone who didn’t put 10–1,000. (This was primarily to exclude applicants who couldn’t follow instructions and gave a price per word.)
  • Years of experience in SEO – We excluded anyone with “<1” year of experience.
  • Have you ever used Ahrefs before? – We excluded those who answered “no.”

B. Score

Next, you want to give each applicant a preliminary score to get a better sense of their knowledge and experience at a glance. 

To do this, we created a formula that checked their answers to our multiple-choice questions, their Ahrefs experience, and whether they still had an active Ahrefs subscription. The result was a score between 0 and 5.

Preliminary applicant score in Google Sheets

C. Simplify

Even with the filters and scores in place, your sheet will be overwhelming because it contains so much data. So it’s worth hiding columns you don’t need for the vetting process, such as applicants’ names and email addresses. This also helps to eliminate potential biases.

For us, we added conditional formatting to our preliminary score to make eyeballing the quality of each applicant easier.

Here’s what we ended up with:

Final applicant sheet

Manual vetting

We skimmed applicants’ preliminary scores and definitions to decide whether they should progress to the next stage of the process. Each applicant took no more than a few seconds, allowing us to vet over 100 applicants per hour.

For example, it only took a second to decide not to continue with this applicant because they defined SEM rather than SEO.

Example of a not-so-good definition from an applicant

For the applicant below, on the other hand, we could tell pretty quickly that it’d be worth giving them a shot. Both of their definitions were accurate and well written, and they scored well on our multiple-choice questions:

Example of a good applicant

To track our decision for each applicant, we added one final column:

Decision column for the applicants

Most people test writers by having them write a test article from a content brief. We do the same but don’t send unique briefs to each applicant. We send them all the same brief. 

Here are three reasons why we recommend this: 

  1. It’s easier to grade their work – You can create a checklist to score writers on the same criteria. 
  2. It’s quicker You don’t have to create a new content brief for each writer.
  3. It can be automated – You can set up systems to send the brief to writers when they hit this stage.

Let’s look at how to ensure tests run smoothly and efficiently. 

A. Explain the deal and get their details

For applicants who pass our initial vetting process, we need to explain the deal for the test article and get their details. We created a template in Gmail for this. 

It explains that we:

  • Want to offer them a paid test article.
  • Pay a flat fee per test article (and how much).
  • Send the same brief to every applicant and that their article won’t be published.
  • Own all the work they produce for us.
  • Pay invoices at the end of the month.
Email we sent to approved applicants

That may seem like information overload, but it’s best to make sure everyone’s on the same page from the beginning.

Our email ends with a link to a Google Form asking for their invoice email address (this is often different from the one they use daily) and how they want to be paid.

Link to the next part of the process in the email we sent to applicants

B. Send the content brief for the test

We show a confirmation message when an applicant submits the Google Form in the previous step. It links them to a Google Doc with the content brief for the test article.

Link to the test job at the end of our Google Form

Here’s what that looks like:

Our content brief for writers

The document also contains instructions that tell the applicant to:

  1. Make a copy of the document.
  2. Write as much or as little as they like (no word count quotas).
  3. Email us the test article and invoice us when they’re done.
  4. Read our content guidelines before they start writing.

I recommend everyone create content guidelines when working with freelance writers. We link to ours in the doc. It explains what we expect regarding style and content.

Here are a couple of excerpts:

Excerpt from our style guidelines
Excerpt from our style guidelines

C. Review their content

Given that you’ll be testing a few writers, you need a fair and consistent way to judge their relative quality. For this, we created a checklist to review and score their efforts quickly.

Here’s an excerpt from our checklist:

Our content checklist for vetting submissions

You can see that we check the article as a whole for a few basics, then review each section in more depth. 

For example, here are a few of our criteria for the definition:

  • Is it written in the correct format (according to our guidelines)?
  • Is it accurate?
  • Is it well written?

In total, there are 30 things on our checklist. Each one equals one point, so we score each applicant out of 30. 

Hiring is the easy part. If an applicant scores well on their test article, we assign them an actual one to write. If that goes well, we give them more.

Here are a few things to look out for when working with freelancers:

  • Quality deterioration – Freelancers can sometimes get lazy or even begin subcontracting work. Both of these things lead to a reduction in quality. 
  • Reliability issues – Life occasionally gets in the way for everyone but watch out for consistent unreliability. 
  • Communication issues – For whatever reason, disappearing off the face of the earth is surprisingly common. 

Don’t hesitate to drop writers if you face these issues. Continuing to work with them will suck your time and energy. 

By that same token, when you come across reliable freelancers who consistently produce high-quality work, treat them well. Great writers are worth their weight in gold, so keep giving them work and review their rates periodically. 

Final thoughts

Hiring freelance writers is easy when you have a system in place. You can even automate many of the steps with tools like Zapier. 

For example, we have zaps for outline requests, logging details, and sending payment requests to our office manager. 

How we automated our freelance application process with Zapier

Got questions? Ping me on Twitter.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address


Optimize Your SEO Strategy For Maximum ROI With These 5 Tips



Optimize Your SEO Strategy For Maximum ROI With These 5 Tips

Wondering what improvements can you make to boost organic search results and increase ROI?

If you want to be successful in SEO, even after large Google algorithm updates, be sure to:

  1. Keep the SEO fundamentals at the forefront of your strategy.
  2. Prioritize your SEO efforts for the most rewarding outcomes.
  3. Focus on uncovering and prioritizing commercial opportunities if you’re in ecommerce.
  4. Dive into seasonal trends and how to plan for them.
  5. Get tip 5 and all of the step-by-step how-tos by joining our upcoming webinar.

We’ll share five actionable ways you can discover the most impactful opportunities for your business and achieve maximum ROI.

You’ll learn how to:

  • Identify seasonal trends and plan for them.
  • Report on and optimize your online share of voice.
  • Maximize SERP feature opportunities, most notably Popular Products.

Join Jon Earnshaw, Chief Product Evangelist and Co-Founder of Pi Datametrics, and Sophie Moule, Head of Product and Marketing at Pi Datametrics, as they walk you through ways to drastically improve the ROI of your SEO strategy.

In this live session, we’ll uncover innovative ways you can step up your search strategy and outperform your competitors.

Ready to start maximizing your results and growing your business?

Sign up now and get the actionable insights you need for SEO success.

Can’t attend the live webinar? We’ve got you covered. Register anyway and you’ll get access to a recording, after the event.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading


TikTok’s US Future Uncertain: CEO Faces Congress



TikTok's US Future Uncertain: CEO Faces Congress

During a five-hour congressional hearing, TikTok CEO Shou Zi Chew faced intense scrutiny from U.S. lawmakers about the social media platform’s connections to its Chinese parent company, ByteDance.

Legislators from both sides demanded clear answers on whether TikTok spies on Americans for China.

The U.S. government has been pushing for the divestiture of TikTok and has even threatened to ban the app in the United States.

Chew found himself in a difficult position, attempting to portray TikTok as an independent company not influenced by China.

However, lawmakers remained skeptical, citing China’s opposition to the sale of TikTok as evidence of the country’s influence over the company.

The hearing was marked by a rare display of bipartisan unity, with the tone harsher than in previous congressional hearings featuring American social media executives.

The Future of TikTok In The US

With the U.S. and China at odds over TikTok’s sale, the app faces two possible outcomes in the United States.

Either TikTok gets banned, or it revisits negotiations for a technical fix to data security concerns.

Lindsay Gorman, head of technology and geopolitics at the German Marshall Fund, said, “The future of TikTok in the U.S. is definitely dimmer and more uncertain today than it was yesterday.”

TikTok has proposed measures to protect U.S. user data, but no security agreement has been reached.

Addressing Concerns About Societal Impact

Lawmakers at the hearing raised concerns about TikTok’s impact on young Americans, accusing the platform of invading privacy and harming mental health.

According to the Pew Research Center, the app is used by 67% of U.S. teenagers.

Critics argue that the app is too addictive and its algorithm can expose teens to dangerous or lethal situations.

Chew pointed to new screen time limits and content guidelines to address these concerns, but lawmakers remained unconvinced.

In Summary

The House Energy and Commerce Committee’s hearing on TikTok addressed concerns common to all social media platforms, like spreading harmful content and collecting massive user data.

Most committee members were critical of TikTok, but many avoided the typical grandstanding seen in high-profile hearings.

The hearing aimed to make a case for regulating social media and protecting children rather than focusing on the national security threat posed by the app’s connection to China.

If anything emerges from this hearing, it could be related to those regulations.

The hearing also allowed Congress to convince Americans that TikTok is a national security threat that warrants a ban.

This concern arises from the potential for the Chinese government to access the data of TikTok’s 150 million U.S. users or manipulate its recommendation algorithms to spread propaganda or disinformation.

However, limited public evidence supports these claims, making banning the app seem extreme and potentially unnecessary.

As events progress, staying informed is crucial as the outcome could impact the digital marketing landscape.

Featured Image: Rokas Tenys/Shutterstock

Full replay of congressional hearing available on YouTube.

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading


Everything You Need To Know



Everything You Need To Know

Google has just released Bard, its answer to ChatGPT, and users are getting to know it to see how it compares to OpenAI’s artificial intelligence-powered chatbot.

The name ‘Bard’ is purely marketing-driven, as there are no algorithms named Bard, but we do know that the chatbot is powered by LaMDA.

Here is everything we know about Bard so far and some interesting research that may offer an idea of the kind of algorithms that may power Bard.

What Is Google Bard?

Bard is an experimental Google chatbot that is powered by the LaMDA large language model.

It’s a generative AI that accepts prompts and performs text-based tasks like providing answers and summaries and creating various forms of content.

Bard also assists in exploring topics by summarizing information found on the internet and providing links for exploring websites with more information.

Why Did Google Release Bard?

Google released Bard after the wildly successful launch of OpenAI’s ChatGPT, which created the perception that Google was falling behind technologically.

ChatGPT was perceived as a revolutionary technology with the potential to disrupt the search industry and shift the balance of power away from Google search and the lucrative search advertising business.

On December 21, 2022, three weeks after the launch of ChatGPT, the New York Times reported that Google had declared a “code red” to quickly define its response to the threat posed to its business model.

Forty-seven days after the code red strategy adjustment, Google announced the launch of Bard on February 6, 2023.

What Was The Issue With Google Bard?

The announcement of Bard was a stunning failure because the demo that was meant to showcase Google’s chatbot AI contained a factual error.

The inaccuracy of Google’s AI turned what was meant to be a triumphant return to form into a humbling pie in the face.

Google’s shares subsequently lost a hundred billion dollars in market value in a single day, reflecting a loss of confidence in Google’s ability to navigate the looming era of AI.

How Does Google Bard Work?

Bard is powered by a “lightweight” version of LaMDA.

LaMDA is a large language model that is trained on datasets consisting of public dialogue and web data.

There are two important factors related to the training described in the associated research paper, which you can download as a PDF here: LaMDA: Language Models for Dialog Applications (read the abstract here).

  • A. Safety: The model achieves a level of safety by tuning it with data that was annotated by crowd workers.
  • B. Groundedness: LaMDA grounds itself factually with external knowledge sources (through information retrieval, which is search).

The LaMDA research paper states:

“…factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator.

We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible.”

Google used three metrics to evaluate the LaMDA outputs:

  1. Sensibleness: A measurement of whether an answer makes sense or not.
  2. Specificity: Measures if the answer is the opposite of generic/vague or contextually specific.
  3. Interestingness: This metric measures if LaMDA’s answers are insightful or inspire curiosity.

All three metrics were judged by crowdsourced raters, and that data was fed back into the machine to keep improving it.

The LaMDA research paper concludes by stating that crowdsourced reviews and the system’s ability to fact-check with a search engine were useful techniques.

Google’s researchers wrote:

“We find that crowd-annotated data is an effective tool for driving significant additional gains.

We also find that calling external APIs (such as an information retrieval system) offers a path towards significantly improving groundedness, which we define as the extent to which a generated response contains claims that can be referenced and checked against a known source.”

How Is Google Planning To Use Bard In Search?

The future of Bard is currently envisioned as a feature in search.

Google’s announcement in February was insufficiently specific on how Bard would be implemented.

The key details were buried in a single paragraph close to the end of the blog announcement of Bard, where it was described as an AI feature in search.

That lack of clarity fueled the perception that Bard would be integrated into search, which was never the case.

Google’s February 2023 announcement of Bard states that Google will at some point integrate AI features into search:

“Soon, you’ll see AI-powered features in Search that distill complex information and multiple perspectives into easy-to-digest formats, so you can quickly understand the big picture and learn more from the web: whether that’s seeking out additional perspectives, like blogs from people who play both piano and guitar, or going deeper on a related topic, like steps to get started as a beginner.

These new AI features will begin rolling out on Google Search soon.”

It’s clear that Bard is not search. Rather, it is intended to be a feature in search and not a replacement for search.

What Is A Search Feature?

A feature is something like Google’s Knowledge Panel, which provides knowledge information about notable people, places, and things.

Google’s “How Search Works” webpage about features explains:

“Google’s search features ensure that you get the right information at the right time in the format that’s most useful to your query.

Sometimes it’s a webpage, and sometimes it’s real-world information like a map or inventory at a local store.”

In an internal meeting at Google (reported by CNBC), employees questioned the use of Bard in search.

One employee pointed out that large language models like ChatGPT and Bard are not fact-based sources of information.

The Google employee asked:

“Why do we think the big first application should be search, which at its heart is about finding true information?”

Jack Krawczyk, the product lead for Google Bard, answered:

“I just want to be very clear: Bard is not search.”

At the same internal event, Google’s Vice President of Engineering for Search, Elizabeth Reid, reiterated that Bard is not search.

She said:

“Bard is really separate from search…”

What we can confidently conclude is that Bard is not a new iteration of Google search. It is a feature.

Bard Is An Interactive Method For Exploring Topics

Google’s announcement of Bard was fairly explicit that Bard is not search. This means that, while search surfaces links to answers, Bard helps users investigate knowledge.

The announcement explains:

“When people think of Google, they often think of turning to us for quick factual answers, like ‘how many keys does a piano have?’

But increasingly, people are turning to Google for deeper insights and understanding – like, ‘is the piano or guitar easier to learn, and how much practice does each need?’

Learning about a topic like this can take a lot of effort to figure out what you really need to know, and people often want to explore a diverse range of opinions or perspectives.”

It may be helpful to think of Bard as an interactive method for accessing knowledge about topics.

Bard Samples Web Information

The problem with large language models is that they mimic answers, which can lead to factual errors.

The researchers who created LaMDA state that approaches like increasing the size of the model can help it gain more factual information.

But they noted that this approach fails in areas where facts are constantly changing during the course of time, which researchers refer to as the “temporal generalization problem.”

Freshness in the sense of timely information cannot be trained with a static language model.

The solution that LaMDA pursued was to query information retrieval systems. An information retrieval system is a search engine, so LaMDA checks search results.

This feature from LaMDA appears to be a feature of Bard.

The Google Bard announcement explains:

“Bard seeks to combine the breadth of the world’s knowledge with the power, intelligence, and creativity of our large language models.

It draws on information from the web to provide fresh, high-quality responses.”

Screenshot of a Google Bard Chat, March 2023

LaMDA and (possibly by extension) Bard achieve this with what is called the toolset (TS).

The toolset is explained in the LaMDA researcher paper:

“We create a toolset (TS) that includes an information retrieval system, a calculator, and a translator.

TS takes a single string as input and outputs a list of one or more strings. Each tool in TS expects a string and returns a list of strings.

For example, the calculator takes “135+7721”, and outputs a list containing [“7856”]. Similarly, the translator can take “hello in French” and output [‘Bonjour’].

Finally, the information retrieval system can take ‘How old is Rafael Nadal?’, and output [‘Rafael Nadal / Age / 35’].

The information retrieval system is also capable of returning snippets of content from the open web, with their corresponding URLs.

The TS tries an input string on all of its tools, and produces a final output list of strings by concatenating the output lists from every tool in the following order: calculator, translator, and information retrieval system.

A tool will return an empty list of results if it can’t parse the input (e.g., the calculator cannot parse ‘How old is Rafael Nadal?’), and therefore does not contribute to the final output list.”

Here’s a Bard response with a snippet from the open web:

Google Bard: Everything You Need To KnowScreenshot of a Google Bard Chat, March 2023

Conversational Question-Answering Systems

There are no research papers that mention the name “Bard.”

However, there is quite a bit of recent research related to AI, including by scientists associated with LaMDA, that may have an impact on Bard.

The following doesn’t claim that Google is using these algorithms. We can’t say for certain that any of these technologies are used in Bard.

The value in knowing about these research papers is in knowing what is possible.

The following are algorithms relevant to AI-based question-answering systems.

One of the authors of LaMDA worked on a project that’s about creating training data for a conversational information retrieval system.

You can download the 2022 research paper as a PDF here: Dialog Inpainting: Turning Documents into Dialogs (and read the abstract here).

The problem with training a system like Bard is that question-and-answer datasets (like datasets comprised of questions and answers found on Reddit) are limited to how people on Reddit behave.

It doesn’t encompass how people outside of that environment behave and the kinds of questions they would ask, and what the correct answers to those questions would be.

The researchers explored creating a system read webpages, then used a “dialog inpainter” to predict what questions would be answered by any given passage within what the machine was reading.

A passage in a trustworthy Wikipedia webpage that says, “The sky is blue,” could be turned into the question, “What color is the sky?”

The researchers created their own dataset of questions and answers using Wikipedia and other webpages. They called the datasets WikiDialog and WebDialog.

  • WikiDialog is a set of questions and answers derived from Wikipedia data.
  • WebDialog is a dataset derived from webpage dialog on the internet.

These new datasets are 1,000 times larger than existing datasets. The importance of that is it gives conversational language models an opportunity to learn more.

The researchers reported that this new dataset helped to improve conversational question-answering systems by over 40%.

The research paper describes the success of this approach:

“Importantly, we find that our inpainted datasets are powerful sources of training data for ConvQA systems…

When used to pre-train standard retriever and reranker architectures, they advance state-of-the-art across three different ConvQA retrieval benchmarks (QRECC, OR-QUAC, TREC-CAST), delivering up to 40% relative gains on standard evaluation metrics…

Remarkably, we find that just pre-training on WikiDialog enables strong zero-shot retrieval performance—up to 95% of a finetuned retriever’s performance—without using any in-domain ConvQA data. “

Is it possible that Google Bard was trained using the WikiDialog and WebDialog datasets?

It’s difficult to imagine a scenario where Google would pass on training a conversational AI on a dataset that is over 1,000 times larger.

But we don’t know for certain because Google doesn’t often comment on its underlying technologies in detail, except on rare occasions like for Bard or LaMDA.

Large Language Models That Link To Sources

Google recently published an interesting research paper about a way to make large language models cite the sources for their information. The initial version of the paper was published in December 2022, and the second version was updated in February 2023.

This technology is referred to as experimental as of December 2022.

You can download the PDF of the paper here: Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models (read the Google abstract here).

The research paper states the intent of the technology:

“Large language models (LLMs) have shown impressive results while requiring little or no direct supervision.

Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios.

We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting.

We formulate and study Attributed QA as a key first step in the development of attributed LLMs.

We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures.

We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development.

Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?).”

This kind of large language model can train a system that can answer with supporting documentation that, theoretically, assures that the response is based on something.

The research paper explains:

“To explore these questions, we propose Attributed Question Answering (QA). In our formulation, the input to the model/system is a question, and the output is an (answer, attribution) pair where answer is an answer string, and attribution is a pointer into a fixed corpus, e.g., of paragraphs.

The returned attribution should give supporting evidence for the answer.”

This technology is specifically for question-answering tasks.

The goal is to create better answers – something that Google would understandably want for Bard.

  • Attribution allows users and developers to assess the “trustworthiness and nuance” of the answers.
  • Attribution allows developers to quickly review the quality of the answers since the sources are provided.

One interesting note is a new technology called AutoAIS that strongly correlates with human raters.

In other words, this technology can automate the work of human raters and scale the process of rating the answers given by a large language model (like Bard).

The researchers share:

“We consider human rating to be the gold standard for system evaluation, but find that AutoAIS correlates well with human judgment at the system level, offering promise as a development metric where human rating is infeasible, or even as a noisy training signal. “

This technology is experimental; it’s probably not in use. But it does show one of the directions that Google is exploring for producing trustworthy answers.

Research Paper On Editing Responses For Factuality

Lastly, there’s a remarkable technology developed at Cornell University (also dating from the end of 2022) that explores a different way to source attribution for what a large language model outputs and can even edit an answer to correct itself.

Cornell University (like Stanford University) licenses technology related to search and other areas, earning millions of dollars per year.

It’s good to keep up with university research because it shows what is possible and what is cutting-edge.

You can download a PDF of the paper here: RARR: Researching and Revising What Language Models Say, Using Language Models (and read the abstract here).

The abstract explains the technology:

“Language models (LMs) now excel at many tasks such as few-shot learning, question answering, reasoning, and dialog.

However, they sometimes generate unsupported or misleading content.

A user cannot easily determine whether their outputs are trustworthy or not, because most LMs do not have any built-in mechanism for attribution to external evidence.

To enable attribution while still preserving all the powerful advantages of recent generation models, we propose RARR (Retrofit Attribution using Research and Revision), a system that 1) automatically finds attribution for the output of any text generation model and 2) post-edits the output to fix unsupported content while preserving the original output as much as possible.

…we find that RARR significantly improves attribution while otherwise preserving the original input to a much greater degree than previously explored edit models.

Furthermore, the implementation of RARR requires only a handful of training examples, a large language model, and standard web search.”

How Do I Get Access To Google Bard?

Google is currently accepting new users to test Bard, which is currently labeled as experimental. Google is rolling out access for Bard here.

Google Bard is ExperimentalScreenshot from, March 2023

Google is on the record saying that Bard is not search, which should reassure those who feel anxiety about the dawn of AI.

We are at a turning point that is unlike any we’ve seen in, perhaps, a decade.

Understanding Bard is helpful to anyone who publishes on the web or practices SEO because it’s helpful to know the limits of what is possible and the future of what can be achieved.

More Resources:

Featured Image: Whyredphotographor/Shutterstock

Source link

Keep an eye on what we are doing
Be the first to get latest updates and exclusive content straight to your email inbox.
We promise not to spam you. You can unsubscribe at any time.
Invalid email address
Continue Reading