A ChatGPT Content Marketing Experiment With Surprising Results

I’m excited. I see ChatGPT as a real-life learning tool to help humans see something we usually talk about theoretically – the potential of artificial intelligence.

Sure, AI content creation tools have been around for a while, but they aren’t as ubiquitous or as free as ChatGPT. Now that an AI text-creation tool fits my budget, I decided to conduct a few experiments to better understand AI’s potential for creating content.

At first, I wanted to prove real-life writers are better than a bot. (I have a vested interest in the matter.)

Are human writers better than bot writers?

I wondered what ChatGPT thought of that idea. So I asked.

Prompt: Are human writers better than bot writers

Result (paragraph breaks added for easier reading): Human writers and AI-powered writing bots have different strengths and weaknesses. Human writers have the ability to understand and convey emotions, personal experiences, and cultural nuances that may be difficult for AI models to replicate. They also have the ability to think creatively and generate new ideas.

On the other hand, AI-powered writing bots can produce large amounts of content quickly and can be programmed to use specific language and tone for different audiences. They can also be used to analyze data and generate reports. 

In summary, both human writers and bot writers have their own unique advantages and can be used effectively in different contexts. However, for a content that requires emotional touch, creativity and cultural understanding, human writers are considered more suitable.

The image below shows what the prompt and result looked like in the interface.

My conclusion: I like ChatGPT more than I thought I would. The bot appreciates human writers like me. On a more serious note, the text answers the question intelligently, succinctly explaining the strengths of both perspectives. It also draws a well-stated conclusion. Frankly, that result is better than a lot of human-created writing I review and edit.

Is ChatGPT copying from my (and others’) content?

Now that I had a warm feeling about ChatGPT, I wanted to learn more about it and see how we might work together. So I took it for a spin, starting with a standard topic at the Content Marketing Institute:

Prompt: Explain content marketing

Result: Content marketing is a strategic approach to creating and distributing valuable, relevant, and consistent content to attract and engage a clearly defined audience with the goal of driving profitable customer action. 

It involves creating and distributing various forms of content such as blog posts, videos, infographics, and social media posts that provide value to the target audience, rather than solely promoting a product or service. 

The goal of content marketing is to establish trust and credibility with the target audience, ultimately leading to increased brand awareness and customer loyalty.

My first conclusion: OK, ChatGPT’s shine faded a bit. Yes, the text explains content marketing well. It not only defines it but explains the forms and goals of content marketing. But the first paragraph seemed familiar. A visit to the CMI page defining content marketing made me realize why. That page features this definition:

Content marketing is a strategic marketing approach focused on creating and distributing valuable, relevant, and consistent content to attract and retain a clearly defined audience — and, ultimately, to drive profitable customer action.

ChatGPT reworded CMI’s definition about as well as a high schooler whose clumsy attempt to “write” an essay earned an F for plagiarism. The use of specific phrases, “clearly defined” and “profitable customer action,” tells me it copied from CMI.

That’s where a human editor comes into play. Always run the text (human- or AI-created) through a plagiarism checker (another AI tool) before publishing, and always incorporate proper attribution.

But wait, I thought, maybe the attribution problem was my fault. Maybe my prompt was poorly written. What if I asked for attribution?

Follow-up prompt: Who says?

Result: Content Marketing Institute, a leading website and organization in the field of content marketing, defines content marketing as “a strategic marketing approach focused on creating and distributing valuable, relevant, and consistent content to attract and engage a clearly defined and understood target audience – with the objective of driving profitable customer action.” This definition is widely accepted in the industry and is often cited as a standard explanation of the concept.

My second conclusion: Attribution is possible with ChatGPT. You just have to ask for it in either your initial or follow-up prompt. I used an attribution prompt for another scenario, and it listed a handful of experts by name, title, and a brief mention of what they said on the topic.

(Interestingly, I ran this article through the plagiarism checker in Grammarly (another AI-powered tool), but it failed to cite CMI for the content marketing definition. Instead, Grammarly cited pieces and parts of the definition as coming from other sites that seemingly relied on CMI’s definition. So, recognize that even plagiarism checkers aren’t a 100% guarantee that the content is original or cited correctly.)

Can ChatGPT explain topics better than Google?

In researching topics, I often input them into Google search to better understand them. Then, I wade through the results, clicking a link and hoping it takes me to a helpful article or site. I wanted to see if ChatGPT could help cut down my search engine time.

Prompt: What should content marketers do about Web3?

Google result: The top six results my search returned are articles about Web3 and content marketing. Google bolded some words in the brief explanatory descriptions, seemingly to highlight attributes so the searcher doesn’t have to read the article. 

These are the article titles (shown in the image below) followed by the words Google bolded:

• How Content Marketing Fits Into Web 3.0 (may bring immersive participation and intelligent peer-to-peer transactions to the customer experience approach)
• Web3 Marketing: What You Need To Know – Zen Media (will emphasize content marketing)
• Web 3.0: How the Decentralized Web Will Impact Digital … (Bring new challenges and opportunities to marketing)
• Web3 marketing: What marketers need to know! – DMEXCO (allow users to comprehensively manage their own data and content)
• How will Web 3 Shake Up the Content Marketing Landscape (consumers embracing cryptocurrencies)
• Web 3.0 is changing content marketing, here’s a primer (blockchain technology will provide Web 3.0 with data security and data decentralization)

ChatGPT result: Content marketers should familiarize themselves with Web3 technologies such as blockchain and decentralized platforms, and consider how these technologies could potentially impact their industry.

They should also keep an eye on how their target audience is engaging with these new technologies, and adjust their strategies accordingly.

Additionally, they should consider experimenting with creating content that is native to decentralized platforms, such as using non-fungible tokens (NFTs) to monetize digital assets. 

My conclusion: ChatGPT delivered a better surface-level answer for quickly understanding the topic. The phrases Google bolded on its search results page didn’t answer my question as well as the ChatGPT paragraphs. But, if I wanted a more in-depth understanding of the topic, I’d want to click to read the articles in the search result links rather than trying to get ChatGPT to say more. 

And that brings me to my next experiment.

Can ChatGPT write long-form content? 

All the answers to the dozen or so prompts I entered returned less than 100 words. Certainly, ChatGPT can’t work well for long-form content, right? Let’s find out.

Prompt: How does TikTok indirectly or directly affect content marketing? 

Result: TikTok has had a significant impact on content marketing by providing a new platform for brands and businesses to reach and engage with their target audience. The app’s short-form video format and focus on creative content make it well-suited for showcasing products, services, and behind-the-scenes looks at a company. 

Additionally, TikTok’s algorithm promotes content that is engaging and well-received by users, which can lead to organic reach and virality for brands that create popular content. Furthermore, the ability to use hashtags, challenges, and trends within the TikTok platform has given opportunities for marketers to create viral campaigns and increase brand awareness. Overall, TikTok has opened up a new avenue for brands to connect with their audiences and create compelling, shareable content.

Follow-up prompt: Tell me more.

Follow-up result: Sure, TikTok’s algorithm, which uses machine learning to determine which content users will find most engaging, is designed to promote popular and well-received content. This means that brands that create high-quality, creative content that resonates with TikTok users are more likely to have their content seen by a wider audience. Additionally, TikTok’s short-form video format allows brands to quickly and easily convey their message, whether it be promoting a product, showcasing a service, or giving a behind-the-scenes look at their company.

TikTok’s use of challenges, hashtags, and trends, are also ways for brands to create viral campaigns and increase brand awareness. For example, a brand can create a challenge using a specific hashtag and encourage users to participate, which can lead to increased engagement and visibility for the brand.

Furthermore, TikTok’s demographic is mostly Generation Z and Millennials, which makes it an excellent platform to reach out to younger audiences. TikTok’s user engagement also tends to be higher than other platforms, meaning that brands are more likely to see a return on investment when advertising on the app.

In summary, TikTok’s short-form video format, popular algorithm, and use of challenges, hashtags and trends make it an effective platform for brands to reach and engage with their target audience, create viral campaigns and increase brand awareness, and reach out to younger demographics.

My conclusion: ChatGPT isn’t a substitute for long-form content. Even when I directly asked for 1,000 words on the topic, ChatGPT only returned 350. However, I see ChatGPT’s potential for creating TL;DR content, social media posts, and alt-text descriptions for images (to a degree that surprised me).

Should content marketers use ChatGPT?

I entered that exact question as my final prompt. Turns out, the AI response works fairly well as the conclusion for this piece:

Content marketers can certainly use ChatGPT to help with various tasks such as content creation, editing, and generating ideas. For example, ChatGPT can be used to generate blog post titles, summaries, and even entire articles. Additionally, it can be used to help come up with new ideas for content marketing campaigns and to identify trends and topics that would be of interest to their target audience. However, it should be kept in mind that while ChatGPT can be a useful tool, it is not a replacement for human creativity and judgement.

But that ending also reveals why you shouldn’t hit the publish button right away. Grammarly found ChatGPT spelled judgment incorrectly in the last sentence and didn’t include a comma before “such as” in the first sentence.

A human editor would clean up the clunkier language to make the paragraph easier to read and adjust for brand tone and voice. I’d edit the paragraph to read like this:

ChatGPT can help content marketers with content creation, editing, and generating ideas. You can use ChatGPT to generate blog post titles, summaries, and even entire articles. You also can experiment with using it to come up with new ideas for content marketing campaigns and to identify trends and topics of interest to your audience. But keep this in mind: ChatGPT can be a useful tool, but it’s no replacement for human creativity and judgment.

Editors – human and AI – will always be needed, but ChatGPT and other AI text-creation programs have a place in the toolbox, too.

Cover image by Joseph Kalinowski/Content Marketing Institute