Can ChatGPT Create Accurate Documents for How-To Guides?

James Kenny

Head of Programmatic and Paid Social


Industry News

One of the main questions people ask about generative AI is whether you can trust the tool when trying to build a how-to guide or complete an unfamiliar process successfully.

Due to its ubiquitous presence in the news for over a year now, many people use ChatGPT – but according to a 2023 Statista study about use cases for generative AI, most use for fun instead of professional reasons. Our own Creative team previously tested ChatGPT to see how it fared in creating content, delivering eye-opening results.

This may be the best (and possibly only) way to find success with ChatGPT currently. However, our Programmatic team thought differently.

To test their theory, the team gave ChatGPT a more practical task: creating a document that helps organizations set up a business account on Facebook. And, much like with our Creative team’s experiment, our Programmatic specialists saw results that were reassuring yet surprising.

Why did the team decide to use ChatGPT?

 

The speed at which ChatGPT can create content

 

Although humans can type fast, ChatGPT is faster and, for the most part, can create content much quicker than humans. If a client urgently needs a document (a situation our Programmatic team is very familiar with), ChatGPT is a better option because of all the time it can save.

 

ChatGPT has the best reputation

 

Since ChatGPT launched in November 2022, other generative AI tools have appeared, including Google Bard and Bing Chat by Microsoft. Yet, according to a poll of over 2,000 individuals in the US, ChatGPT is still the most trusted generative AI tool, which was the reason the Programmatic team decided to play around with it. 

 

 

What was the prompt our programmatic team gave ChatGPT?

 

As knowledge increases about achieving the most effective results from generative AI, tools like ChatGPT are only as good as the information you enter into them. One advantage of our ChatGPT test is that our Programmatic team knows how to create a Facebook business manager account inside and out. Our specialists created a detailed prompt to ensure ChatGPT interpreted the information correctly, and our team was very capable of spotting any omissions.

The prompt said:

Hello. Can you please write me a guide based on the following requirements: 

Background: I am working for an agency for a new client. The client needs to setup a new Facebook Business Manager, A New Ad Account, A New Pixel and then to share access with us. 

Requirements: This needs to be client-facing, so from the point of view of the client. It’s unclear what technical capacity they have. The client did have an old Business Manager, but the old Admin created it and left, so no one has experience in setting something, but we do need to build security and future safeguarding into consideration (to ensure that they don’t lose access if the current Admin leaves)

Did ChatGPT describe the process correctly?

 

As expected, ChatGPT created a fairly sucessful process document for how to set up a Facebook business account. ChatGPT formatted the document into simple steps, matching the prompt, which asked for an easily digestible template for new and experienced users. According to our team, they estimated it would take about 30 minutes to create the document without ChatGPT. With ChatGPT, the process only took 10 minutes, confirming the time efficiency of the generative AI tool.

While ChatGPT created the document quickly, the team checked the steps to ensure that ChatGPT listed factually accurate information. As the team is very familiar with creating a Facebook business account, fact-checking only took a few minutes, still much faster than creating a brand-new document, and everything generally looked correct.

However, if this were an unfamiliar process, problems may still arise without extensive background research. We advise your team to have an in-depth understanding of what you are prompting of ChatGPT, or at least a serviceable amount of time to review the document to check for any incorrect information.

 

 

 

Search Laboratory | Digital Marketing Agency

ChatGPT is a brilliant tool for helping businesses become more time-efficient, especially in reactive industries such as digital marketing. As our Programmatic team found out, along with 31% of technology professionals worldwide, ChatGPT can create documents for practical use, but there still needs to be a strong focus on human oversight to ensure the outputs are accurate, effective, and successful.

Interested to learn more about the future of AI and digital marketing?Download our whitepaper


"