AI vs Humans: The battle of content detection accuracy
The release of Open AI's ChatGPT has rocked the world. AI’s impressive content generation skills have potentially revolutionised the way to create content. We know that AI can produce quality content, but is it possible to detect AI-generated content? There are various AI detection tools on the market, but how accurate are they? We put them to the test.
How does Katana use AI for content generation?
At Katana, we use AI tools to enhance our internal processes. AI supercharges our team's productivity and streamlines our work flows. We know AI is here to stay. That's why we embrace the use of AI. The better we become at using the AI tools, the better our outcomes.
An AI tool, however, cannot produce the required results without human intervention. AI tools wouldn’t be much use without its’ users. And, using an AI tool to the proper effect, requires experience and judgement.
Our experience enables our team to make the call on whether the AI tool has got it right…or got it wrong. If you don’t know what to ask, how do you know the answer is right?
Simultaneously, we know original content is a priority to enable exceptional SEO performance. We don’t use AI to completely generate content, unlike others. Instead, we use AI for things like:
- Ideation: Creating ideas for blog posts or advertisements can be difficult and time-consuming. We use AI tools to help us generate topics and ideas.
- Analysing tone and style: Analysing tone and style can be contextually difficult. When we need to analyse a paragraph, or change a sentence from passive voice to active voice, AI is our friendliest editor, who we don't have to buy wine for.
- Summarising: Sometimes we’re a little busy to read that 2000 word article a team member shared. Two clicks, and we've got a short summary thanks to AI.
- Translation: Sometimes we need to read text that is in a different language. AI helps us do this instantly.
Katana sets the record straight.
We recently tested a range of AI detection tools. We weren’t very impressed by the results. At least, not yet.
We tested four of the most popular AI detection tools on the market, including:
We first tested 5 pieces of human written content we created for one of our clients. These 500-word content pieces were 100% human written and generated by our content team, with absolutely no AI help.
Here are the results:
Our results show AI detectors to be mostly accurate in detecting human-generated content. That said, Originality does exhibit some anomalies.
Using the same 5 topics as our original content, we asked ChatGPT to generate 500 word blog posts for each topic. After pasting this AI-generated content into the AI detectors, the results were disappointing. There is definitely room for improvement. Only 1 out of 4 tools was accurate at detecting AI content.
This data proves most AI detection tools fail to accurately detect AI-generated content. Originality seems to be the most accurate of all the tools, with an average accuracy of 99.2%.
AI detection tools claim to be top-notch at detecting AI-generated content. Our research reveals that the opposite is true.
The final verdict.
Current AI content detection tools fail to, accurately and consistently, detect AI-generated content. While this may be the case for now, we expect this to change. Over time, these tools should become more accurate.
If you think you’re going to outsmart search engines with your AI-generated content, you’re wrong. Eventually, search engines will catch on. While you may see your content start to rank initially, this will not last. Furthermore, AI is not completely accurate. People who blindly accept the content AI creates could make mistakes that they may regret. Make sure your agency isn’t abusing your trust by using AI to create content. Your rank could be at risk.
Some search engines may not be able to detect AI-generated content right now. Google has built its own AI called Bard. If you think Google can’t detect AI-generated content, you are mistaken.
Imagine all the content on your website is AI-generated. How will this affect you when search engines start penalising AI-generated content? Say goodbye to your domain rating and your organic traffic.
You need to rely on your own creativity and expertise. Use AI as a helpful tool rather than a replacement. Don’t build your content strategy based on AI-generated content. Instead, use AI to assist you.
Armed with your own creativity, and assistance from AI, it's time to shoot, and score, big!
Direct access to founders
At Katana, there are no corporate bureaucracies, red tape cartwheels, or customer support horror stories. You already deal with enough faceless companies. Our clients talk directly to our founders, Kerry and Mark.
