22 min
0
04.08.2024
Artificial Intelligence has advanced rapidly over the last few years, especially since ChatGPT entered the scene in November 2022. AI writing software has also emerged as a useful tool to help you stay competitive.
However, such tools are notoriously unreliable, especially in academia.
For this reason, schools must ensure you’re not presenting raw content made with AI writing tools for accuracy and ethical reasons. It’s unimaginable the amount of trust that would be lost if people were allowed to write about serious topics without fact-checking, giving generic AI-generated legal advice, or publishing journal articles that haven’t been peer-reviewed.
In this article, we’ll discuss how to check if something is written by AI and tools you can use depending on your interest.
What is AI Writing?
AI writing involves the use of AI tools to come up with written content. All you have to do is submit a prompt, sit back, and let it generate an essay, write a short story, draft a screenplay, edit your content for syntax and grammar errors, or whatever else you may require.
The limits are, frankly, only set by the particular software you’re using. AI writing tools collect information based on what others say in response to a prompt similar to yours, search the internet for information about your topic, and then compile it into a coherent response.
Until recently, these responses used to come back as robotic and clunky, but through better programming and algorithms, AI writers can now generate human-like responses.
AI Detection Explained
AI Detection
The term is used to describe the process of finding whether a piece of text was generated by an artificial intelligence (AI) system or a human through detectors.
AI detectors are often based on language models that resemble those used in the AI writing tools they’re designed to detect.
To put it plainly, the detector looks at your input and asks, "Does this look like the kind of thing I would write?" If the answer turns out to be "yes," then the conclusion becomes that the text is likely AI-generated.
Here’s how to tell if something is written by AI using AI detectors. So, the models look for two specific variables in a text: perplexity and burstiness. The lower their occurrence, the higher the probability that the text is going to be AI detectable.
Perplexity measures a text’s unpredictability or how likely it is to perplex (confuse) a reader, meaning the text is read unnaturally or makes no sense. This is a good attribute to look for because AI language models aim to produce low-perplexity texts that read smoothly and make more sense but also make them more predictable.
Imagine you're reading a book and trying to guess the next word in a sentence. If the language and context are familiar, you can often make accurate predictions. Similarly, a language model trained on vast amounts of data uses perplexity to assess how accurately it can predict or understand the next word or piece of data.
On the other hand, human writing usually has higher perplexity and more creativity, but it also contains more typos. Therefore, low perplexity is regarded as evidence of AI written content.
Burstiness measures the variation in the length and structure of a sentence. It is similar to perplexity but focuses on sentences instead of words. A sentence has low burstiness with little variation in sentence length and structure, while a text with a greater variation has high burstiness. Thus, low burstiness is an indicator of AI-generated text.
How to Detect AI-Written Content
AI is becoming more powerful every day. It is becoming increasingly difficult to identify the kind of material presented to us.
Do you want to know how to tell if writing is AI-generated? The most obvious way of checking for content originality is through AI software tools. However, when it comes to AI, you can rely on more than just tools; look for what we call AI indicators features. For each of the indicators we have provided an illustrating example and explanation so it would be easier for you to assess the writings in the future.
Tone and Style
An effective marker when learning how to tell if AI wrote something is looking for a tone and style that either feel impersonal and detached or change in a moment out of nowhere. Human writers naturally inject their work with emotion and unique perspectives, while AI presents its work sterilely.
Readers may pick up on a mismatch between the tone of voice and the subject matter being discussed, indicating ai-generated work.
| Example | Explanation |
| The rapid advancement of artificial intelligence has been hailed as a groundbreaking development, poised to revolutionize virtually every facet of modern society, from healthcare to transportation and beyond. Heck yeah, AI is the bomb diggity and is gonna make everything super dope and epic! | The casual and slang language used in the second sentence is inappropriate for the formal, academic tone established in the first sentence. |
Accuracy
AI written work can appear to be very well done. However, you should always double-check the accuracy of the information provided. Find out, is it AI? Because AI writers have been known to provide straight-up nonexistent facts occasionally.
This shortcoming comes up because most bots are trained on very limited data sets (source, form, or time) and may lack access to the most complete and latest information.
| Example | Explanation |
| The Great Pyramids of Giza, one of the Seven Wonders of the Ancient World, were constructed by ancient aliens using advanced technology and levitation techniques. These massive structures, built over 4,500 years ago with extreme precision and massive stone blocks weighing several tons each, could not have been erected by humans with the primitive tools available at the time. According to recent archaeological evidence, the pyramids were actually power plants used by extraterrestrials to provide energy for their spacecraft and settlements on Earth. | This paragraph contains multiple inaccurate claims about the construction and purpose of the Great Pyramids of Giza. The scientifically accepted explanation is that they were built as monumental tombs for ancient Egyptian pharaohs by skilled human workers, not by aliens with advanced technology. The purported "archaeological evidence" cited is also fictitious, highlighting the potential for AI systems to generate and propagate misinformation if not properly trained on reliable sources. |
Lack of Personal Touch
This is how to detect AI-generated text: AI writers lack a personality, so they often don’t have a personal touch and may sound emotionless and robotic.
Unlike a real person, an AI writing tool cannot have real conversations with experts in the subject matter they’re writing about.
This means that they can’t engage in interesting stories, share relatable opinions, or gain a deeper understanding, meaning that they cannot provide any examples from real-life examples.
| Example | Explanation |
| In recent years, the healthcare industry has embraced the use of AI-powered diagnostic tools and virtual assistants to streamline patient care. While these technological advancements have undoubtedly improved efficiency, some patients may feel a sense of detachment or lack of personal connection when interacting with these impersonal systems. |
The paragraph lacks a personal, empathetic tone when discussing the potential drawbacks of AI in healthcare, which could benefit from a more patient-centric perspective. |
Repetitive Language
Sometimes, AI writers can glitch and end up using the same phrases over and over again.
So, any time you wonder, "Was this AI-generated?" look at how it approaches the answer. The repetitiveness may come from a specific prompt you provide that the AI then repeats word for word, or it could come from repetitive or limited training data.
Additionally, AI bots are generally designed to be neutral and cautious, which may appear repetitive because they rely heavily on more conservative language patterns.
| Example | Explanation |
| The Internet of Things (IoT) has emerged as a revolutionary paradigm, enabling seamless connectivity between devices and the internet. The IoT has the potential to transform various industries by facilitating the exchange of data between devices, devices, and devices, allowing for enhanced efficiency, improved decision-making, and a more interconnected world. |
The repetitive use of the word "devices" three times in a row makes the paragraph feel redundant and unnatural. |
Sources Authenticity
One of the easiest ways to determine whether a text is AI-written or not is by verifying the sources provided. AIs often make up their own sources or provide resources that do not exist. Human writers can meticulously cite credible references to support their arguments, while AI may reference unverifiable sources, articles, or studies.
AI’s inability to provide genuine sources affects its credibility, as it reveals that the AI may have been trained on fictional data. If you want to check AI-generated content, scrutinize the validity of the cited sources.
| Example | Explanation |
| According to a groundbreaking study published in the esteemed Journal of Pseudoscientific Nonsense, consuming a daily dose of homeopathic crystal energy supplements has been shown to significantly reduce the risk of contracting various forms of psychic vampirism. |
The source cited, "Journal of Pseudoscientific Nonsense," is clearly not a reputable or authentic source, undermining the credibility of the claim made in the paragraph. |
Punctuation
AI bots often adhere to strict punctuation rules that lack human writers' fluidity and nuance characteristics. Humans use punctuation to convey emphasis, rhythm, and meaning through dashes, ellipses, and commas that guide readers.
On the other hand, AI uses punctuation in a mechanical manner that follows stricter rules that have no regard for the natural flow of language. It’s relatively simple to detect artificial content by looking for robotic quality in the punctuation of the content in front of you.
| Example | Explanation |
| The advent of 5G technology promises to usher in a new era of digital connectivity with unprecedented speeds and reduced latency this will enable a wide range of innovative applications from autonomous vehicles to remote surgery. |
The lack of a period or other punctuation between the two sentences makes the paragraph difficult to read and understand. |
Keywords Overuse
While trying to maintain relevance to the topic provided, AI may overuse certain keywords or phrases, resulting in a formulaic style.
Humans normally vary their sentence structure and vocabulary to keep their readers interested, avoiding the monotony characteristic of AI-generated content.
AI algorithms use keyword repetition as a strategy for signaling relevance, and you can check for excessive keywords use to see if something was written by AI.
| Example | Explanation |
| Blockchain technology has emerged as a disruptive force, disrupting traditional business models across industries. This disruptive innovation leverages the power of decentralized, distributed ledgers to disrupt established processes, enabling secure, transparent, and tamper-proof transactions that disrupt the status quo. |
The excessive use of the word "disrupt" and its variations makes the paragraph feel forced and unnatural, detracting from the overall message. |
Contextual Clues
Human communication is rich with nuance that relies on cultural references, humor, and shared experiences to convey meaning. In contrast, AI algorithms may struggle to grasp subtle contextual cues that human writers effortlessly incorporate into their work; therefore, detecting AI generated text becomes easier.
AI-generated work lacks the sophistication and depth to interpret subtle contextual clues accurately. If you read text generated by an AI, you may notice a disconnect between the content and its broader context.
| Example | Explanation |
| Jane smiled wistfully as she gazed out the window at the autumn leaves swirling in the crisp breeze. She had always found this time of year invigorating, like an old friend returning after a long absence. The sound of browser notifications pinging filled the air as Jane opened up her trading app. Perhaps now was an opportune moment to rebalance her portfolio and load up. After all, what better way to welcome the fall season than by hodling some more digital assets? |
In this example, the opening sentence sets a wistful, nostalgic tone as Jane reflects on the autumn season. However, the paragraph then abruptly shifts gears by bringing up trading and blockchain terminology like "hodling" digital assets. This jarring change in topic and tone demonstrates a lack of understanding of the contextual cues established in the initial sentence. |
Lack of Details
AI writers can generate detailed content; the problem arises when they give out predictable and excessive details that may feel contrived.
Human writers employ discretion when choosing what details to include and how to weave them to enhance a reader’s experience.
Conversely, AI content may bombard readers with unnecessary information at the expense of readability and coherence. You can use this indicator when detecting AI generated text.
| Example | Explanation |
| The scientific community has long debated the origins of the universe, with various theories attempting to explain the cosmic phenomena that gave rise to the vast expanse of space and time. Some researchers have proposed intriguing hypotheses, while others have put forth compelling arguments, but the true nature of the universe's beginnings remains shrouded in mystery. |
While the paragraph touches on the topic of the origins of the universe, it lacks specific details, examples, or explanations of the various theories and arguments mentioned, leaving the reader with an incomplete understanding of the subject matter. |
AI Content Detection Tools
Do you know how to check if something was written by AI? The quickest and easiest way is to use AI detection tools to identify AI-generated content. Some of the top tools stand out for their good reviews, readability, and excellent features.
-
Copyleaks
Copyleaks website has impressive readability, full model coverage (including OpenAI GPT-4 and Bard), and offers sentence-level analysis. It has a self-reported accuracy of 99.1% and understands 30 languages. It also has an extension that will be useful as a quick generation recognizer.
-
ZeroGPT
This identifier is all about simplicity and convenience. It detects numerous languages, including Hindi, German, Indonesian, French, and English. This finder also offers a ZeroGPT API feature to incorporate into other programs and apps and enjoy real-time generator content detection.
-
Crossplag
This free online tool is a very well-liked plagiarism checker. Recently, it has added an AI content detector feature that uses machine learning algorithms to determine whether the origin of a certain text is human or artificial.
The AI tracker recognizes the patterns characteristic to human content and identifies deviations from these patterns in percentages that indicate the origin of the content.
-
Winston AI
This scanner is very easy to use. It does in-depth text analysis and displays the probability of the text being ai-generated in percentages. It offers a sentence-by-sentence scan that lets you go deeper and identify the problematic areas.
The tool has a self-reported accuracy of 99.6%. It currently supports French, Spanish, German, and English languages.
We have conducted extensive research to determine which AI content detector shows the most accuracy. For this, we took a NY Times article, a legal document, an article about AI, four essays on different topics written in 2017–2018, an excerpt from the book written in 1923, and two texts generated by GPT-3 and GPT-4. We tested 9 AI detectors: OpenAI, Writer, GPTzero, SEO AI, zeroGPT, Crossplag, ContentDetector, Winston AI, and Sapling.
In this table, you can see the results of our research, where 0% indicates human-written content and 100% is AI-generated content. We have highlighted in green when the detectors successfully identified content as human- or AI-generated. Yellow highlights moderate results, while red indicates incorrectly detected content.
As you can see from our research, the best results were shown by GPTzero, Crossplag, and zeroGPT. Here is a graph to illustrate it more clearly:
Conclusion
We can conclude that AI writing is a significant innovation, but you should remember to only use it as a tool, not a replacement for hard work and originality.
As a student, the consequences of using AI for your essays can be painfully severe, and you should know how to detect AI, so if you need any help with your academic paper, consider using human writers you can trust from Studybay.
However, if you’re willing to take the risk, at least use Studybay AI, as it is an excellent academic writing tool.
Resources
- LibGuides: Use of Generative AI in Research: Perplexity and Burstiness. https://guides.library.unlv.edu/c.php?g=1361336&p=10054021
- Is AI-Generated content actually detectable? (2024, April 8). College of Computer, Mathematical, and Natural Sciences | University of Maryland. https://cmns.umd.edu/news-events/news/ai-generated-content-actually-detectable
- Rogers, R. (2023, February 8). How to detect AI-Generated Text, according to researchers. WIRED. https://www.wired.com/story/how-to-spot-generative-ai-text-chatgpt/
- Detecting Artificial intelligence (AI) plagiarism. Missouri Online | Teaching Tools. https://teachingtools.umsystem.edu/support/solutions/articles/11000119557-detecting-artificial-intelligence-ai-plagiarism
- Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023, April 6). GPT detectors are biased against non-native English writers. arXiv.org. https://arxiv.org/abs/2304.02819
- Turnitin. (n.d.). Ethical AI use checklist for students. https://www.turnitin.com/lessons/academic-integrity-in-the-age-of-ai-ethical-ai-use-checklist-for-students
FAQ
How to detect if something was written by AI?
You can detect AI writer using AI detection tools. They work by analyzing text patterns, syntax, and semantics. Their accuracy varies depending on their algorithm and training data, making them quite effective but not infallible in detecting ai-generated writing. AI detection tools include ZeroGPT, Winston AI, and Copyleaks.
How can you detect AI writings?
You can identify bot-generated content manually by analyzing the tone, sources, punctuation, keyword density, context, and the degree of detail in the work you’re interested in. This method will help you distinguish between human-written and AI-generated content by identifying characteristics that are unique to AI.
Can AI text be detected?
Yes, ai-generated text can be detected either manually through analysis of human writing characteristics or by specialized detection tools. It is possible to discern whether a piece of writing was ai-generated by leveraging knowledge of common AI characteristics.
How accurate are AI writing detectors?
The accuracy of AI writing detectors varies from one tool to another. Some tools excel at identifying ai-generated content with high confidence, but in general, the results depend on factors such as algorithm complexity, training data quality, and text nuances. It's essential to evaluate their performance critically.
