While AI tools struggle to detect AI-generated content, a new AI detection tool by University of Kansas can identify ChatGPT-generated content with 99% precision.
Latest AI models such as OpenAI’s ChatGPT, Google’s Bard have provoked a pitfall, making it difficult to differentiate between AI-generated and human-written content, leading to rise of plagiarism. Affected the most are educational staffs and professionals, being not able to detect if the students’ assignments or reports are AI-generated or not.
While few tools including OpenAI’s own ‘Text Classifier’ aims to identify AI-content, the precision is not to the standards, sometimes falsely accusing students. However, researchers from University of Kansas had published a paper claiming a 99% efficacy in their AI-detection method.
99% accurate AI-detecting Tool
Heather Desaire, author of the paper, and a chemistry professor at University of Kansas said that her AI-detection tool was focused on scrutinizing the scientific writing of her peers’ journals, i.e., human-written papers than on AI-generated.
She says, it’s easy to differentiate human-made researches from AI-content than detecting AI-content amidst human-written content.
““You can easily build a method to distinguish human from ChatGPT writing that is highly accurate, given the trade-off that you’re restricting yourself to considering a particular group of humans who write in a particular way,” Desaire said.
So, it’s basically not an AI-classifier, but a human-classifier from AI-texts.
How they made it?
In their progress, she used 64 human-written documents and 128 AI documents as training data, which might be 100,000 times smaller than size of datasets used to train other detectors.
With such a smaller dataset, they made the AI to emphasize on human texts rather than AI texts. They say that human writing styles is peculiar and identifiable, as human texts are rich in vocabulary, have longer paragraphs containing more diverse words than machines. Further, few writing syllables like punctuations, and more use of equivocal words such as “however”, “but”, “although”, “this”, “because” helps the AI detector to identify human-written texts easily.
Initial experiments made the differentiator to achieve 100% accuracy few times, but it dropped to 92%, when the inspection discerned out to paragraph-level.
Skeptic about results
The researchers, however, says the results should be taken as a grain in salt. It’s not clear how robust the algorithm is against AI-generated studies that are edited by humans or against real papers from other scientific journals.
“Since the key goal of this work was a proof-of-concept study, the scope of the work was limited, and follow-up studies are needed to determine the extent of this approach’s applicability,” the researchers wrote in their paper. “For example, the size of the test set (180 documents, ∼1,200 paragraphs) is small, and a larger test set would more clearly define the accuracy of the method on this category of writing examples.”
Why the need of AI-detecting tool?
The emerge of artificial general intelligence technology paved the way for more number of fake contents in the internet, especially in research papers. As reported by scientific journal websites like nature, AI entangles the genuineness of research articles, as ChatGPT or other AI-created content are filling up the website’s papers.
“We have to really look at how we identify those things, and how we make sure that people have actually done the research,” says one of the research-integrity consultant Deborah Kahn.
Indeed, people have been deluded by AI-generated which had let to legal actions – including a Chinese man getting arrested for using ChatGPT to generate fake news in China and a lawyer who used ChatGPT facing penalty for making up AI-generated citations that never happened.
Professional applications hereafter pose a need of valuation if the data and information is accurate and verifiable, and AI-detection tools might help for the screening process of this valuation.
Other Similar AI-Detecting tools
TurnitIn
Turnitin launches AI detection to help educators identify when AI writing tools such as ChatGPT have been used in students’ submissions. TurnitIn claims that their AI detection tool is 98% accurate at spotting AI-written texts.
Its AI model can detect texts generated by GPT-3 and GPT-3.5 language models, from where the AI is trained on.
CopyLeaks
Copyleaks claims of developing an AI detection at 99% precision. Unlike TurnitIn, Copyleaks can detect content generated by various AI models including, ChatGPT and Jasper AI. Moreover, it can detect AI content in multiple languages, including Spanish, Russian, French, Dutch and German.
Winston AI
Winston AI too claims that it can perform the task with 99% accuracy. Supporting detection in English and French, Winston AI can detect content made using ChatGPT, Bard, Bing Chat, GPT-4 and other text generation tools. It’s looking to expand compatibility of languages to Spanish and German.
Kindly add ‘The Inner Detail’ to your Google News Feed by following us!
Hope you find the page useful!
(For more such interesting informational, technology and innovation stuffs, keep reading The Inner Detail).
