AI Detection in Science: A Step Forward or a Misstep?
Like many others, I have been very skeptical about AI detectors. And my default position, up to now is that you should not use them. Ever.
Last week, there was a news bit on the Nature website suggesting that there had been huge progress in AI detection. It was entitled: ‘ChatGPT detector’ catches AI-generated papers with unprecedented accuracy and reported on a paper published in Cell Reports Physical Science titled: Accurately detecting AI text when ChatGPT is told to write like a chemist.
So, maybe I was wrong? The new tool has impressive accuracy, and outdoes previous models by a long shot. Maybe it has cracked the code on AI-written texts?
Too good to be true?
But this apparent breakthrough does not withstand scrutiny. I have reservation about the approach for two main reasons.
First, the tool is trained on specific introductory sections of chemistry papers which could reduce its effectiveness in other contexts. There is a risk of overfitting, the tool does great on very specific narrow types of texts and badly on all other input.
If this critic has legs, the tool would struggle with generalization. And guess what? It does. When given articles from university newspapers, it couldn’t recognize them as human-written. If it can’t adapt to different types of academic writing, its utility becomes pretty limited.
But, you might say, it is still useful for detecting AI written chemistry introductions. It may be so, but I doubt that even this would be true. I am also not convinced that this is even something we should want to do. Let me explain.
The tool detected text generated by AI that was unmodified. If you have used LLMs for a while, you know that the raw output is often badly flawed, and cannot be used as is, but it can be a very useful starting point for refinement. So most users will refine and modify AI outputs, making the detector’s effectiveness lower, or negating it completely.
Another, maybe more important, question is should we try to detect AI writing? The assumption behind this new detector seems to be that AI-assisted science is inferior to human-only science. Surely, the only thing that should matter is the quality of the knowledge created, not how it has been created. In addition, the evidence that is starting to accumulate suggest that LLMs are productivity and creativity boosters (see Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality). 🚀🧠 If that is the case, wouldn’t we be better off focusing on how we can use AI to enhance our productivity and do better science, rather that going on a fool’s errand trying to develop AI detectors?
I think this underscores why critical thinking and data literacy are essential in our AI-driven world. Reading past the headline, understanding the nuances, and questioning the capabilities and limitations of technologies are key to enable you to understand the world and make good decisions.
In conclusion, while AI detectors like this might seem promising on the surface, they still do not work. And even if they worked, using them still seems to be the wrong thing to do. So, once again, do not use AI detectors. Ever!
What are your thoughts on the use of AI detection tools in academic research? Let me know in the comments below.
Thanks to Ethan Mollick for tweeting about the news article in Nature that led to this rant. 🙂
References
‘ChatGPT detector’ catches AI-generated papers with unprecedented accuracy
https://www.nature.com/articles/d41586-023-03479-4
Accurately detecting AI text when ChatGPT is told to write like a chemist.
https://www.sciencedirect.com/science/article/pii/S2666386423005015?via%3Dihub
Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321
