May 25th, 2024
When I first heard about AI detection for text or images, I thought, yeah, sure, that makes sense. Lots of people are freaking out about whether there’s AI even in the same room, so odor detectors might have to be installed. I didn’t give it much more thought.
I was already subscribing to Ethan Mollick’s AI newsletter, One Useful Thing. Over a year ago he reported that it’s “impossible to build a perfect AI detector.” By July last year he was cautioning against the use of AI detection, particularly in schools, observing that “detectors have high false positive rates, accusing people (and especially non-native English speakers) of using AI when they are not.” This could easily be disastrous for students, leading to unfair accusations and penalties.
Mollick pointed to scholarly papers on this topic, and I headed off in that direction. A key paper, from February, 2023, “Can AI-Generated Text be Reliably Detected?” concluded (among other things) that “these detectors are not reliable in practical scenarios.”
I multiplied “impossible to accurately detect” by “big risk in false positives” and wrote the topic off.
The Challenges of AI Detection
Then, in January of this year, the “Binoculars” paper appeared, claiming that across “a wide range of document types, Binoculars [software] detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data.” This claim seemed promising but was quickly challenged.
Just a few weeks ago, in the paper “RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors” the researchers concluded that “detectors are not yet robust enough for widespread deployment or high-stakes use.”
Mollick, meanwhile, challenged the Binocular paper, pointing out that “tiny easy tweaks break detectors and they don’t test those approaches.”
Well, what was I to do?
Obviously, schedule a conversation on the topic!
And so, this Tuesday, supported by the Book Industry Study Group (BISG), I’ll be moderating a one-hour webinar titled “Can AI Be Detected in Writing?” The goal is to explore whether AI can be reliably detected, assess the accuracy of detection systems, and understand the potential downsides of inaccurate assessments, particularly for authors and publishers.
AI in Education
Our first presenter will be José Antonio Bowen, a Senior Fellow at the American Association of Colleges and Universities (AAC&U). José has just co-authored (with C. Edward Watson) a book on AI in education, called Teaching with AI: A Practical Guide to a New Era of Human Learning. It’s currently #1 in “Computers & Technology Education” on Amazon, as it has been for several weeks.
What drew me to the book was its chapter on “Cheating and Detection.” The authors examine both the ethical and procedural issues surrounding AI detection. The chapter stands out for its detailed analysis of various AI-detection software, acknowledging their strengths while clearly outlining their many failings. It offers a balanced view that’s crucial for understanding the complexities of AI detection in an educational context.
Bowen has a template that he uses for student disclosure around the use of AI, which makes for interesting reading if you imagine using such a checklist with book authors:
Template Disclosure Agreement for Students
- I did all of this work on my own without assistance from friends, tools, technology, or AI.
- I did the first draft, but then asked friends/family, AI paraphrase/grammar/plagiarism software to read it and make suggestions. I made the following changes after this help:
- Fixed spelling and grammar
- Changed the structure or order
- Rewrite entire sentences/paragraphs
- I got stuck on problems and used a thesaurus, dictionary, called a friend, went to the help center, used Chegg or other solution provider.
- I used AI/friends/tutor to help me generate ideas.
- I used assistance/tools/AI to do an outline/first draft, which I then edited. (Describe the nature of your contribution.)
It points to the intricacies of the issues around AI use for students… are the issues so different for book authors? How are authors to perceive their responsibilities surrounding the use of AI? And, as importantly, a disclosure of its use, knowing that there can be severe penalties imposed.
24 Ways for Authors to Use AI
I chatted with several bots on the topic of “What are all the ways an author can use AI to help in their writing?” Here’s the consensus, as edited by me:
Content creation
- Content ideation
- Content structuring
- World-building
- Dialogue and character development
- Emotion and sentiment analysis
- Research assistance
- Feedback and critique
- Writing prompts for overcoming writer’s block
- Summarization
Content editing
- Spelling check
- Grammar check
- Style and tone analysis
- Plagiarism detection
- Thesaurus and synonyms
- Line editing and proofreading
Special tasks
- Language translation
- Interactive storytelling
- Audiobook generation
- Visual content creation
- SEO optimization
- Market analysis/competitive analysis
- Voice recognition for transcription
- Formatting for publication
- Metadata generation
It’s obviously impossible for any software tool to detect all of these uses, even just the ones listed under “content creation.” At the same time, what fiction author wouldn’t want to consider using a tool that could help them with “world-building” and “dialogue and character development”?
Another chapter in the book Teaching with AI looks in part at plagiarism, a major challenge within academia, but also a problem for book publishers (often discussed alongside the conundrum of fact-checking for books).
Recognizing how AI shakes up the concept of plagiarism, the authors propose a somewhat relaxed redefinition within an academic context. But this wouldn’t work for book publishers — revealed instances of plagiarism will certainly undermine the lifespan of a book, and can be a career-ending experience for an author. It’s provocative to ponder the potential role for AI within these challenges, but that’s for another day.
The Role of Publishers
After my chat with José I’ll welcome Greg Britton, who’s the editorial director of Hopkins Press Books at Johns Hopkins University Press. His publishing experience runs deep, including a stint as publisher for Getty Publications. I was fortunate to meet him for the first time in April at the Johns Hopkins University Press Advisory Board meeting.
Greg happens to be the commissioning editor for Teaching with AI, but I knew that his breadth of experience would make him the ideal candidate to pose the question: Should publishers be using AI detection software to screen the manuscripts they review?
What a dilemma! It’s easy just to answer, yes, sure they should. Why wouldn’t they?
But what a can of worms that opens. What is the problem that publishers are trying to solve for which AI detection software is the solution?
I can think of two main justifications for wanting to do everything you can as a publisher to detect whether AI was used to create a specific manuscript. Scenario one is just a matter of trust, as it might be with a student. Dear author: on your submission questionnaire, you indicated that you did not use AI in writing your book. But we ran your manuscript through some software that we (mostly) rely on and it indicated otherwise.
Then what?
Copyright
A far more pragmatic issue is the copyrightability of AI-generated text and other content. Much has been written on the topic — google it — but the short form of the short form is that AI-generated material cannot be protected by copyright — anyone can reuse it. Obviously book publishers aren’t going to issue books commercially where they can’t enforce copyright, so they need to know if authors are using AI to generate their text.
Except, as with so many AI issues, this one quickly degrades. What if, for example, an author employs AI just for use #8 above, “writing prompts for overcoming writer’s block.” The resulting text might include a phrase or a sentence that’s AI-generated. But the work as a whole is the writer’s own — is it just that sentence that’s in the public domain? And, if the author denied using AI at all, could any of the AI detection software actually find the offending sentence? And if it could?
Detection Software
The conversation will move over next to Max Spero, who is the co-founder & CEO of Pangram Labs, one of two dozen or more startups that are embracing the challenge of using AI to create tools to detect AI-generated content. Pangram’s website claims that it has “the most accurate technology to detect AI generated content at scale,” and Max will take us into the weeds that surround that claim.
One of the things that struck me in my first conversation with Max was his admonition that AI-detection software should not be used “with a punitive mindset.”
Evading Detection
As surely as there is software that undertakes to detect AI in writing, there’s another class of software that undertakes to help evade AI detection in writing, by “humanizing” your AI-generated text. We won’t be looking at these tools in the webinar, but if you’re interested, check out sites like Undetectable.ai and StealthGPT (a mention here ≠ an endorsement).
Detecting AI in Images
Detecting AI in images is another big topic that we’re going to sidestep altogether. It clearly has relevance to nonfiction publishing, with new illustrations appearing alongside reproductions of existing ones. I’m not seeking to diminish the importance of this part of the story; only acknowledging that nearly all books published contain text, but many of them are image-free.
The AI Series
This webinar is the first in a series of webinars — currently a series of two. The next event, on June 25, will tackle the question “can AI be useful in book translation?” Then we’ll take the summer off, and afterwards, based on the response to these first two programs, figure out how to best resume the conversation in September.
“Conversation” is the operative word here. These aren’t presentations, they’re conversations. AI is running at a breathtaking pace, and all we can do is grab a moment in time to drill part-way down on one of the many subjects that surround its impact on authoring and publishing.
I certainly don’t have the answers, but I hope that I’m learning how to formulate the questions.
Hope to see you on Tuesday. If you can’t be there in person, go ahead and register for the webinar recording. I’ll also post an edited and annotated transcript on this blog.
PS: People wonder which of the many AI text detection programs are worth trying. Based on several tests reported in scholarly papers, the leading contenders appear to be (in no particular order) GPTZero, Originality, Turnitin, Winston, and, of course, Pangram Labs.