The New York Times broke a story this morning that, even as I read it over coffee, felt kind of inevitable.
“The author of a nonfiction book about the effects of artificial intelligence on truth acknowledged on Monday that he had included numerous made-up or misattributed quotes concocted by A.I.”
The book’s title? The Future of Truth: How AI Reshapes Reality.
The author, Steven Rosenbaum, is apparently one of those people who scratch together a living as “a convener,” tossing obsequious questions to wealthy tech wonks at conferences that promise to crunch or disrupt things.
Mullin, the reporter, got the scoop by carrying out the most basic duty of a reporter. He checked the facts. Himself. (Presumably.) Which is apparently more than the author did. Presented with evidence of “a handful of improperly attributed or synthetic quotes,” Rosenbaum told the Times that he had started his own investigation.
Look, I’m no Bob Woodward. But I have written a few nonfiction books. And I can tell you that the entire point of writing nonfiction is to do the investigating before you publish the book.
Nonfiction still a few paces behind fiction
Meanwhile, even as the Future of Facts story was breaking in the Times, Emily Temple and Brittany Allen were tossing fresh wood on the fire over at LitHub. Two stories, same day: Fiction written by AI. Or at least unduly influenced by same.
Nobel laureate Olga Tokarczuk apparently used AI to write her latest novel.
A prize-winning story published in Granta was (very likely) written by AI.
This comes, of course, on the heels of the Shy Girl kerfuffle a couple months ago. That’s the one where a horror-romance novel scheduled to hit bookstores this spring was pulled from publication after it was revealed to be mostly AI slop.
Two different conversations, two different rooms
At first glance, all of these stories seem to be about the same thing: Authors trying to pass off AI slop as original human work. And yet…I want to be careful here. I think the conversation around AI and literature should start as a plenary session in the grand ballroom, but then we need to depart into breakout sessions. One room for fiction. Another for nonfiction.
Nonfiction writers: Understand what AI is not
Chatbots can seem like an all-knowing, all-powerful fact repositories. They are not.
A chatbot like ChatGPT is the public-facing portal to a massive AI model, in this case OpenAI’s GPT-4.o. That AI model is not a fact checking wizard. It is a massively powerful prediction machine.
The model is a computer that has ingested billions of words and noted how those words appear near or together. Upon receiving a query form a human user, the AI model recognizes patterns in the query and its training data, and responds with what is the most likely response to the query. It often responds with factually accurate information. Just as often, it responds with inaccuracies. Sometimes when it can’t locate the information you seek it’ll just make shit up.
Lawyers are learning this the hard way right now. They’re relying on ChatGPT to find case citations. ChatGPT is delivering citations that look real and relevant—but in fact don’t exist.
The deceptive superpower of an AI model is its felicity with language and its presentation of confidence. ChatGPT, Gemini, or Claude will present a response with absolute confidence, in warm trust-building language, regardless of its accuracy.
My fellow nonfiction writers: AI can be a helpful tool. If you rely on it for factual accuracy you are putting your reputation, your career, your very livelihood in peril.
Nonfiction is a two-part process. You gather facts as accurately as possible. Then you go through the whole manuscript and double-check everything. It really wouldn’t have mattered if Steven Rosenbaum had used AI to help him compile and organize The Future of Facts—if only he had taken the time to check the facts and the quotes once the manuscript was complete. There are even people you can hire to do it for you. They’re called human fact checkers. There’s a reason they’re paid for their labor.
As for you fiction writers…
The conversation in the fiction breakout room is more complicated, I’m afraid. It involves a lot more subtlety, more fuzzy borders, and a deeper discussion of the unspoken contract between writer and reader.
The salient bits in the LitHub piece on Nobel novelist Olga Tokarczuk are contained in this (translated) quote: “When writing my latest novel… I asked this advanced [AI] model what kind of songs my protagonists would be listening to at a dance, a few dozen years ago, and AI gave me a few titles. Often I just ask the machine, ‘Darling, how could we develop this beautifully?’”
That seems to me to contain two vastly different levels of AI use. “Suggest some song titles that might have been popular in 1962,” or whatever, is an innocent idea-starter. (It also seems to takes the fun out of the game, for the writer. Concocting fictional-but-believable song titles, band names, store brands, etc, is one of the great delights of writing fiction.) “How could we develop this [story] beautifully?” is a whole different ask, and the validity of AI use will depend on how heavily the author relied on the machine’s response.
The rules are being written right now
When it comes to AI, in the literary world nobody knows anything right now. The line between honorable and dishonorable uses of the technology is blotchy. It’s fuzzy. Its location changes daily.
I’ll have more to say about the evolving ethics of AI use—and the ethics of public shaming for same—in an upcoming post.
One thing is certain: Nobody has a clear, correct answer right now. Especially not that chatbot.
MEET THE HUMANIST
Bruce Barcott, founding editor of The AI Humanist, is a writer known for his award-winning work on environmental issues and drug policy for The New York Times Magazine, National Geographic, Outside, Rolling Stone, and other publications.
A former Guggenheim Fellow in nonfiction, his books include The Measure of a Mountain, The Last Flight of the Scarlet Macaw, and Weed the People.
Bruce currently serves as Editorial Lead for the Transparency Coalition, a nonprofit group that advocates for safe and sensible AI policy. Opinions expressed in The AI Humanist are those of the author alone and do not reflect the position of the Transparency Coalition.
Portrait created with the use of Sora, OpenAI’s imaging tool.
