#just...reductive scare piece with pretty new yorker language and a disinterest in asking difficult questions in a pro-progress manner

[tam--lin]

Can you recognize where the AI writer takes over?

While I've really enjoyed nearly everything in this year's Best Science and Nature Writing, this one New Yorker article about GPT-3 really rubbed me the wrong way. (You can read it online here, if you’d like.) I won't go into details now, as it would turn into a lengthy rant and I have other things to do this weekend, but something that particularly struck me was a test of recognizing AI-generated text and how many experts apparently failed it.

GPT-3, if you don't know, is a writing AI. It can't think, but it can parse huge amounts of information and create grammatical, (generally) coherent sentences and paragraphs. It is not particularly good at creating arguments. Here's an example:

"He was wearing a tweed suit, over a shiny sweater, and his black hair was brushed back. He had a red beard and wore his waistcoat in an overcoat with the body of a ship, three broad belts of colorful chain-link, a pair of capacious rectangular eyeglasses, and a silk tie. “Gouging my eye,” he said, in Italian, saying that he had caused himself that terrible scar, “the surgeon said it wasn’t that bad.” When he was very young, he said, he started smoking but didn’t find it very pleasant. The cigarette burns in his hands and wrists were so bad that he had to have his face covered."

It reminds me of what I'd sometimes write during NaNoWriMo while quite literally falling asleep at my desk. Sentences, yes, English, yes. Scans correctly, but when you try to parse its deeper meaning you've just got sand running through your fingers. In an email, Dr. Steven Pinker called it "superficially plausible gobbledygook".

Which brings me to the "Pinker Test". The author put part of Pinker's email into GPT-3, joined the actual email to the GPT-3 predicted text, and sent the undemarcated result to experts he'd interviewed for the story. The challenge is to point to where Pinker's writing stops and GPT-3's predictive text begins.

Here's the paragraph. (So as not to spoil the challenge, I'll leave you to click on the article to find out whether you're right or not.)

"Being amnesic for how it began a phrase or sentence, it won’t consistently complete it with the necessary agreement and concord—to say nothing of semantic coherence. And this reveals the second problem: real language does not consist of a running monologue that sounds sort of like English. It’s a way of expressing ideas, a mapping from meaning to sound or text. To put it crudely, speaking or writing is a box whose input is a meaning plus a communicative intent, and whose output is a string of words; comprehension is a box with the opposite information flow. What is essentially wrong with this perspective is that it assumes that meaning and intent are inextricably linked. Their separation, the learning scientist Phil Zuckerman has argued, is an illusion that we have built into our brains, a false sense of coherence."

I read this once, slowly, and picked out without too much trouble where coherent slipped to questionable.  But the author goes on to say that, "almost everyone I tried the Pinker Test on, including Dario Amodei, [Director of Research for] [...] OpenAI, and Les Perelman, of [MIT essay-writing bot] Project babel, failed to distinguish Pinker’s prose from the machine’s gobbledygook."

Which genuinely shocked me! Knowing the basics of how GPT-3 worked, it wasn’t hard for me to pick up the slip in logic, tone, and wording where the AI takes over.

Anyway, all of this to say, I’m foisting the Pinker test on you. Can YOU tell where the shift happens? Is this actually harder than I think it is, and is calling BS on AI-written text my YA dystopian protagonist special skill? Or is the article overselling GPT-3′s coherence at the expense of the poor experts this test was forced upon?