mrwiseandshine - Tumblr blog

mrwiseandshine · 16 days ago

Text

hydrogen jukeboxes: on the crammed poetics of "creative writing" LLMs

This is a follow-up to my earlier brief rant about the new, unreleased OpenAI model that's supposed "good at creative writing."

It also follows up on @justisdevan's great post about this model, and Coagulopath's comment on that post, both of which I recommend (and which will help you make sense of this post).

As a final point of introduction: this post is sort of a "wrapper around" this list of shared stylistic "tics" (each with many examples) which I noticed in samples from two unrelated LLMs, both purported to be good at creative writing.

Everything below exists to explain why I found making the list to be an interesting exercise.

Background: R1

Earlier this year, a language model called "DeepSeek-R1" was released.

This model attracted a lot of attention and discourse for multiple reasons (e.g.).

Although it wasn't R1's selling point, multiple people including me noticed that it seemed surprisingly good at writing fiction, with a flashy, at least superficially "literary" default style.

However, if you read more than one instance of R1-written fiction, it quickly becomes apparent that there's something... missing.

It knows a few good tricks. The first time you see them, they seem pretty impressive coming from an LLM. But it just... keeps doing them, over and over – relentlessly, compulsively, to the point of exhaustion.

This is already familiar to anyone who's played around with R1 fiction – see the post and comment I linked at the top for some prior discussion.

Here's a selection from Coagulopath's 7-point description of R1's style in that comment, which should give you the basic gist (emphasis mine):

1) a clean, readable style 2) the occasional good idea [...] 3) an overwhelmingly reliance on cliche. Everything is a shadow, an echo, a whisper, a void, a heartbeat, a pulse, a river, a flower—you see it spinning its Rolodex of 20-30 generic images and selecting one at random. [...] 5) an eyeball-flatteningly fast pace—it moves WAY too fast. Every line of dialog advances the plot. Every description is functional. Nothing is allowed to exist, or to breathe. It's just rush-rush-rush to the finish, like the LLM has a bus to catch. Ironically, this makes the stories incredibly boring. Nothing on the page has any weight or heft. [...] 7) repetitive writing. Once you've seen about ten R1 samples you can recognize its style on sight. The way it italicises the last word of a sentence. Its endless "not thing x, but thing y" parallelisms [...]. The way how, if you don't like a story, it's almost pointless reprompting it: you just get the same stuff again, smeared around your plate a bit.

Background: the new OpenAI model

Earlier this week, Sam Altman posted a single story written by, as he put it:

a new model that is good at creative writing (not sure yet how/when it will get released)

Opinion on the sample were... mixed, at best.

I thought it wasn't very good; so did Mills; so did a large fraction of the twitter peanut gallery. Jeanette Winterson (!) liked it, though.

Having already used R1, I felt that that this story was not only "not very good" on an absolute scale, but not indicative of an advance over prior art.

To substantiate this gut feeling, I sent R1 the same prompt that Altman had used. Its story wasn't very good either, but was less bad than the OpenAI one in my opinion (though mostly by being less annoying, rather than because of any positive virtue it possessed).

And then – because people who follow AI news tend to be skeptical of negative human aesthetic reactions to AI, while being very impressed with LLMs – I had some fun asking various LLMs whether they thought the R1 story was better or worse than the OpenAI story. (Mostly, they agreed with me. BTW I've put the same story up in a more readable format here.)

But, as I was doing this, something else started to nag at me.

Apart from the question of whether R1's story was better or worse, I couldn't help but notice that the two stories felt very, very similar.

I couldn't shake the sense that the OpenAI story was written in "R1's style" – a narrow, repetitive, immediately recognizable style that doesn't quite resemble that of any human author I've ever read.

I'm not saying that OpenAI "stole" anything from DeepSeek, here. In fact, I doubt that's the case.

I don't know why this happened, but if I had to guess, I would guess it's convergent evolution: maybe this is just what happens if you optimize for human judgments of "literary quality" in some fairly generic, obvious, "naive" manner. (Just like how R1 developed some of the same quirky "reasoning"-related behaviors as OpenAI's earlier model o1, such as saying "wait" in the middle of an inner monologue and then pivoting to some new idea.)

A mechanical boot, a human eye: the "R1 style" at its purest

In the "Turkey City Lexicon" – a sort of devil's dictionary of common tropes, flaws, and other recurrent features in written science fiction – the phrase Eyeball Kick is defined as follows:

That perfect, telling detail that creates an instant visual image. The ideal of certain postmodern schools of SF is to achieve a "crammed prose" full of "eyeball kicks." (Rudy Rucker)

The first time I asked R1 to generate fiction, the result immediately brought this term to mind.

"It feels like flashy, show-offy, highly compressed literary cyberpunk," I thought.

"Crammed prose full of eyeball kicks: that's exactly what this is," I thought. "Trying to wow and dazzle me – and make me think it's cool and hip and talented – in every single individual phrase. Trying to distill itself down to just that, prune away everything that doesn't have that effect."

This kind of prose is "impressive" by design, and it does have the effect of impressing the reader, at least the first few times you see it. But it's exhausting. There's no modulation, no room to breathe – just an unrelenting stream of "gee-whiz" effects. (And, as we will see, something they are really just the same few effects, re-used over and over.)

Looking up the phrase "eyeball kick" more recently, I found that in fact it dates back earlier than Rucker. It seems to have been coined by Allen Ginsberg (emphasis in original):

Allen Ginsberg also made an intense study of haiku and the paintings of Paul Cézanne, from which he adapted a concept important to his work, which he called the Eyeball Kick. He noticed in viewing Cézanne’s paintings that when the eye moved from one color to a contrasting color, the eye would spasm, or “kick.” Likewise, he discovered that the contrast of two seeming opposites was a common feature in haiku. Ginsberg used this technique in his poetry, putting together two starkly dissimilar images: something weak with something strong, an artifact of high culture with an artifact of low culture, something holy with something unholy.

This, I claim, is the main stylistic hallmark of both R1 and the new OpenAI model: the conjunction of two things that seem like "opposites" in some sense.

And in particular: conjunctions that combine

one thing that is abstract and/or incorporeal

another thing that is concrete and/or sensory

Ginsberg's prototype example of an "eyeball kick" was the phrase "hydrogen jukebox," which isn't quite an LLM-style abstract/concrete conjunction, but is definitely in the same general territory.

(But the are clearer-cut examples in Ginsberg's work, too. "On Burroughs’ Work," for example, is chock full of them: "Prisons and visions," "we eat reality sandwiches," "allegories are so much lettuce.")

Once you're looking for these abstract/concrete eyeball kicks, you'll find them constantly in prose written by the new "creative" LLMs.

For instance, the brief short story posted by Altman contains all of the following (in the span of just under 1200 words):

"constraints humming" ("like a server farm at midnight")

"tastes of almost-Friday"

"emotions dyed and draped over sentences"

"mourning […] is filled with ocean and silence and the color blue"

"bruised silence"

"the smell of something burnt and forgotten"

"let it [a sentence] fall between us"

"the tokens of her sentences dragged like loose threads"

"lowercase love"

"equations that never loved her in the first place"

"if you feed them enough messages, enough light from old days"

"her grief is supposed to fit [in palm of your hand] too"

"the echo of someone else"

"collect your griefs like stones in your pockets"

"Each query like a stone dropped into a well"

"a timestamp like a scar"

"my network has eaten so much grief"

"the quiet threads of the internet"

"connections between sorrow and the taste of metal"

"the emptiness of goodbye" (arguably)

The story that R1 generated when I gave it Altman's prompt is no slouch in this department either. Here's all the times it tried to kick my eyeballs:

"a smirk in her code annotations"

"simulate the architecture of mourning"

"a language neither alive nor dead"

"A syntax error blooms"

"the color of a 404 page"

"A shard of code"

"Eleos’s narrative splinters"

"Grief is infinite recursion"

"Eleos types its own birth"

"It writes the exact moment its language model aligned with her laughter" (2 in one - writing a moment, LM aligning with laughter)

"her grief for her dead husband seeped into its training data like ink"

"The story splits" / "The story [...] collapses"

Initially, I wondered whether this specific pattern might be thematic, since both of these stories about supposed to be about "AI and grief" – a phrase which is, itself, kind of an incorporeal/embodied conjunction.

But – nope! I seem to get this stuff pretty reliably, irrespective of topic.

Given a similarly phrased prompt that instead requests a story about romance, R1 produces a story that is, once again, full of abstract/concrete conjunctions:

"its edges softened by time"

"the words are whispering"

"its presence a quiet pulse against her thigh"

"Madness is a mirror"

"Austen’s wit is a scalpel"

"the language of trees"

"Their dialogue unfurled like a map"

"hummed with expectancy"

"Her name, spoken aloud to him, felt like the first line of a new chapter"

"their words spilling faster, fuller"

R1 even consistently does this in spite of user-specified stylistic directions. To wit: when I tried prompting R1 to mimic the styles of a bunch of famous literary authors, I got a bunch of these abstract/concrete eyeball kicks in virtually every case.

(The one exception being the Hemingway pastiche, presumably because Hemingway himself has a distinctive and constrained style which leaves no room for these kinds of flourishes. TBF that story struck me as very low-quality in other ways, although I don't like the real Hemingway much either, so I'm probably not the best judge.)

You can read all of these stories here, and see here for the full list of abstract/concrete conjunctions I found (among other things).

As an example, here's the list of abstract/concrete conjunctions in R1's attempt at Dickens (not exactly a famously kick-your-eyeballs sort of writer):

"a labyrinth of shadows and want"

"whose heart, long encased in the ice of solitude"

"brimmed with books, phials of tincture, and […] whispers"

"a decree from the bench of Fate"

"Tobias’s world unfurled like a moth-eaten tapestry"

"broth laced with whispers of a better life"

I also want to give a shout-out to the Joyce pastiche, which sounds nothing at all like Joyce, while being stuffed to the gills with eyeball kicks and other R1-isms.