How George Orwell predicted AI
Note: I keep as much of this Substack free as I can, but writing takes time and energy and if you’re able to subscribe here at one of the paid tiers and support my work you’d be doing me a solid. If you’re unable to afford a paid subscription, consider subscribing for free or send me a message and I’ll gladly comp your subscription.
The image of totalitarianism found in Orwell’s Nineteen Eighty-Four has become so iconic that the subtleties of its world can sometimes be overlooked. Oceania — the novel’s thinly-fictionalized version of postwar Britain — is most often remembered as a place of absolute centralized control; a panoptic police state so intensely authoritarian that the private lives of individual people have effectively ceased to exist. This is true for people like the novel’s main characters Winston and Julia, who belong to Oceania’s ruling party. What’s often forgotten is that life outside the party’s inner confines is allowed to carry on under much less scrutiny.
The “proles”, who comprise Oceania’s vast underclass, actually enjoy certain rights that party members don’t, notably when it comes to leisure and entertainment. Here, the actual production of culture is still managed by the state, but the culture it ultimately produces — which is variously formulaic, dull, and titillating — is for the proles’ consumption alone:
And the Records Department, after all, was itself only a single branch of the Ministry of Truth, whose primary job was not to reconstruct the past but to supply the citizens of Oceania with newspapers, films, textbooks, telescreen programs, plays, novels—with every conceivable kind of information, instruction, or entertainment, from a statue to a slogan, from a lyric poem to a biological treatise, and from a child’s spelling book to a Newspeak dictionary. And the Ministry had not only to supply the multifarious needs of the Party, but also to repeat the whole operation at a lower level for the benefit of the proletariat. There was a whole chain of separate departments dealing with proletarian literature, music, drama, and entertainment generally. Here were produced rubbishy newspapers containing almost nothing except sport, crime, and astrology, sensational five-cent novelettes, films oozing with sex, and sentimental songs which were composed entirely by mechanical means on a special kind of kaleidoscope known as a versificator. There was even a whole sub-section—Pornosec, it was called in Newspeak—engaged in producing the lowest kind of pornography, which was sent out in sealed packets and which no Party member, other than those who worked on it, was permitted to look at.
Later in the book, when Winston watches a prole woman singing in her backyard, we even catch a glimpse of what the versificator produces:
Whenever her mouth was not corked with clothes pegs she was singing in a powerful contralto:
“It was only an ‘opeless fancy,
It passed like an Ipril dye,
But a look an’ a word an’ the dreams they stirred
They ‘ave stolen my ‘eart awye!“
The tune had been haunting London for weeks past. It was one of countless similar songs published for the benefit of the proles by a sub-section of the Music Department. The words of these songs were composed without any human intervention whatever on an instrument known as a versificator.
In writing this scene, Orwell was clearly drawing inspiration from media that already existed: e.g. tabloid newspapers, mass market fiction, the formulaic films churned out by big studios. His innovation, however, was to imagine a version of these things that required no human input whatsoever.
In the hands of a different writer, it’s easy to imagine the same idea manifesting as a promethean fantasy about machines surpassing Goethe or outdoing Shakespeare. But the versificator exclusively exists to produce an endless output of cultural slop whose sole purpose is distract, stimulate, and anesthetize. “It would probably not be beyond human ingenuity,” he relatedly observed in a 1946 essay called The Prevention of Literature, “to write books by machinery,” adding that “a sort of mechanizing process can already be seen at work in the film and radio, in publicity and propaganda, and in the lower reaches of journalism.”