Stuff They Don’t Tell You — Should you collaborate with AI?

In 2019, Berlin-based musician Holly Herndon released an album called “PROTO,” which, in its most elaborate and interesting tracks, married Sacred Harp singing with a “nascent machine intelligence” named Spawn.

Herndon created Spawn in partnership with collaborator Mat Dryhurst and AI expert Jules LaPlace; together they trained the AI on a diet of vocal polyphony, including the genre popularized by the rural Christian communities of New England in the mid-1800s. Sacred Harp is sung a capella, without instrumental accompaniment, and is traditionally printed as sheet music, using “shape notes” that allow people with no musical training to sight read effectively. As an introduction to the musical form and structure of the Western canon, it’s a pretty good place to start an AI on its journey.

The resulting album is not only an extraordinary sonic experience, but also a useful primer on the way AI can be used as a creative collaborator. “Spawn is a member of the ensemble like any other, and contributes ideas to the work as an equal,” said Herndon in an interview with the Barbican. “This is a special, and symbolic, relationship to develop with the subject matter.”

As Spawn’s neural networks processed the various harmonies and melodies it was fed, it returned weird and wonderful variations on those themes. Some of these Herndon was able to incorporate into the final album, but much of it was discarded for being too inaccessible, or just plain unpleasant to the ear.

“‘Godmother’ is obviously a really ugly piece of music,” said Dryhurst of the album’s penultimate track. “But that was the point.” By including the AI’s unedited output on the final album, Herndon and Dryhurst clearly demonstrated the incredible potential of human collaboration with AI, while simultaneously highlighting the limitations of the technology at the time. Unfiltered and unedited by Herndon, Spawn’s output was largely dissonant noise, but as part of an ensemble, the contribution of the AI was extraordinary, and pushed Herndon’s music in a direction it couldn’t otherwise have taken.

In the five years since “PROTO” landed, AI technologies have found their way into nearly every facet of modern work and creative practice. “There’s not one part of the creative industries that AI isn’t attempting to influence,” says Tim Noakes, formerly editor in chief of Dazed, who more recently led the content creation at Synthesia, an AI unicorn that produces lifelike avatar video content from text prompts. “Journalism, music, art, photography, radio, fashion, film and video production—all of these disciplines have AI tools that have been designed to help simplify, improve or evolve the creative process.”

While the availability of AI tools has become near ubiquitous, and barriers to entry almost non-existent (you don’t have to train your own neural net to use AI anymore, you can just plug in some prompts and go), a lot of the output of AI is still just as ugly and imperfect as “Godmother,” and the value AI can bring to creative practice depends an awful lot on the kind of practitioner you are.

In a recent episode of “This American Life,” in a spot called “I Wish I Knew How to Force Quit You,” writer Simon Rich “grapples with an AI chatbot that threatens to make him obsolete.” Or at least that’s the pitch made by the episode summary. In reality, despite his initial awe, Rich is repeatedly underwhelmed when he looks back at the bizarre musings of an early, unpublicized OpenAI chatbot trained on a whole internet of data and prompted to reflect on deep existential questions or simply directed to write snappy one-liners. When reciting code-davinci-002’s Onion-style headlines, Rich and presenter David Kestenbaum laugh weakly at one or two. But the rest of them suck. They’re not funny. This AI definitely isn’t coming for Rich’s job.

Later in the episode the AI spouts ramblings that read like they’re from every sci-fi novel you’ve ever read about machine intelligence. Because they are. Everything the AI knows about itself comes from the world of science fiction and, worse, the last ten years of internet journalism, where humanity’s deepest fears about the tech have been writ large in countless think pieces, reports and headlines. Speaking those words back to us through the medium of a chatbot renders them uncanny, and weirder than they’d sound written in a book, but for people like me, who write very specific words about specific subjects, they don’t yet seem to pose a threat. Yet.

Noakes agrees. “AI can enhance human capabilities, but as we all know, it can also produce a lot of derivative rubbish.” While Noakes and his teams at Synthesia have created hugely successful viral videos to promote the brand’s avatar products to a Gen Z audience, the creative work itself was largely carried out by humans. “When ChatGPT was first introduced, we tried using it to generate ideas for content campaigns,” he says, “but it didn’t produce anything particularly interesting.” 

Unique ideas, he says, “...are inherently human and leftfield. I’ve found that AI helps push the admin of creativity forward, serving as a tool, rather than being the source of creativity itself.”

The same sentiment is echoed by New York-based photographer Charlie Engman, who has been embedding AI tools like Midjourney into his artistic practice since the algorithm spat out an image that gave him pause. “It made some images that I actually thought were really compelling,” he says, “not because they’d been produced by an emerging technology, per se, but because the image had an inherently interesting quality to it. I was drawn to it.”

For Engman, the appeal of the technology comes from the uncanny and sometimes freakish qualities of the images it produces. On the surface, the images he’s trained Midjourney to make appear perfectly sensible, but scratch the surface and you’ll see that the world within the image is not our own; its physical limitations of time and space do not apply. “It fucks things up and creates relationships that are impossible in real physical space, but are somehow plausible,” says Engman, “and those are ideas that already interested me in my work more generally.”

Like Noakes, Engman doesn’t feel the AI is doing the creative work (although he does allow it to participate more directly in the final creative output). Rather, he sees it as a tool that can enhance his own thinking, not to mention enable the creation of images that would be impossible, or at least exorbitant, to create IRL.

As with any collaboration, working with AI requires certain conditions and parameters to enable smooth interaction between human and machine intelligence, as well as a bit of critical thinking to decide whether it’s a piece of tech you need your creative practice to engage with at all.

“Contrary to the common fear that people are gonna lose their jobs, I think I’ve only gained work since using AI,” says Engman. “But obviously, that’s because I’ve really invested in the technology, and I’m very proactive about it.” If you want AI to become a collaborator, then you’ll need to stop fearing that it might be out to get you.

“From an abstract and creative point of view,” says Engman, “AI is great.”

If you’ve still got the heebie-jeebies, perhaps you need to better understand or value your craft and appreciate the unique set of skills you bring to the table. If you’re a great storyteller, AI might enable you to push your thinking in new directions, or if you write music, perhaps it could help you to experiment with different sounds. Maybe like me, you just want it to speed up your interview transcriptions so you can write articles more efficiently. You can’t work with AI unless you clearly understand your own creative utility.

“As humans, we value culture and the act of creation,” says Noakes. Working with AI doesn’t have to devalue what you personally bring to that act.

In a recent LinkedIn post, British designer and entrepreneur Nicolas Roope wrote about how creative teams were using language learning models (LLMs) like ChatGPT to ensure their own thinking wasn’t cliche and boring.

“When starting a brainstorm or ideation session, quizzing the LLMs leads to a list of suggestions of what creative teams are generally most likely to suggest. At which point the team knows what NOT to do… The list of the obvious.”

Despite what the AI hype machine would have you believe, working with AI tools takes time and understanding. Whether you’re using Runway or Pika for your text-to-video needs, each has their quirks and differences. In order to make any interesting work, Engman first had to train Midjourney over countless hours. “I became addicted to it,” he says. But that addiction led to a better understanding of the AI’s capabilities and the way that different prompts and inputs could achieve better results.

“I think the lack of people making interesting work using AI tools really demonstrates that it’s not so easy to use. Like photography, the process has an immediacy to it that suggests ease, but actually using it in a way that makes something engaging is hard.”

You can’t really talk about AI without rubbing up against the thorny subject of ethics, and whether or not we should allow both tech companies and individuals to profit from creativity that’s not entirely their own. All AIs are trained on vast datasets, most of which have been scoured from online resources without the express permission of their creators. 

As far as Engman’s concerned, this is no different to sampling in hip-hop, or borrowing and developing an aesthetic style that you’ve seen elsewhere. For Noakes, however, it points to a growing disregard for human creativity and fair compensation: “Many of the companies building the new AI-powered creative suite don’t prioritize human creativity. If they were transparent about their data sources and could prove that the training sets had been approved by the artists whose styles they implicitly reference – and that these approved artists were fairly paid for their input and each generative output  – then that would be a huge step in the right direction.”

Perhaps think twice before prompting your chosen AI to mine the creative work of others, or at least make sure you understand the legal issues before you do (there’s a whole other article on this to follow). Better still, why not try doing as Herndon and Dryhurst did: build your own AI from scratch and train it on your very own datasets. That way, you know you’re not embedding historic prejudice and biases into your brand new creative work.

Of course you don’t have to engage with AI at all if it doesn’t feel like a valuable tool in your workflow or a worthwhile creative partner. And if you object to your work being included in datasets to train future AI, you can always opt out thanks to this tool, created by Herndon and Dryhurst who, despite their love of AI, are also acutely aware of its dubious ethics.

Engman is against this idea in principle: “You can’t just opt out of life. That’s annoying!” he says. Though he does concede that there’s a very real risk that “we all just become slaves to OpenAI and Meta, where all of our data is just constantly being harvested and monetized and we don’t get any compensation.”

There’s a growing community online with similar concerns, and platforms, such as Cara, have emerged to service creatives and clients who want to avoid AI in their work, or at least make sure that work made using AI can be easily identified. The more people sign up, the more longevity such platforms are likely to have.