How to unlearn an Atomic Bomb
The technical and legal challenges to perform an AI lobotomy.
To paraphrase a very good Mitch Hedberg joke, “A TV commercial told me, ‘Forget everything you know about slipcovers,’ so I did. Then it tried to sell me slipcovers. I didn’t know what the hell they were.” We covered the Proof News / WIRED investigation about stolen YouTube data being used to train AI, and since then, there’s been one line from the investigation that has stuck with me since reading it: “Once ingested by AI, data cannot be unlearned.”
It’s a scary thought.I think that fear is borne as much out of our continued use of AI = brain metaphors as it is anything else. What does that mean for an artist’s copyrighted work, a business’ confidential trade secrets, or even a private citizen’s personal information? What about military secrets, up to and including the workings of our nuclear program? While there are legal consequences for obtaining such “forbidden knowledge,” if a human learns something they shouldn’t know, there’s no undo mechanism for forgetting. Is that necessarily true for AI models, or is that just convenient hand waving that protects model owners from having to roll back potentially unethical or illegal training?
Let’s explore that. To train an AI to “forget” something requires two key forces:
The technology to pull off the AI lobotomy, and
The legal will to operationalize and enforce such a practice
The latter we’re currently seeing play out on multiple fronts in the US. This week, Condé Nast became the latest to push back against this AI gaslighting. According to The Information, the media conglomerate has sent a cease-and-desist letter to Perplexity—which somewhat famously plagiarized a WIRED investigation about Perplexity’s accusations of plagiarism (WIRED is a Condé property, for context). This is one of the biggest companies to begin legal proceedings against AI companies following The New York Times’ ongoing copyright infringement suit filed against OpenAI and Microsoft. And these are just two examples of many, many companies that are trying to balance whether to fight back against AI or to accept some level of assimilation and strike up licensing deals—an all-too tempting offer at a time when the media business is struggling. (As one media executive recently quipped to me, “at least we’d be getting paid for what the AI companies are probably going to steal anyway.”)
But we’re also seeing this play out in our legislature. Earlier this month, a bipartisan group of Senators introduced the COPIED Act, which stands for Content Origin Protection and Integrity from Edited and Deepfaked Media Act (sidenote for non-US readers: we as a nation love a law built on an acronym). The bill is designed to protect creatives from their work being stolen by AI models. As reported by The Verge:
Content owners, including broadcasters, artists, and newspapers, could sue companies they believe used their materials without permission or tampered with authentication markers. State attorneys general and the Federal Trade Commission could also enforce the bill, which its backers say prohibits anyone from “removing, disabling, or tampering with content provenance information” outside of an exception for some security research purposes.
It’s a necessary step to make clear existing laws that don’t cleanly protect copyright material—which in many ways is what The New York Times lawsuit is aiming to resolve, as its challenges are rooted in copyright law that’s over two centuries old. But even if it were to pass in some form or another, enforcement will still be tricky and not without faults—the Digital Millennium Copyright Act (DMCA), for example, has been weaponized to cause takedowns on material that otherwise shouldn’t be. Still, having the COPIED Act or an equivalent bill that provides some protection is certainly better than what we have right now.
But let’s play out the hypothetical of what if the content owners were to win their case? I’ve seen far too many people say that the only way to truly get rid of copyright materials is to scrap a model and start over from scratch. Tempting though that may be, such thinking leads to a slippery slope that creates inertia and gives those infringing companies added ground to say, “okay, so maybe some training material was used that shouldn’t have been, but is it fair to make us scrap everything and start over?” And it certainly wouldn’t be a one time re-do, as laws around this are sure to evolve—as FT reports, Japan is contending with its own dated copyright laws that has attracted many AI companies keen on gobbling up as much information as it legally can, geography be damned.
Teaching an AI to forget something is admittedly not a clean process, with many proposals I’ve read lately falling somewhere between AI hypnosis and an AI lobotomy. (For example, researchers at The University of Texas at Austin and JPMorgan Chase recently proposed a method that would block specific imagery.) Axios rounded up several such “machine unlearning” initiatives earlier this year, each with their own set of problems. My favorite? Microsoft researchers attempting to teach the Meta-trained Llama 2 model to forget Harry Potter, which was initially seen as successful until an audit discovered that rewording the prompts would get the model to reveal it still “knew” some things about the Wizarding World.
And that latter point leads to a rather philosophical question that has to be contended with in parallel to the technical challenges: what knowledge is truly unique and copyrightable, and what knowledge is fair to consider universal? Teach an AI to “forget” Harry Potter, sure, but wizard schools, secret societies, and ne’er-do-wells with odd grudges against superpowered children are all tropes that even someone who hasn’t read those specific books or movies are likely to know about. Like a human, is it enough to simply tell them to “not” use the infringing knowledge, bound by legal ramifications for breaking that vow? I think users are far too clever here: if an AI model has knowledge, it’s fair to assume that someone will engineer the right prompt to retrieve that information.
This isn’t just about copyrightable material to protect creatives. It’s also about trade secrets—as Samsung found out last year when employees gave ChatGPT source code while trying to fix a bug. It’s also about the right to privacy, should an AI model accidentally learn private, identifiable details about individuals that it can share with the right prompt. It’s about sensitive military secrets, the kind that already get leaked on mobile game forums, and other violent, illegal, and dangerous content.
As J. Robert Oppenheimer once said of the atomic bomb program, “the physicists have known sin; and this is a knowledge which they cannot lose.” Granted, he was speaking in metaphors, but I think there is something very literal to take in account here as well. All of these things, taken together, make it clear that we must teach the AI to do something extraordinary, something superhuman: it needs to learn how to forget.