The “Authenticity Collapse”
It’s now what we don’t know, it’s what we do know may be fake
Well, that didn’t take long.
In December 2024 a colleague and I co-authored a piece for USC’s Annenberg Center on the future of AI, which was included in a publication called the Relevance Report. You can check out the full report here.
Our piece focused on the question of how AI might impact polarization in America. We were not optimistic.
Comparing the early days of AI to the formative stages of the Industrial Revolution and the Information Revolution, we predicted that there was likely to be a wave of harmful effects from AI before a general regulatory framework was put in place.
“And so it is with the rise of generative AI, the latest and most profound iteration of the Information Revolution,” we wrote. “For all the benefits it promises, in the early days we’re likely to see as much harm as good from this new technology, particularly in the social realm. To the extent that social media has contributed to polarization and the collapse of trust in society, which has been well-documented, AI has the potential to increase the trend by an order of magnitude.”
“Social media is fueled by algorithms,” we continued. “They feed us what we want to see and hear based on our history, creating a funnel effect that increasingly constricts our world view.
“Generative AI poses a two-fold challenge. It can supercharge the algorithm and greatly amplify the funnel effect. And it brings with it the ability to create fake content that increases engagement and credibility. The rapid spread of the pets-as-dinner rumor that arose in Springfield, Ohio in September (2024) was no doubt fueled by the proliferation of AI-generated memes featuring Donald Trump as the leader of an army of cats.”
All of that and more is now happening, say experts. The dam has broken and the cats are loose.
Alon Yamin, the chief executive of Copyleaks, an AI verification company, calls it a wholesale “authenticity collapse.”
“The internet is lying by default, and the media ecosystem is just flooded with content that you know looks real, sounds real but is definitely not real,” he said. “There is a danger here of almost losing touch with reality.”
Memes of cat armies have now given way to sophisticated and pervasive fake imagery across every major social media platform, according to the New York Times, like this one, which purports to show Ilhan Omar rubbing shoulders with the man who attacked her recently. Social media trollers claimed the two had set up the whole thing.
Fake content is now so pervasive, it’s almost like a buffet of images from which one can select to fit a particular narrative, as Sen. Dick Durbin’s staff did regarding the death of Alex Pretti. It’s a compelling photo, except for the fact that it’s not real. When fake images are lifted from the bowels of the internet to the floor of the U.S. Senate, there’s some truth-washing going on.
Or this, a photo of Nekima Levy Armstrong, who was arrested for disturbing a church service in St. Paul, MN on Sunday as part of the anti-ICE protests. The photo on top was posted by DHS Secretary Krisi Noem; the one below is an altered image posted by the White House. Why? Who knows. Whatever the motivation, the result is an attack on veracity.
This level of AI-aided misinformation, whether verbal or visual, is rising, driving greater levels of polarization to the point where most Americans think that politicians can’t even agree on basic facts, much less policies.
“In moments past, we thought that this online fever would break, and now it is a systemic feature rather than a bug,” Graham Brookie, the senior director of the Digital Forensic Research Lab, told the Times. “This is just how it is right now — we’re all collectively navigating that for the worse.”
It’s hard to say where this is all going. Our thesis in the piece we wrote for USC was that we would gradually lurch toward some type of regulatory regime for AI that would rein in its worst abuses. And AI itself, as it gets better, may evolve into an authenticator of content to counterbalance the slop. In the meantime, the cats are out, nobody can agree on anything, and the fog is getting thicker.
Uh oh, you mean I can’t trust the internet anymore? Snopes is my fallback sometimes. Otherwise, it’s approach everything with a skeptical eye…and that’s not what we all were hoping a higher degree of information and education would be delivered. I remember my old captain once said (well before the internet/AI) “believe half of what you read, and nothing of what you hear.” Now we’re in a new reality.
I really like your “authenticity collapse” framing; it names a feeling many of us have but struggle to articulate. I do wonder, though, about Alon Yamin’s line that “the internet is lying by default.” It works as a warning, yet it overshoots what’s actually happening: most of what we see online is still mundane, human, and not designed to deceive. Hell, I'd argue that most of the stuff on the internet is actually boring! However, we tend to remember the extremes. We also studied this before we had cellphones too.
Why does that matter? Well, I think the essence of this is changing. When we look at the White House sharing an altered arrest photo and brushing it off with a “the memes will continue”, its more telling than a warning.
Nice read on the article. I got me thinking in a good way, as all good articles do!