Intro to LLM Grooming
I am a ferocious reader — I tear through a hundred books a year, albeit across a fairly narrow range of topics (tech policy, international affairs, history, espionage, thrillers). Large Language Models (LLMs) like ChatGPT or Claude put me (or anyone) to shame. They don’t just read widely — they “read” widely AND deeply. LLMs ingest everything on the internet, detecting patterns across billions of words.
So, think of my home library as the training data used to inform my thinking, analysis, and assessment of the world around me. (Of course, there’s also the radio, streaming services, news, and so forth, but let’s keep the analogy simple.)
Now, imagine someone (perhaps a character from one of my spy thrillers!) sneaking biased books with fake or retouched covers into my home library — carefully planted so I unconsciously absorb their slant.
That’s essentially what LLM grooming is: bad actors deliberately flooding or manipulating the training data environment to shape how AI systems learn, especially about sensitive or contested topics, and what the models provide when prompted.
When GPTs or other models perform real-time web searches in response to prompts, they often draw on the very content that bad actors have seeded online — low-quality articles and agenda-driven sites that lead to misleading or misinformed AI generated summaries.
But That’s Just Propaganda, Right?
At first blush, it might sound like just another form of propaganda — and in some ways, it is. Governments have long seeded stories through pliant journalists or fake newspapers and magazines. Just think about America’s Cold War anticommunist crusade and disinformation campaigns that bolstered regimes in Indonesia and across Latin America. Even today, PR firms regularly whitewash the misdeeds of governments or corporations through “strategic communications” – meaning, propaganda.
But LLM grooming operates on a different layer. Rather than at the level of public persuasion, it works beneath the surface, targeting the neural infrastructure that underpins how AI understands the world and embeds itself there. And beyond the training data per se, your GPT results may also be logged, rated, and recycled into future improvements.
In this way, grooming creates a feedback loop: engineered content influences AI responses, which are then reinforced and amplified across the system.
In a recent paper titled “LLM Grooming: A New Cognitive Threat to Generative AI,” researcher Didier Danet defines this emerging threat as:
the large-scale contamination of training data with biased or deceptive content, thereby transforming generative AI systems into powerful vectors of disinformation.”
Because LLMs don’t distinguish truth from falsehood — they treat most data as valid unless explicitly told not to — they are especially vulnerable to subtle distortions. Such LLM grooming influences how the models answer our queries, which facts and narratives are prioritised, and how contested issues get framed.
As researchers put it just last week in Dialogues on Digital Society:
“Unlike traditional propaganda, tainted datasets embed lasting distortions in digital memory, subtly reshaping how societies understand history and truth.”
They shared a further warning:
“This technological evolution reveals how modern information infrastructures are being weaponised to fragment shared reality — and to weaken the potential for political resistance.”
According to the latest report on Foreign Information Manipulation and Interference Threats from the European External Action Service, last year saw upwards of 90 countries targeted by foreign state-sponsored efforts to manipulate information. It further discusses how Generative AI accelerates threat actors’ ability to create inauthentic content and increase the scale of their activities, while exacerbating efforts to thwart the polluting effect of false information in our information ecosystem.
But I Can Smell Info-Ops a Mile Away!
And now, from Russia to Iran, disinformation campaigns targeting AI systems are no longer hypothetical; these information operations are happening now.
Russia, for example, has pursued a strategy of saturation over precision that is cheap, scalable, and fast. They primarily aim to inundate the web with large volumes of low-quality but consistent disinformation so that LLMs begin to treat those narratives as plausible, common, or legitimate.
This approach seeks to influence outputs from GPT-style models, pollute the web to poison both training data and real-time web search, while manipulating the overall information environment.
By contrast, more sophisticated forms of LLM grooming aim to quietly embed a preferred bias within the AI ecosystem by producing content that appears polished, legitimate, and algorithmically credible. These strategies go beyond sheer volume: they leverage traditional media networks, influencer campaigns, SEO optimization, and even diplomatic framing to present themselves as respectable public relations.
Here, the goal is longer-term: to normalise select narratives within the AI’s training and retrieval layers, shape default outputs, and subtly crowd out competing perspectives — all while minimizing the appearance of propaganda. When these efforts are strategically distributed across mainstream media, social platforms, news portals, and web infrastructure, they blur the line between influence and information, making detection more difficult and their impact far more persistent. The aim isn’t just to sway opinion in the moment, but to introduce lasting distortions into the neural memory of AI systems themselves.
These more sophisticated info-ops aimed at grooming LLMs are much more difficult to smell from a mile away.
To better understand and appreciate the broader, real-world risks and implications of this more refined approach to LLM grooming, we need to consider a case study. And perhaps the most telling, and currently most geopolitically significant, example comes from an avowed democracy — in the midst of one of the deadliest and most politically polarising and violently contested conflicts in recent history, a conflict many now consider a genocide.
Case Study: Israel and Clock Tower X
—> Background
In September, investigative journalists reported that the Germany division of Havas Media Network, acting on behalf of the Israeli government, had signed a US$6 million contract with an American company Clock Tower X, an spin firm owned by Brad Parscale (former Trump 2.0 campaign manager – and the same one who hired Cambridge Analytica for Trump 1.0 back in 2016). The brief is to “combat antisemitism” in the United States. Parscale registered last week as a foreign agent for Israel under U.S. law.
Setting aside ongoing debates over Israel’s use of antisemitism allegations to deflect or delegitimise criticism of its behaviours, there’s something more revealing in the contract they’ve taken out with Parscale’s shop: Clock Tower X would create “websites and content to deliver GPT framing results on GPT conversations.”
Translation: content directly engineered not for you or me — but for LLMs. To shape the kind of material these models scrape, absorb, and surface in future interactions – to ultimately influence the “framing” or narrative of GPT conversations. This goes well beyond propaganda to win over target audiences through PR or stratcomms. It enters the realm of recalibrating the AI models themselves to distort what people perceive as reality.
Details from the contract attached to the FARA (Foreign Agents Registration Act) disclosures further reveals that the broader information operation strategy also includes:
deploying advanced SEO tools like MarketBrew AI to ensure content created on behalf of Israel ranks highly in search results;
tailoring 80% of messaging to Gen Z audiences across TikTok, YouTube, and podcasts;
blurring traditional media, influence networks, and PR by integrating with outlets like Salem Media Group (of which Parscale is the new chief strategy officer); and
paying influencers per post to amplify content.
Though publicly framed as a PR campaign to combat antisemitism, the deeper aim, visible through both the tools used and the structure of the operation, is to embed specific narratives preferred by Israel into the digital and algorithmic infrastructure — subtly, persistently, and with long-term effect. This is what we will focus on here.
Thus, the information operations to be undertaken by Control Tower X isn’t your run-of-the-mill hearts-and-minds public relations campaign. Rather, it is aimed to tilt the AI systems that shape how people experience facts and knowledge in the future.
What emerges is a coordinated effort: shape the content → shape the algorithm → shape the AI → shape the reality.
But it’s just one small contract. What harm can that really cause?
—> Context
Everything happens in a context and things are defined by their relationships.
Let’s consider the broader context of this case study. Authoritative global figures and organisations with expertise in war crimes have concluded that Israel is engaged in genocide against the Palestinians of Gaza. Israel denies this.
A plan recently developed and unveiled by the United States and endorsed by Israel for a Gaza International Transitional Authority (GITA) would have billionaires on a board headed by US President Donald Trump and would also include former UK prime minister Tony Blair (with his Tony Blair Institute for Global Change, or TBI), with a focus on “generating investment returns.” Israeli Finance Minister Bezalel Smotrich recently said that the Gaza Strip could be a “real estate bonanza,” adding that the “demolition phase has already been completed, the first stage in the city’s renewal.” (Note: GITA is a developing news story.)
Meanwhile, last week while in New York, Prime Minister Netanyahu told a roundtable of American supporters gathered at the Israeli consulate that social media platforms are the most important new tools of warfare. (Note: As of November 2024 there is an International Criminal Court warrant for Netanyahu’s arrest for war crimes. The United States has not acted on that warrant and allowed him to enter the United States to attend UN General Assembly (UNGA) and other activities. The United States barred the Palestinian President Mahmoud Abbas from entering the U.S. to attend UNGA.)
The New York Times uncovered efforts in regard to leveraging social media a few months ago, reporting that Israeli officials authorised the creation and use of hundreds of fake social media accounts and fake English-language news sites, while employing ChatGPT to create believable, content to target US lawmakers that would encourage them to fund Israel’s military. (Meta and OpenAI said they had found this network and disrupted the operations on the basis of coordinated inauthentic behaviour.) Reflecting on these revelations, POLITICO noted that this “approach would track with other state-sponsored disinformation attacks out of Iran, Russia, North Korea and China, which often involve the use of fake accounts on social media to spread narratives favorable to those governments. The disinformation campaign underscores the concerns about psy-ops muddying political discourse worldwide…”
Late last month it also emerged that billionaire Oracle co-founder Larry Ellison – a Trump loyalist who briefly surpassed Elon Musk recently as the richest person in the world – leads an American investor group acquiring TikTok’s operations in the United States. The platform had for some time been criticised for an alleged pro-Palestinian slant, blamed for causing the significantly weakened support for Israel among young Americans (Gen Z).
An American citizen billionaire, Ellison may be the largest private donor to the Israeli Defense Forces and is a close friend of PM Benjamin Netanyahu (including having hosted him at his private island in Hawaii). Oracle executives have repeatedly traveled to Israel and expressed the company’s commitment to the country, notwithstanding global concerns over its current war in Gaza.
Oracle will now oversee Tiktok’s algorithmic operations in the United States while hosting all data on its domestic cloud infrastructure.
Consider the roundtable mentioned above, in which Netanyahu told supporters that social media platforms are the new tools of war. If you watched the video link you’ll have heard him also say that the most consequential platforms are Tiktok and Twitter/X, expressing hope that the sale of Tiktok goes through (he didn’t mention his friend Ellison but he did say praise Elon Musk, owner of X, as a great friend of Israel).
An investigative report by the New Statesman and Lighthouse Reports shined a light on the close relationship between Ellison’s Oracle and Blair’s TBI – the same Tony Blair who Trump wants to install in Gaza. The report noted that “Oracle’s and TBI’s connections aren’t just rhetorical. By 2023, joint retreats had become commonplace.” One staffer noted that “it’s hard to get across just how deeply connected the two [organisations] are. The meetings were like they’re part of the same organisation.” Larry Ellison has given or pledged more than $350 million to the Tony Blair Institute for Global Change.
Tony Blair and TBI may have major roles to play in shaping Gaza’s future. And therefore Ellison, too.
At the ecosystem level, Larry Ellison’s son, David, now heads the newly merged Paramount-Skydance media empire (approved on 24 July 2025 under the Trump Administration) that includes studios, streaming, CBS broadcasting network, and much else within its expanding online and offline portfolio.
According to investigative reporters at Dropsite News:
“[David] is moving to take over large swaths of the media, including CBS News, CNN, Warner Brothers, and Paramount, reportedly bringing in the Free Press’s Bari Weiss to shape editorial direction.
The Ellison family is cornering the market on attention and data the same way the Vanderbilts did railroads and the Rockefellers did oil,” as Wired recently characterized it. How they plan to operate that monopoly is on course to be tested out in what President Donald Trump is calling “New Gaza,” the techno-dystopian free trade zone that is to be administered by a Board of Peace led by Trump and Ellison’s longtime political and business vehicle, Tony Blair. Ellison has given or pledged more than $350 million to the Tony Blair Institute, which Blair has used to advance Ellison’s vision of a marriage between government, corporate power, and tech surveillance.”
—> The Broader Picture
What emerges here is a web of political leaders, billionaires, media moguls, and tech barons with aligned interests along with the power to pursue those interests. Through their pursuits, they all possess the ability to shape key elements of the information ecosystem.
To be clear, while there’s no evidence of concerted action here, the picture that emerges from this overall context is nonetheless revealing. It shows how LLM grooming operates within — and is amplified by — a much broader ecosystem of influence. Ultimately, much of the content created across this media landscape — from Clock Tower X’s targeted websites to TikTok’s algorithmic feeds, from CBS News editorial lines to fringe blogs to Salem Media Group properties — flows into the same vast digital sea that trains LLMs.
Contracts like the one taken out with Control Tower X are foreign state-sponsored weaponising of our information infrastructures specifically in order to fragment reality — and, as the researchers above noted, weaken the potential for political resistance.
Those who shape that environment — politically, financially, and ideologically — often have a vested interest in shaping not just the present narrative, but controlling the historical record itself.
In this case, that includes how future generations will come to understand Gaza and Palestine — not only since October 2023, but even much further back. But this approach could also very much apply — with alignment of political, financial, and technological power — to how people perceive and remember events in Ukraine, South Sudan, Yemen, Myanmar, Central African Republic and other contested regions in our contemporary world.
—> This Impacts Us All
Regardless of where one stands politically or ideologically on the question of Palestine and Israel, the implications of polluting the information ecosystem and deliberately tilting AI models must concern us all.
When state or private actors working alone or in concert inject biased, misleading, or manipulative content into the digital environments specifically to feed our LLMs, they may think they are simply winning a narrative war. In fact, they are seriously compromising our collective ability to access a genuine, 360-degree understanding of events.
The danger isn’t just that one worldview dominates, but that trust in shared facts erodes, first partially and then entirely. And when democratic governments engage in information operations that target the neural foundations of our AI systems — the very tools millions or billions now rely on to understand the world — they risk undermining the very democratic principles they claim to defend.
The Lessons of LLM Grooming
Policymakers must remain sharply attuned to the dangerous convergence of technology, political power, and concentrated wealth. In the case of LLM grooming — whether through the case study above or in other contexts related to ongoing efforts by Russia and other bad actors — it’s essential to situate these tactics within their broader social, geopolitical, economic, and security contexts.
Only then can we grasp not just the immediate, first-order effects, but also the more profound second- and third-order consequences of this grooming of training data that is quietly fed into today’s AI models and generative AI systems. Including:
the subtle shifting of the information ecosystems;
the hardening of synthetic narratives; and
the long-term shaping of political consciousness and future political realities.
For nations in the Global Majority, the stakes are even more complex. When AI systems are trained primarily on American English content — often sourced from a narrow band of geopolitical, ideological, and historical perspectives — it poses risks not only to data sovereignty or AI sovereignty, but to what we might call historical sovereignty: the right of communities and nations to their histories, told and remembered accurately and truthfully.
More broadly, historical sovereignty speaks to resisting erasure, distortion, and digital colonisation of the historical record. Sans vigilance, synthetic narratives meant to reframe the past as much as the present — shaped by the powerful — risk hardening, and reinforced and universalised by the machines of the future.
To prevent these harms we need a new generation of information industrial policies — policy frameworks that treat the information environment as critical infrastructure, not just a marketplace of content where anything goes. Last month, the Forum on Information and Democracy convened a high-level panel of eleven leading economists — including Nobel laureates Daron Acemoğlu and Joseph Stiglitz — to sound the alarm. Their open letter calls on governments to proactively shape information markets to “foster independent, pluralistic, and reliable information sources.” Their message is worth heeding.
Such an information industrial policy approach could address LLM grooming by investing in high-integrity public knowledge production, regulating algorithmic content curation, and setting global standards for data provenance and diversity in AI training sets. Crucially, it could also ensure that historical narratives from the Global Majority — too often underrepresented, mistranslated, or omitted altogether — are not just preserved, but empowered to inform the models that will increasingly mediate our collective understanding of the past, present, and future.
It is worth taking note of efforts like those of Reporters without Borders. Its Journalism Trust Initiative can serve as a global, neutral standard for trustworthy media that could be used to train AI models.
Global Majority Leaders, Please Take Note
Last week, I wrote in these pages about Indonesia’s first president, Soekarno, and his efforts to forge a path for newly independent nations of the “Third World” to break free from the suffocating binaries of the Cold War. What I didn’t address in that piece were the extensive disinformation campaigns orchestrated by the CIA and its allies to counter such movements. From Indonesia to Chile, Brazil to Guatemala, the United States worked clandestinely and aggressively to shape local media narratives, developing and stoking fears of an imagined communist threat that would spur many to deadly action while encouraging others to disengage politically.
These campaigns had degrees of success at the time. And they did leave deep, lingering effects. In many countries, especially among militaries that carried out mass killings of alleged communists or those believed to be too sympathetic to leftist causes, this manufactured narrative became a kind of retroactive justification. And has stigmatised millions for life.
The propaganda shaped how history was written, remembered, and revered.
Nuri Hanafi, the young daughter of Indonesia’s Ambassador to Cuba in 1965 and now living in France, reflected on the 1965 extermination campaign of communists in Indonesia – events shaped, fueled and remembered in part by American propaganda that vilified leftists, especially members of the non-violent Indonesian Communist Party.
She said:
“When I talk with young people from Indonesia now, I realise we don’t have the same history. I don’t mean that we have different personal stories. I mean they don’t even know the truth of what our country used to be – our struggle for independence, and the values we held.” (See Footnote 1)
She was speaking to the history and memory shaped by the propaganda, the information operations, of old.
Today’s challenges go well beyond what Hanafi and Indonesians experience in 1965. Its now about much more than just buying out corrupt journalists or tricking wire services, whispering some false information at the bar of the Foreign Correspondents Club, or planting fake stories to be picked up and published. And later remembered.
Today, when powerful actors with ideological agendas gain control over media, platforms, and data pipelines, we need to become very concerned. Policymakers must think not just about immediate risks to their citizens, but the future risks to national memory – to historical sovereignty.
This is not a fire drill, folks.
Left unchecked, powerful actors can pollute our information environment in two compounding ways:
Algorithmic mediation bias
Algorithms (in search engines, social media, video feeds) prioritize what’s already dominant. So when one worldview floods the content ecosystem, those views become algorithmically “neutral.” Alternate perspectives are de-ranked, drowned out, or flagged as fringe.Training data capture & model tilt
LLMs like ChatGPT are trained on massive web data. If that data is skewed, the model’s priors (its default statistical assumptions about the world) become skewed too. That means it’s more likely to (a) frame events through a preferred lens, (b) prioritize certain “facts” over others, and (c) dismiss alternative narratives as implausible.
Remember, unlike humans who “look up facts,” LLMs are plausibility engines that generate the plausible next word or phrase based on patterns in the data they’ve seen. They are not fact-checkers. They generate what sounds right.