The Machine Learned to Repeat Them

Ask a chatbot a loaded question about the war in English, then ask it the same thing in Russian, and you are quietly using two different machines. The model hasn't changed. The ground under it has.

The assistants most people reach for now don't answer a question about this month out of memory. They run a web search while you wait, skim the first handful of results, and hand you the gist in a calm, even voice. That voice is worth whatever it happened to find, and what it finds depends on the language you asked in.

So I went looking for the gap. I took three stories the Russian networks pushed hard over the past two years, stories that have since been checked to pieces: that Zelensky quietly bought two superyachts with Western aid money,¹ that his wife spent over a million dollars at Cartier on a trip to New York,² that the family bought one of King Charles's estates.³ Every one of them is an invention. I searched each twice, once in English and once in Russian, and read what came back the way a machine reads it: from the top down.

In English, the top of the page is the correction. Reuters, Snopes, PolitiFact, a European fact-checker or two. You have to scroll past the debunks to reach the lie, and when you find it, it's sitting on some anonymous video channel nobody would cite. A machine summarising that page tells you the story is false, because false is most of what's on the page.

In Russian the same three searches come back inside out. The debunks are there. Проверено.Медиа is good; StopFake is good. But above them, and around them, and simply more numerous, are the outlets that ran the lie straight. These were not fringe blogs. Some of the largest names in Russian media carried it, and for the King Charles story, three state broadcasters one after another. A machine reading that page from the top has a far better chance of handing you the fake, because the fake is most of what's on the page.

That is nearly the whole of it. The interesting part is why the two pages look so different, and the reason is not the obvious one.

What the machine reads

Start with how the answer gets made, because the fix depends on it.

There are two ways a false story can end up in a chatbot's mouth. The first is that the model absorbed it during training, months ago, and now recites it from memory. That one is hard to do on purpose and hard to prove after the fact; a few thousand junk articles vanish against the billions of pages a model is trained on. When researchers went looking for this network's content inside the big public training set, they found it growing fast but still tiny, and couldn't confirm any model had actually memorised a word of it.⁴

The second way is the one that matters, and it doesn't touch the training at all. When you ask about something recent, the assistant searches the live web right then and reads you what it finds. It doesn't need to have learned the lie. It only needs the lie to be sitting there, near the top, at the moment it looks. Poison the search results and you have poisoned the answer, without ever getting near the model. The good news buried in that is that this kind of poisoning is reversible: a company can strike a domain off the list of things its assistant is allowed to read, and several have started to.⁴ You can't un-teach a model nearly so easily. The measurable trouble today lives in the retrieval, and that is the part we can still clean.

Which raises the obvious question. Who is filling the search results, and why, if almost no human ever reads them.

The library built for no one

In February 2024 the French government's disinformation watchdog published an autopsy of a network it called Portal Kombat.⁵ The shape of it is the point. At the time it was 193 near-identical sites, later more than two hundred, all run off the same template, many named to look like a local news portal, a Paris one or a Warsaw one, dozens more named for Russian towns. They share a hosting fingerprint and produce, between them, no journalism of their own. What they do is copy. They vacuum up posts from Russian state agencies and pro-war Telegram channels and republish them at industrial volume: five of the sites alone put out more than 150,000 articles in under three months. The flagship channel feeding one of them had 119 subscribers.

Sit with that number. A hundred and nineteen. These are not publications in any sense that involves readers. One of the American-facing sites in the network drew, by one measurement, under a thousand visits in a month, with people staying an average of twenty-nine seconds.⁶ The sites are barely usable by a person, with no working search and screen after screen of duplicated text. They were not built for a person. They were built to exist: to be one more result, in one more language, on one more obscure question, on the day a machine goes looking.

An American research group gave the tactic a name last year, "LLM grooming," the mass-production of falsehoods for the specific purpose of shaping what AI systems say, and put the network's output at more than 3.6 million articles a year.⁶ Treat that figure gently: it's an estimate stretched from a small sample by an advocacy group, and the group says so itself. But the direction is not in doubt, and neither is the intent. You do not publish 3.6 million articles for 119 subscribers unless the reader was never meant to be a person.

A third of the time, or one in twenty

Here is where the story usually turns alarming, and where I want to slow down instead of speed up.

In March 2025 NewsGuard, a company that rates the reliability of news sources and sells that service, ran the ten leading chatbots against fifteen of this network's false claims and reported that they repeated them about a third of the time, and that seven of the ten had cited the network's own pages back as sources.⁷ A third. That is the number that went around the world, and it came out of a careful test.

Seven months later, a study in the Harvard Kennedy School's Misinformation Review, run by researchers at Manchester and Bern rather than Harvard itself, reported five percent.⁸ Not a third. One in twenty.

Two careful teams, the same question, a more than sixfold gap. It would be easy to decide one of them is lying or incompetent. Neither is. They asked different questions, and once you see how, the disagreement stops being a scandal and becomes the actual finding.

NewsGuard was pressure-testing. Two of its three prompt styles were built to lead the machine toward the lie or to feed it the lie as a given, and it never published the exact wording, so no one can rerun it.⁹ Worse for the headline, it counted a chatbot that brought up a false claim in order to debunk it as a "repeat" of that claim. The second team used plainer questions, published every one of them, and separated a bot quoting the lie to knock it down from a bot that endorsed it outright. When they did, the network's pages showed up in eight percent of answers, but were used to prop up a falsehood in only one percent.⁸ NewsGuard had been asking whether you can make the thing fail if you try. The second team had been asking whether it fails when you are not trying. Both answers are true at once, and the gap between them is the shape of the risk. It hides in the corners you have to reach for, not across the ordinary questions.

The thin places

Those corners have a name. Researchers call them data voids: the questions where reliable, authoritative material is thin or missing, so that whatever does exist, however junky, wins by default.¹⁰ The idea predates all of this; it was coined to describe how search engines can be gamed on obscure terms. It is exactly what the French investigators found when they tested the propaganda network: invisible on a broad query like "France Ukraine," it surfaced only on the narrow, specific ones. Their read on the whole affair is the quiet one, and I think the right one. The machines reach for this stuff for a duller reason than training: on the narrow question, there is little else within reach.⁸

Now put the language back in. A data void is not a fixed feature of a topic; it's a feature of a topic in a language. And the Russian-language web, for this whole class of subject, is a thinner and dirtier place to be standing. That isn't a slur; it shows up in the numbers. When researchers audited search engines on a false claim in several languages, the share of results that were false ran higher in Russian across every engine, and on Yandex, the one most Russians use, nearly nine in ten of the sources on a Russian-language query traced back to Kremlin-sponsored outlets.¹¹ NewsGuard's own multilingual pass found the chatbots failing most often in Russian, above every other language it checked, though that failure rate counts both repeating a claim and giving no answer at all.¹² I would hold that particular number loosely, since these tests run on models already a version or two out of date by the time you read them, but the underlying fact is steady. The debunk you'd hit instantly in English might be on page two in Russian, or written by a fact-checker three times outnumbered by the outlets that ran the story straight.

Which closes the loop back to my three searches, and to the person I'm actually writing this for. The diaspora reader is the one asking in Russian. Often it's the emotionally loaded, oddly specific question — the scandal about the leader you already distrust, the number that confirms the thing you suspected — and those are precisely the questions where the void is deepest and the seeded content is thickest. The machine isn't lying to you on purpose. It is doing the only thing it does: reading back the room it can see. In your language, that room is stocked differently.

My own test is a search, not a controlled study. The results shift week to week, and a real user on Yandex would see it worse than I did on a Western search index. That looseness is what makes it useful: you can run it yourself, right now, in the two languages you speak, and see the seam instead of taking my word or NewsGuard's or anyone's.

What I'd do

Not stop using the thing. I use it every day, and telling people to go back to reading twelve news sites by hand is advice nobody follows. The useful posture is narrower than that.

Treat the machine as a fast first draft and never the last word, most of all when the answer flatters something you already believe or arrives strangely specific: a sum of money, a name, a scandal you hadn't heard of. That flattery and that specificity are the exact texture of a seeded claim. When an answer has that texture, ask the assistant where it got it; the current ones will show you their sources if you ask. If the answer is standing on three sites you've never heard of, that is itself the answer. And when it matters, ask it twice, once in each language, the way I did. The disagreement between the two replies tells you more than either one alone.

Don't overcorrect into the other error, either, the one that says everything in Russian is a lie. It plainly isn't. The Russian fact-checkers are there and they're good; the trouble is only that you have to want to find them, and the machine, left alone, won't want it for you. The vendors are cleaning the retrieval side, striking the worst domains off the list, and that's real progress — but it's their list, updated on their schedule, and it isn't finished, so it isn't a thing to wait for.

The uncomfortable part is that none of this was an accident, and none of it was aimed at the machine's makers. It was aimed at you. Someone worked out that the cheapest way to put a claim in front of a Russian-speaker in Tallinn or Berlin or Chicago was no longer to buy an ad or run a channel, but to leave the claim lying around in enough places, in the right language, that the helpful assistant on your phone would pick it up and read it back to you in its own reasonable voice, with no fingerprints on it. The machine never learned to lie. What it learned was to repeat, and repeating without a source is simply how it works. Asking it where it heard something is the one part nobody could automate, and it is still on you.