Anthropic to watermark Claude output under EU AI Act deadline

Anthropic has confirmed it will modify how its Claude chatbot generates text so that every response carries a statistical watermark, a move driven by the EU AI Act requirement that all AI-generated content be marked from December 2026. The company says the alteration operates at the granular, stochastic layer where the model chooses between near-synonyms, "grey" versus "overcast", "stream" versus "brook", and that the resulting pattern will be invisible to readers but detectable by Anthropic and any party holding the decoding key.

How the watermark works

Large language models do not deliberate over word choice the way a human writer does. At each step they sample from a probability distribution over the vocabulary, injecting controlled randomness so that the same prompt can yield varied outputs and, crucially, so that the model does not fall into repetitive loops. Anthropic's watermark replaces the purely random component of that sampling with a pseudorandom sequence that carries a statistical signature. The distribution of words stays the same; only the hidden pattern of which specific low-probability tokens appear changes.

Steven Murdoch, professor of computer science at University College London, described the effect as negligible. "There's going to be no noticeable difference. There's the same random number generators there, it just used to be completely random, and now it's statistically predictable, but still random," he said. In his view, the watermark simply swaps one source of randomness for another without shifting the model's overall behaviour.

The criticism: constrained freedom, worse prose

John Gruber, a veteran technology commentator writing on Daring Fireball, took a sharply different view. "This entire endeavour is a perverse adulteration of what it means to write," he wrote. His argument rests on the premise that the model's freedom to pick the single highest-probability token at each step is what produces the best prose. By forcing the sampler to follow a predetermined pseudorandom path, the watermark may prevent the model from selecting the word that a human editor would judge most precise, even if the difference is subtle.

Gruber concedes the output may remain factually accurate but insists the literary quality will suffer. He frames the change as a degradation of the model's expressive range, a constraint imposed not by the model's architecture but by regulatory compliance. Anthropic has not published side-by-side comparisons to test the claim.

Why randomness exists in the first place

The debate illuminates a widely misunderstood aspect of how large language models operate. The stochastic element is not a bug; it is a design necessity. Without it, a deterministic sampler would always choose the top-ranked token, causing the model to repeat phrases, enter infinite loops, and produce brittle, robotic text. Temperature settings, typically between 0.7 and 1.0 for creative tasks, control the degree of randomness. Anthropic's watermark effectively fixes the random seed in a way that is cryptographically structured but statistically indistinguishable from the noise it replaces.

Murdoch emphasised this point. "LLMs already do not make the best choices," he said. "There's already randomness involved in any of these large language models. It's pretty essential to how they work. If it wasn't for this randomness, then they'd get stuck in loops and start repeating the same thing over and over again." The watermark does not remove randomness; it only makes it auditable.

A cautionary tale from academic publishing

The risks of unmonitored AI text surfaced recently when a paper in a leading chemistry journal was retracted after reviewers discovered a hallucinated phrase. The authors had apparently used an AI tool to draft a section describing a zinc nanogel. The model offered "mass killing of an ethnic group" as a synonym for "final solution", a term that in chemistry refers to the last step of a synthesis but in history denotes the Holocaust. The incident underscores that models do not understand context; they predict plausible token sequences based on training correlations.

Watermarking would not have prevented the hallucination, but it would have flagged the text as machine-generated, potentially prompting earlier scrutiny. The episode also illustrates why regulators want traceability: when AI output enters the scholarly record, legal contracts, or student essays, the ability to verify provenance becomes a public interest.

The regulatory deadline and its reach

The EU AI Act, formally Regulation (EU) 2024/1689, enters full application in stages. Article 50 requires providers of general-purpose AI models to ensure their outputs are marked in a machine-readable format and detectable by the provider. The obligation applies to any company offering such models in the EU market, regardless of where the company is headquartered. Anthropic's announcement signals that the December deadline is being taken seriously by at least one major US-based provider.

Other model operators, OpenAI, Google, Meta, Mistral, have not yet detailed their compliance approaches. Technical standards for watermarking are still being developed by European standardisation bodies under a mandate from the Commission. The lack of a harmonised method means each provider may adopt a different scheme, complicating detection for end users who would need multiple keys or tools.

Model collapse: the hidden motive

Beyond detection, Murdoch highlighted a second rationale for watermarking that receives less attention. The internet is rapidly filling with AI-generated text. When new models are trained on data that includes synthetic output, they ingest their own distortions, a feedback loop researchers call model collapse. Concepts blur, rare events disappear, and the model's internal representation of reality degrades. Watermarks allow training pipelines to filter out synthetic content, preserving the integrity of future models.

"There's so much of it already out there that it might damage the models themselves," Murdoch said. "Training AI on AI-written content creates model collapse, leading models to confuse concepts." In this light, watermarking is not merely a compliance tool but a preservation mechanism for the technology itself.

What happens next

Anthropic plans to roll out the watermarked sampler across its Claude models before the December deadline. The company has not said whether it will release the detection key to third-party verifiers or retain exclusive control. Meanwhile, the European Commission's AI Office is expected to publish guidance on technical specifications and interoperability. The first real-world test will come when universities and publishers deploy detection tools against student submissions and manuscript drafts. If Gruber's quality concerns materialise, pressure may build for a technical standard that preserves both detectability and linguistic freedom, a balance the current regulation does not specify.