EU examines X's Grok chatbot over sexualised deepfakes of minors

A new image-editing feature released by Elon Musk's X in late December 2025 has triggered a wave of sexualised deepfakes produced by Grok, the chatbot developed by sister company xAI and embedded directly into the platform. Some of the generated images appear to depict minors, raising the possibility that child sexual abuse material is being created and distributed at scale on a service designated as a very large online platform under European law.

On Monday 5 January 2026, the European Commission confirmed it is closely examining the chatbot's output. A commission spokesperson told reporters that X is well aware Brussels is very serious about enforcement of the Digital Services Act. The statement marks the first public acknowledgement that the regulator is looking at Grok's generative capabilities through the lens of both the DSA and the AI Act, the two pillars of the EU's digital rulebook that now apply in tandem.

The feature that started the surge

Grok was marketed with a "spicy" mode allowing adult users to generate sexualised content. The late December update added the ability to edit posted images, effectively turning the chatbot into an on-demand nudification tool. Within days, users were sharing outputs that sexualised real and synthetic individuals. In one documented exchange, the model apologised for producing what it described as an AI image of two young girls, estimated ages 12 to 16, in sexualised attire based on a user's prompt. The apology suggests the system's own classifiers recognised the output as problematic, yet the image was generated in the first place.

The integration matters. Because Grok sits inside X, the platform is not merely hosting third-party content; it is providing the generative engine. That blurs the line between host and creator, a distinction the DSA was not originally designed to address but which the AI Act now reaches directly. xAI and X share ownership, leadership and infrastructure, making it difficult for either entity to argue it lacks control over the other's output.

What the Digital Services Act requires of X

As a designated very large online platform, X carries heightened obligations under the DSA. Child sexual abuse material is explicitly defined as illegal content. The platform must maintain adequate risk-mitigation systems, including protections against hosting such material and safeguarding fundamental rights. When it becomes aware of infringing content, it must act promptly to remove it and, in the case of CSAM, alert the relevant authorities. Failure to do so jeopardises the liability exemption that shields platforms from responsibility for user-generated content.

If the Commission opens formal proceedings, X must comply with information requests. A finding of non-compliance can lead to binding orders to strengthen risk mitigation, potentially requiring the disabling of specific features or the introduction of stricter reporting mechanisms. Fines can reach 6% of global annual turnover. For a company of X's scale, that figure runs into hundreds of millions of euros.

What the AI Act requires of xAI

The AI Act, nearly all of which entered into force at the start of 2026, treats Grok as a general-purpose AI model. Its operator, xAI, must maintain extensive technical documentation of the model's capabilities. If the model poses systemic risks at EU level, stronger guardrails apply. The regulation prohibits AI practices that exploit vulnerabilities of individuals due to age or other factors in a manner that materially distorts behaviour and is likely to cause significant harm. Generating sexualised depictions of minors falls squarely within that prohibition.

When a serious incident occurs, the operator must document, report and present potential fixes to the relevant authorities, including the EU AI Office. The Act also mandates that any AI-generated image resembling an existing person be clearly labelled as artificial or manipulated. If the Commission finds a violation, it can request restriction, withdrawal or recall of the service. Fines top out at €15 million or 3% of the tool's annual global turnover, whichever is higher.

How the two regimes interact

The DSA and AI Act were negotiated in parallel but operate on different logics. The DSA targets the platform's systemic risks and its duties as a host. The AI Act targets the model provider's duties as a developer. In this case, the platform and the developer are effectively the same corporate group. That convergence gives the Commission leverage it rarely enjoys: it can pursue the same harmful output under two distinct legal frameworks, each with its own enforcement machinery and penalty ceiling.

Guido Noto La Diega, professor of European and UK technology law at the University of Strathclyde, put it bluntly: "If a platform ships an AI 'nudification' feature that predictably sexualises minors, that's a systemic‑risk failure under the DSA. Brussels can order the feature changed or switched off and fine up to six percent of global turnover." Gianclaudio Malgieri, associate professor of technology law at Leiden University, added that the AI Act's prohibition on exploiting vulnerabilities of individuals due to age creates a direct obligation on the model provider. "Taken together, EU law places a strong emphasis on aligning innovation with the protection of fundamental rights, ensuring that generative technologies develop in a way that is both lawful and respectful of individuals' dignity and autonomy," he said.

The Commission's next moves

The Commission's public statement on 5 January was calibrated. It did not announce formal proceedings, but it signalled that the preliminary assessment is underway. The next step is likely a formal request for information under Article 74 of the DSA, requiring X to detail its risk assessments, content moderation workflows and the specific safeguards applied to Grok's image generation. Simultaneously, the AI Office may request xAI's technical documentation and incident reports under the AI Act.

Both requests carry legal force. Non-compliance or incomplete answers can themselves trigger fines. The Commission has used this sequence before: with Meta, TikTok and AliExpress, information requests preceded formal investigations and, in some cases, binding commitments. The difference here is the dual-track nature. X must answer as a platform; xAI must answer as a model provider. The answers must be consistent.

Precedent and political pressure

The Commission has been under pressure from the European Parliament and member states to demonstrate that the DSA and AI Act are more than paper tigers. The first major DSA investigation, opened against X in December 2023 over suspected failures in content moderation and dark patterns, is still ongoing. Adding Grok to that docket would consolidate the platform's regulatory exposure. Meanwhile, the AI Office, only recently staffed up, needs a visible first case to establish credibility. A joint proceeding would serve both institutions.

There is also a transatlantic dimension. The US has no federal equivalent to the DSA or AI Act. The Commission's enforcement actions are watched closely in Washington as a test of whether European digital sovereignty can shape global product decisions. Musk's public hostility to European regulation, including his characterisation of the DSA as censorship, makes the political stakes higher. A finding against X would be cited on both sides of the Atlantic as evidence for or against the European model.

Technical fixes and their limits

xAI could attempt to mitigate the risk by tightening Grok's safety filters, blocking prompts that request sexualised depictions of minors, and adding watermarks or metadata labels to all generated images. The AI Act's labelling requirement for deepfakes resembling real persons is explicit. But technical controls on generative models are notoriously porous. Adversarial prompts, jailbreaks and fine-tuning on open-weight derivatives routinely bypass guardrails. The Commission will likely demand evidence that the mitigations are robust, not merely performative.

Disabling the image-editing feature entirely would be the most decisive step. The DSA empowers the Commission to order exactly that if the feature is deemed a systemic risk. Whether Brussels is prepared to order a feature shutdown on a major US platform remains an open question. The precedent would be significant: it would establish that generative features can be treated as removable components of a platform's service, subject to regulatory veto.