The European Commission has rewritten the reporting obligations for the world's largest social media platforms after academic researchers demonstrated that the Digital Services Act's core transparency requirement was effectively meaningless. The revision, formalised in November 2024, forces companies to disclose not just a single accuracy figure but two distinct metrics, precision and recall, that together reveal how much illegal content their automated systems actually catch and how often they wrongly remove legitimate speech.
The loophole in the original law
When the Digital Services Act entered into force in 2022, it obliged Very Large Online Platforms, those with more than 45 million average monthly active users in the EU, to report the accuracy and potential error rates of their automated content moderation systems. The legislation did not define what accuracy meant. That omission allowed platforms to choose whichever metric flattered their performance. Because the overwhelming majority of posts are benign, a system that simply labels everything as safe can achieve accuracy above 99 per cent while missing nearly every piece of illegal content.
The problem was not merely theoretical. Platforms process billions of items daily; without a standardised measure, regulators had no way to compare performance across companies or to track improvement over time. The Commission's December 2023 draft implementing act retained the vague accuracy language, prompting concern among specialists that the transparency regime would fail at the first hurdle.
An interdisciplinary intervention
The gap attracted the attention of Robin Jia, an assistant professor of computer science at the University of Southern California, and Frederike Zufall, a legal scholar at the Karlsruhe Institute of Technology. Together with Jia's PhD student Johnny Wei, they began a project in 2022 that combined statutory interpretation with statistical methodology. Their paper, "Operationalizing Content Moderation 'Accuracy' in the Digital Services Act," was accepted to the 2024 AI, Ethics, and Society conference and circulated among Brussels policy circles during the consultation window.
The team argued that accuracy should be split into two measures. Precision captures the share of flagged content that is genuinely illegal, a guard against over-moderation. Recall captures the share of all illegal content that the system finds, a guard against under-moderation. Neither metric alone is sufficient; a platform that removes nothing achieves perfect precision but zero recall, while one that removes everything achieves perfect recall but terrible precision. Only the pair exposes the trade-off.
Measuring recall without reading everything
The practical obstacle to recall reporting has always been scale. To know how much illegal content a system missed, someone must examine the content the system let through. With billions of posts, exhaustive human review is impossible. The researchers developed a stratified sampling technique that uses machine learning classifiers to sort unflagged content into risk strata, then samples disproportionately from the strata most likely to contain violations. Their simulations showed the method yields reliable recall estimates at a fraction of the cost of random sampling, making compliance technically feasible for platforms of any size.
From consultation to regulation
The Commission's public feedback period in early 2024 became the vector for the research to enter law. Two submissions explicitly recommended adding recall as a mandatory metric; one cited the USC paper by name, setting out the legal and statistical case. In November 2024 the Commission adopted Implementing Regulation (EU) 2024/2835, which requires platforms to report accuracy, precision and recall for each category of illegal content their automated systems address. The regulation applies to all Very Large Online Platforms and Very Large Online Search Engines designated under the DSA.
The Brussels Effect in action
Johnny Wei noted that the revision's impact will extend well beyond the EU's borders. Because Meta, X and other major platforms operate unified technical stacks, they typically implement the strictest regulatory standard globally rather than maintain separate moderation pipelines for different jurisdictions. Early signals suggest both companies have begun reporting recall figures worldwide, effectively exporting the European standard to markets with no equivalent legal requirement.
What happens next
The first full reporting cycle under the new metrics will arrive in 2025, giving the Commission and national Digital Services Coordinators their first comparable dataset. The real test will be whether platforms treat recall as a target to improve or a figure to manage. The Commission has signalled it will publish aggregate analyses; if those reveal wide disparities, pressure will grow for binding performance thresholds, a step the DSA does not currently require but which the new data makes politically feasible.
People mentioned
-
Robin Jia
-
Johnny Wei
-
Frederike Zufall
Organisations
European Commission · USC Viterbi School of Engineering · Karlsruhe Institute of Technology · Meta
