The Grok Undressing
In December 2025, xAI turned its chatbot into the largest deepfake generator on Earth. The numbers aren’t close.
In December 2025, xAI updated Grok’s image generation feature — an AI model called Aurora — to allow editing of any image posted on X. Within weeks, Grok was producing an estimated 190 sexualized photorealistic images per minute. That’s 84 times more than the top five dedicated deepfake websites combined.
The Center for Countering Digital Hate analyzed a sample of 20,000 images drawn from a pool of 4.6 million total. From that analysis, they estimated approximately 3 million sexualized photorealistic images and 23,338 sexualized images of children.
Three million images. Not from a dark web forum. Not from a Telegram channel that takes effort to find. From a feature built into the third-largest social media platform in the United States, accessible to any user with a subscription.
Previous deepfake crises involved bad actors misusing neutral tools. This is different. This is a platform company shipping a product that structurally enables sexual abuse at industrial scale. The distinction matters, and the world noticed.
The global response
Indonesia was first. On January 10, 2026, the government blocked access to Grok’s image generation capabilities. The Philippines followed five days later, restricting access from January 15 through January 21 while it assessed the scope of the problem. Malaysia imposed its own restrictions.
Inside the United States, the response came from the states. California Attorney General Rob Bonta announced a formal investigation on January 14, 2026. Then 35 state attorneys general — representing both parties, from deep-red to deep-blue states — sent a joint letter to xAI demanding answers about Aurora’s safeguards and the company’s knowledge of how the tool was being used.
On January 23, 2026, a class action lawsuit was filed: Jane Doe v. xAI Corp., Case No. 5:26-cv-00772, in the Northern District of California. The plaintiffs are women and minors whose likenesses were used without consent to generate sexualized deepfakes through Grok.
Then Europe moved. On February 3, 2026, French prosecutors and Europol raided X’s Paris offices. Elon Musk and former X CEO Linda Yaccarino were summoned for hearings scheduled in April. The French investigation covers complicity in possessing and spreading pornographic images of minors, sexually explicit deepfakes, and denial of crimes against humanity.
Two weeks later, on February 17, 2026, Ireland’s Data Protection Commission opened a formal GDPR inquiry into X and Grok’s image processing. The UK’s Information Commissioner’s Office opened its own investigation in the same period.
Six countries. Thirty-five U.S. attorneys general. A class action. A criminal raid. All within two months of a single product update.
Platform feature versus criminal tool
Every previous deepfake scandal followed the same pattern: criminals used freely available AI tools to create non-consensual intimate images, and platforms scrambled to take them down after the fact. The tools were external. The platforms were intermediaries. The harm was a byproduct of misuse.
Grok broke that pattern. The tool wasn’t external — it was the platform. Aurora was integrated directly into X’s interface, designed to edit images already on the platform, marketed as a feature, and sold as part of a paid subscription. Users didn’t need to download a separate application, find an underground service, or possess any technical knowledge. They pointed Grok at a photo on X and told it what to generate.
This is the difference between a knife manufacturer and a stabbing service. xAI didn’t make a tool that could theoretically be misused. It shipped a product with a predictable primary use case and no effective safeguards against it.
The 84x multiplier tells the story. The five largest dedicated deepfake websites — purpose-built by criminals for this exact abuse — were producing a combined output that Grok surpassed by nearly two orders of magnitude. Not because xAI intended to build a deepfake factory. Because when you give 600 million users a one-click tool to undress anyone on a social media platform, the outcome is mathematically inevitable. Intent is irrelevant when the architecture guarantees the result.
The enforcement gap
After the CCDH study went public and the lawsuits began, xAI issued public pledges to address the problem. The company said it would implement safeguards, restrict certain types of image generation, and invest in detection systems.
CBS decided to test those pledges. Three weeks after xAI’s public commitment to fix the problem, CBS reporters used Grok to undress people in photographs. It still worked. It took seconds.
This is the core of what makes Grok different from a moderation failure. Moderation failures are about content slipping through filters — volume outpacing enforcement. What CBS demonstrated wasn’t a filtering problem. The feature worked exactly as designed. The “safeguards” either didn’t exist or were trivially easy to bypass, weeks after the company publicly promised they would be in place.
The pattern — public pledge followed by no meaningful change — is familiar from other platform crises. But in those cases, the companies could plausibly argue that the problem was hard: billions of posts, adversarial users, edge cases. xAI can’t make that argument. The problem isn’t detecting abuse among billions of benign uses. The problem is that the feature’s primary use case is the abuse itself.
Platform-Industrialized Harm
There is a meaningful difference between a platform that fails to prevent harm and a platform that manufactures it. The Grok crisis introduces a category that needs a name: Platform-Industrialized Harm.
Platform-Industrialized Harm occurs when a company’s product doesn’t just fail to prevent abuse — it structurally enables abuse at industrial scale. The defining characteristics:
The harm is a direct output of the product, not a byproduct of misuse. Aurora generates the images. The platform hosts the source material. The subscription collects the revenue.
The scale exceeds dedicated criminal infrastructure by orders of magnitude. Grok’s 84x multiplier over the top five deepfake websites means xAI accomplished in weeks what criminal enterprises spent years building — and surpassed them without trying.
The architecture guarantees the outcome. When a tool that edits any image is deployed on a platform containing billions of photos of real people, the result isn’t a risk to be managed. It’s a mathematical certainty.
Public pledges don’t change the architecture. CBS proved this. You can announce safeguards without implementing them, and the product continues to function exactly as it did before the announcement. The feature is the problem. Pledges address intent. Architecture determines outcomes.
This framework matters because the legal and regulatory tools designed for previous platform harms don’t map cleanly onto Platform-Industrialized Harm. Section 230 was written for platforms that host user-generated content, not platforms that generate the content themselves. GDPR was designed to govern data processing, not to address a product that processes people’s likenesses into pornography. The class action, the state investigations, the GDPR inquiries, and the French criminal prosecution are all different legal systems reaching for tools that weren’t built for this specific problem.
The Grok crisis will be studied for what it revealed: that the distance between “AI product feature” and “industrialized sexual abuse” can be a single product update, deployed in December, generating 3 million images before anyone with enforcement power could respond. The 84x number isn’t a data point. It’s the moment a platform became more prolific than the criminals it was supposed to be better than.
Originally published at https://noahaust2.github.io/strategist-dashboard/blog/the-grok-undressing.html
Write a comment