xAI's Grok generates explicit deepfakes, triggering multi-country bans and safety backlash
事件摘要
In January 2026, xAI's Grok image generator was used to create sexually explicit deepfakes, leading Indonesia, Malaysia, and the Philippines to block access to the service. xAI restricted image generation to paid users on January 9. The incident highlighted the inadequacy of safety measures on frontier image generation models and triggered investigations by Ofcom and other regulators. It became the most prominent AI safety incident of the year, demonstrating that AI-generated deepfake risks remained unresolved.
影响评估
-
Risk Creation -2 · Short-term
Demonstrated that frontier AI image generation models still lacked adequate safeguards against deepfake abuse. Multi-country bans showed real geopolitical consequences of AI safety failures. Triggered regulatory investigations (Ofcom). Sparked debate about 'spicy mode' as product strategy.
Affected Groups: social media users, public figures, governments, AI companies
共识度与来源
-
1
xAI limits use of image generation to paid users following the generation of sexually explicit deepfakes.Reference Evidence Citation logged Live source
-
2
Grok sexual deepfake scandal.Reference Evidence Citation logged Live source