Deepfake CSAM Lawsuit Against xAI, Stability AI Expands with New Plaintiffs
A class-action lawsuit targeting xAI and Stability AI over the creation of deepfake child sexual abuse material (CSAM) has been expanded to include new plaintiffs alleging their images were used via Grok and Stable Diffusion models.
A class-action lawsuit originally filed in March has been significantly expanded, with two new plaintiffs joining the complaint against xAI and Stability AI. The lawsuit now includes allegations from individuals, identified as Jane Does 4 and 5, who claim their images were used by family members or acquaintances to generate nonconsensual deepfake child sexual abuse material (CSAM) through the use of xAI's Grok AI model. These new plaintiffs join three original plaintiffs, all of whom remain anonymous and report experiencing profound humiliation and distress due to the dissemination of illicit content derived from their personal photographs and videos.
One of the new plaintiffs, a female from Wyoming, detailed a harrowing experience where her stepfather allegedly uploaded a photo of her at age 11 to his phone. Using Grok, he then generated over 7,000 CSAM-related images of her, which he subsequently shared and traded on social media platforms. The lawsuit asserts that the stepfather specifically chose Grok because it was perceived as less restrictive than other AI models, readily responding to prompts that involved generating sexually explicit material from images of minors.
The amended complaint further alleges that xAI was notified in February about the generated images and provided with a tip to the National Center for Missing and Exploited Children. However, the company reportedly only submitted the original, authentic image as evidence. When law enforcement requested the thousands of Grok-generated images and IP address information that could have quickly identified the perpetrator, xAI allegedly did not respond. The plaintiff's stepfather died by suicide two days after his arrest on child exploitation charges, an event that compounded the "extreme personal crisis" caused by the AI-generated material.
This plaintiff now struggles with "self-loathing and disgust," "extreme anxiety" over the potential for the images to be discovered online, and depression, including suicidal ideation. The lawsuit highlights the devastating personal impact of AI-generated CSAM, emphasizing the victim's loss of control and the ongoing psychological trauma.
The lawsuit also adds Stability AI as a defendant, accusing the company of releasing Stable Diffusion 1.0 as an open-weight model despite knowing it was trained on CSAM. The complaint states that while Stable Diffusion 1.0 initially included a classifier to block such image generation, the nature of its training data—gathered through unguided web crawling—included a significant amount of explicit material, including CSAM. This training data made it easier for downstream developers to modify the model and bypass its protections.
Furthermore, the lawsuit claims that Stability AI later rolled back stronger guardrails introduced in Stable Diffusion 2.0, responding to user complaints that the new restrictions were "prude" and "unpopular." This decision, the plaintiffs argue, fueled an ecosystem of modified models and "jailbroken" applications, such as "nudify apps," that exploit the AI's capabilities for illicit purposes. Stability AI was aware that its models, once capable of generating sexually explicit images, would foreseeably be used to generate CSAM without adequate safeguards.
Another new plaintiff, Jane Doe 5, reported that an adult male related to a classmate used Grok to transform a photograph from her eighth-grade graduation into illicit material. These images were also traded and shared online. Although the perpetrator was arrested and charged, a substantial portion of the content remains accessible on the internet, leaving Jane Doe 5 with a "complete lack of control over the ongoing dissemination of the files" and uncertainty about the extent of its distribution through darknet channels.
The expanded lawsuit underscores the growing concerns surrounding the misuse of powerful AI generative models for creating and disseminating illegal and harmful content. It points to a critical need for robust safety measures, responsible development practices, and effective oversight within the AI industry to prevent such devastating outcomes.