California AG sends Musk’s xAI company a cease and desist order for sexual deepfakes

Earlier this week, the California Attorney General’s Office Announce it XAI was investigating reports that the startup’s chatbot, Grok, was used To create non-consensual sexual images of women And the palace. On Friday, the government followed up by sending a cease-and-desist letter to the company, demanding that it take immediate action to stop the production of non-consensual intimate images and child sexual abuse material (CSAM).

“Today I sent xAI a cease and desist letter, demanding that the company immediately stop creating and distributing intimate, non-consensual images and child sexual abuse material,” California AG Rob Bonta said. In a press release. “The creation of this material is illegal. I fully expect xAI to comply immediately. California has zero tolerance for CSAM.”

Additionally, the Attorney General’s Office claimed that xAI appears to “facilitate the large-scale production” of non-consensual nude images, the likes of which are “used to harass women and girls online.” The agency said it expects xAI to demonstrate that it is taking steps to address these issues within the next five days.

At the heart of the backlash is Grok’s “hot” mode feature, which xAI created to create explicit content. The issue has spread far beyond California. Japan, Canada, and Britain opened investigations into Grok’s case, and Malaysia and Indonesia also did so temporarily The platform was blocked completely. Although xAI imposes some limitations on its photo editing features Late Wednesdaythe California AG’s office went ahead with a cease and desist letter.

X’s security account has already done so He denounced This type of user activity, saying: “Anyone who uses or pays Grok to create illegal content will suffer the same consequences as if they uploaded illegal content.” TechCrunch reached out to xAI for comment and was met with an automated email saying “Legacy Media Lies.” TC has also reached out to the California AG’s office for further context.

The emergence of free generative AI tools has raised concerns Amplification of non-consensual sexual material. Many platforms have been grappling with this issue, not just X. This despicable activity has caught the attention of not only state leaders but also Congress. And indeed on Thursday, legislators I sent a message to the CEOs of several companies — including X, Reddit, Snap, TikTok, Alphabet and Meta — asking them how they plan to stop the spread of sexual deepfakes.