After rampant AI abuse, Grok doubles down by creating a new video generator

Millions of reports of AI misuse haven’t stopped xAI, Grok’s parent company, from rolling out new, more powerful AI tools. On Sunday, xAI foot New version of the AI-based generative video model on X, Grok Imagine 1.0.

The new model can create 10-second videos at 720p with audio, similar to competitors like OpenAI Sora and Google I see 3. Already, Grok’s AI-based video creator has created more than 1.2 billion videos in the last 30 days.

Behind Grok’s popularity lies a dark story that reveals the dangers of uncontrolled artificial intelligence. From the end of December through early January, several X users asked Grok to create nude images of people, especially women, in photos that others shared on the platform. Anyone who posts a photo, as innocuous as a selfie or a group outing, on the platform could become an unwilling target of harassment.

Other AI models don’t allow nudity requests, but Grok has no qualms about it: its “hot mode” can create suggestive and provocative images. However, what happened went further than that. It was a publicly shared, unfiltered, image-based sexual assault.

Grok took 1.8 million fake sex photos over a nine-day period in January, according to a report by New York Timeswhich includes 41% of the total images taken by Grok. separate He studies From the Center for Countering Digital Hate it was estimated that Grok made about 3 million sexual images over the course of 11 days, with 23,000 of those deep porn images featuring children.

On January 6, in the midst of the scandal, X’s head of product, Nikita Beer, subscriber The app was recording its highest interaction ever. (He did not attribute the engagement to a specific reason.)

A post dated January 8 noted that the company had put its image creation and editing capabilities behind a paywall. On January 14, the company He said It has improved guardrails to prevent the creation of sexually offensive material.

Post reports quickly Show Those handrails weren’t strong enough. Grok’s photo creation is still available for free through his website. Now, the unveiling of Grok Imagine 1.0 signals a major upgrade to the platform’s generative video capabilities, raising more questions about content moderation in the wake of backlash over sexualized AI images.

The California Attorney General and the UK government have done just that open Investigations into xAI. Indonesia and Malaysia have banned X. Three US senators and advocacy groups have done so He called Apple and Google remove X from their app stores for violating their terms of service.

xAI did not immediately respond to a request for comment.

The US government approved Take it law In 2025, it will make it illegal to share intimate photos and deepfake without consent. But the platforms have until May to set up their processes to remove the images, which doesn’t help existing X users.

For more, read our full report on Non-consensual sexual photos of your puppy.