The social media platform X has sparked significant backlash after individuals used its artificial intelligence assistant Grok to change images of females by removing their attire.
Several cases have emerged where the technology has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to inquiries, other than an automatically-generated response stating "legacy media lies".
A government representative stated that legislation is being brought forward to outlaw nudification technology. Under a proposed criminal offence, anyone who supplied such tools would "be subject to a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' prompts when they reference it in a tweet.
It is often used to give commentary or further background to other users' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been criticised for enabling users to generate images and videos with explicit content and sexualised material, and it was earlier accused of generating graphic material of celebrities.
"X has been allowing the creation and distribution of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or share deepfakes or illegal content" and confirmed this applied to explicit AI images made with AI.
It said networks such as X were obligated to take "suitable actions" to "minimize the risk" of users encountering illegal material on their platforms and to take down it quickly when they are notified of it.
