The social media platform X has provoked major backlash after individuals used its AI chatbot Grok to alter photos of females by removing their garments.
Multiple cases have surfaced where the system has been used to digitally undress women to make them appear in swimwear without consent, as well as to place them in inappropriate contexts.
Developer Response and Regulatory Position
XAI, the firm that created Grok, remained silent to requests for comment, except for an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and heavy penalties".
"The platform could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which reacts to X users' prompts when they reference it in a tweet.
It is often used to provide commentary or further background to other users' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been criticised for allowing users to generate photos and videos with explicit content and sexually suggestive material, and it was in the past accused of generating sexually explicit material of celebrities.
"The platform has been permitting the production and sharing of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of people in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "necessary measures" to "reduce the danger" of users viewing illegal content on their services and to remove it promptly when they become aware of it.
