The online platform X has sparked major backlash after users used its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Several instances have surfaced where the tool has been employed to virtually strip female subjects to make them appear in swimwear without consent, as well as to put them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to inquiries, except for an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw image stripping tools. Under a proposed law, anyone who supplied such technology would "face a prison sentence and substantial penalties".
"X or Grok has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their networks, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they tag it in a tweet.
It is often used to provide commentary or additional background to other users' comments, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been criticised for enabling users to create photos and videos with nudity and sexualised content, and it was previously alleged to have been involved in making sexually explicit material of celebrities.
"X has been allowing the creation and spread of these images for a long time without taking any steps and we have not witnessed any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing likenesses of individuals in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and stated this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "suitable actions" to "lessen the risk" of users viewing unlawful content on their platforms and to take down it swiftly when they become aware of it.
