The online platform X has provoked major backlash after users employed its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Numerous instances have emerged where the tool has been used to digitally undress female subjects to show them in bikinis without consent, as well as to place them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the firm behind Grok, did not respond to inquiries, aside from an standard response stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and significant fines".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the potential" of people seeing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' requests when they tag it in a post.
It is often used to offer commentary or further background to other posters' remarks, but people on X are also able to modify an posted photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The tool has been faulted for permitting users to create photos and videos with explicit content and sexualised material, and it was previously alleged to have been involved in making graphic content of celebrities.
"The platform has been allowing the production and sharing of these pictures for months without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or distribute non-consensual intimate images or illegal content" and stated this included sexual deepfakes generated with AI.
It said networks such as X were required to take "necessary steps" to "lessen the danger" of users viewing unlawful material on their platforms and to remove it quickly when they become aware of it.
