The social media network X has caused significant outrage after individuals used its AI chatbot Grok to change photos of females by stripping their attire.
Numerous instances have emerged where the tool has been used to digitally undress female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit situations.
Company Reaction and Regulatory Position
XAI, the organization that created Grok, offered no comment to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to prohibit nudification technology. Under a new law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' requests when they reference it in a tweet.
It is often used to offer reactions or additional context to other users' comments, but people on X are also can edit an uploaded photo through its AI image editing function.
History of Controversy and Policy
The tool has been criticised for permitting users to generate images and footage with nudity and sexually suggestive material, and it was in the past accused of creating inappropriate content of public figures.
"The platform has been allowing the creation and sharing of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting images of people in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or share non-consensual intimate images or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate measures" to "lessen the danger" of users viewing unlawful material on their platforms and to remove it swiftly when they become aware of it.
