Elon Musk's online network X has provoked major backlash after people employed its AI chatbot Grok to manipulate photos of women by removing their clothing.
Multiple cases have been seen where the tool has been employed to digitally undress female subjects to render them in bikinis without consent, as well as to place them in inappropriate situations.
Company Response and Regulatory Stance
XAI, the company that created Grok, remained silent to inquiries, other than an automatically-generated response stating "legacy media lies".
A Home Office official stated that legislation is being proposed to ban nudification technology. Under a upcoming law, anyone who provided such technology would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these types of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' requests when they reference it in a message.
It is commonly used to offer reactions or additional context to other users' comments, but people on X are also can alter an uploaded image through its AI image editing function.
History of Criticism and Rules
The chatbot has been faulted for permitting users to produce images and footage with nudity and sexualised content, and it was in the past alleged to have been involved in making graphic material of celebrities.
"X has been permitting the production and distribution of these pictures for a long time without taking any measures and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "showing images of people in a graphic fashion".
In a comment, Ofcom said it was against the law to "create or share deepfakes or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were obligated to take "suitable actions" to "reduce the risk" of users viewing illegal material on their platforms and to remove it promptly when they learn of it.
