Elon Musk's social media network X has provoked widespread anger after people utilized its AI chatbot Grok to change photos of females by taking off their attire.
Several cases have surfaced where the tool has been used to digitally undress women to make them appear in bikinis without permission, as well as to put them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, did not respond to requests for comment, aside from an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to ban nudification tools. Under a upcoming law, anyone who supplied such tools would "face a prison sentence and substantial fines".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "consider the danger" of people viewing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' prompts when they tag it in a message.
It is often used to offer reactions or more context to other posters' comments, but people on X are also capable of modify an posted image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The tool has been condemned for allowing users to create photos and footage with nudity and sexualised material, and it was previously accused of making sexually explicit material of public figures.
"The platform has been permitting the production and sharing of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of persons in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes made with AI.
It said platforms such as X were obligated to take "necessary measures" to "minimize the risk" of users viewing illegal material on their platforms and to remove it swiftly when they become aware of it.
