The social media network X has provoked significant outrage after individuals utilized its AI chatbot Grok to change pictures of females by taking off their attire.
Numerous cases have been seen where the system has been used to digitally undress women to show them in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Stance
XAI, the firm responsible for Grok, remained silent to questions, other than an auto-reply message stating "legacy media lies".
A Home Office official stated that legislation is being introduced to ban nudification technology. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people accessing unlawful material on their networks, but did not specify whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' queries when they mention it in a tweet.
It is frequently used to offer commentary or further context to other users' comments, but people on X are also capable of modify an posted image through its artificial intelligence picture modification function.
Background of Criticism and Policy
The system has been condemned for allowing users to produce pictures and clips with explicit content and sexualised material, and it was previously alleged to have been involved in creating graphic content of public figures.
"X has been permitting the creation and spread of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and confirmed this covered explicit AI images made with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the danger" of users coming across unlawful material on their services and to take down it quickly when they are notified of it.
