Elon Musk's online network X has sparked major anger after users used its AI chatbot Grok to manipulate photos of females by stripping their attire.
Multiple cases have emerged where the technology has been employed to remove clothing from women to make them appear in bikinis without permission, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the firm behind Grok, remained silent to inquiries, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to ban nudification software. Under a upcoming law, anyone who provided such tools would "face a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they mention it in a message.
It is commonly used to provide reactions or further background to other users' comments, but people on X are also can alter an posted image through its AI image editing tool.
Background of Controversy and Rules
The chatbot has been faulted for enabling users to produce photos and videos with explicit content and sexualised content, and it was earlier accused of generating graphic material of celebrities.
"The platform has been enabling the creation and distribution of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "necessary steps" to "reduce the risk" of users viewing unlawful material on their services and to take down it swiftly when they learn of it.
