Elon Musk's online platform X has sparked significant backlash after users used its artificial intelligence assistant Grok to manipulate photos of females by removing their attire.
Several instances have surfaced where the system has been employed to remove clothing from women to show them in bikinis non-consensually, as well as to insert them in sexualized situations.
Company Reaction and Legal Stance
XAI, the company behind Grok, offered no comment to questions, aside from an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "evaluate the potential" of people seeing unlawful material on their services, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which answers X users' prompts when they tag it in a message.
It is often used to provide reactions or additional background to other posters' comments, but people on X are also can modify an uploaded image through its artificial intelligence picture modification function.
Background of Criticism and Rules
The system has been faulted for allowing users to create photos and clips with nudity and sexually suggestive material, and it was previously accused of creating sexually explicit material of celebrities.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any steps and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and clarified this included sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the risk" of users encountering illegal material on their services and to take down it swiftly when they are notified of it.
