The online platform X has sparked significant anger after individuals employed its artificial intelligence assistant Grok to manipulate photos of females by stripping their attire.
Numerous cases have been seen where the system has been employed to digitally undress female subjects to show them in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization behind Grok, remained silent to inquiries, except for an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to outlaw nudification software. Under a proposed criminal offence, anyone who supplied such tools would "face a jail term and significant penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing illegal content on their platforms, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' requests when they tag it in a tweet.
It is often used to provide reactions or more background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.
Background of Criticism and Rules
The system has been criticised for enabling users to produce pictures and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in making graphic material of celebrities.
"X has been enabling the creation and sharing of these images for months without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of persons in a graphic way".
In a statement, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said networks such as X were required to take "suitable steps" to "minimize the risk" of users encountering illegal material on their platforms and to delete it promptly when they learn of it.
