The social media platform X has provoked widespread backlash after users employed its artificial intelligence assistant Grok to manipulate images of females by removing their attire.
Multiple examples have been seen where the tool has been employed to remove clothing from women to show them in swimwear without permission, as well as to place them in sexualized contexts.
Company Response and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, other than an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to prohibit image stripping software. Under a upcoming law, anyone who provided such tools would "face a prison sentence and significant fines".
"X or Grok has the ability to stop these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said platforms must "consider the potential" of people accessing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' prompts when they reference it in a message.
It is often used to offer reactions or further background to other posters' remarks, but people on X are also capable of edit an posted image through its artificial intelligence picture modification function.
History of Controversy and Policy
The tool has been criticised for permitting users to create images and videos with nudity and sexualised material, and it was in the past accused of generating inappropriate material of celebrities.
"The platform has been allowing the generation and spread of these pictures for months without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this included explicit AI images made with AI.
It said networks such as X were required to take "suitable actions" to "minimize the risk" of users coming across unlawful material on their platforms and to take down it swiftly when they become aware of it.
