Elon Musk's online network X has sparked major outrage after people utilized its AI chatbot Grok to manipulate images of females by stripping their attire.
Several cases have surfaced where the technology has been employed to remove clothing from women to make them appear in swimwear without consent, as well as to put them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the organization behind Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A government representative stated that legislation is being brought forward to prohibit image stripping software. Under a proposed law, anyone who provided such tools would "be subject to a prison sentence and significant fines".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people seeing illegal content on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which answers X users' requests when they mention it in a tweet.
It is frequently used to offer reactions or further background to other posters' comments, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been faulted for allowing users to produce photos and clips with explicit content and sexualised material, and it was in the past accused of generating graphic content of public figures.
"The platform has been permitting the production and spread of these images for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying images of individuals in a pornographic way".
In a statement, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and stated this applied to explicit AI images generated with AI.
It said networks such as X were mandated to take "necessary actions" to "minimize the risk" of users viewing illegal content on their services and to take down it swiftly when they become aware of it.
