Elon Musk's social media network X has sparked major anger after people employed its AI chatbot Grok to manipulate images of females by removing their attire.
Several instances have been seen where the technology has been used to remove clothing from female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit situations.
Company Response and Regulatory Stance
XAI, the firm behind Grok, did not respond to requests for comment, except for an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to outlaw image stripping software. Under a proposed law, anyone who provided such technology would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their networks, but did not specify whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' prompts when they reference it in a message.
It is frequently used to offer commentary or further context to other users' comments, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create images and videos with explicit content and sexually suggestive content, and it was in the past accused of creating graphic content of celebrities.
"X has been enabling the production and distribution of these images for months without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said networks such as X were obligated to take "appropriate steps" to "minimize the risk" of users encountering unlawful material on their services and to delete it promptly when they become aware of it.
