The social media network X has provoked major outrage after people employed its AI chatbot Grok to manipulate images of females by removing their garments.
Multiple examples have been seen where the system has been employed to remove clothing from female subjects to make them appear in bikinis without permission, as well as to insert them in sexualized contexts.
Company Response and Legal Stance
XAI, the firm responsible for Grok, remained silent to inquiries, aside from an standard response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to prohibit image stripping software. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they tag it in a post.
It is commonly used to offer commentary or more context to other posters' comments, but people on X are also able to alter an uploaded picture through its AI image editing function.
History of Criticism and Rules
The system has been faulted for enabling users to create photos and videos with explicit content and sexually suggestive content, and it was earlier accused of making sexually explicit material of public figures.
"X has been allowing the production and sharing of these pictures for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or distribute non-consensual intimate images or illegal content" and confirmed this included explicit AI images created with AI.
It said services such as X were obligated to take "appropriate measures" to "minimize the risk" of users coming across illegal content on their services and to delete it promptly when they learn of it.
