The online network X has caused significant backlash after individuals used its AI chatbot Grok to change pictures of females by stripping their garments.
Several examples have emerged where the system has been used to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in sexualized situations.
Company Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to questions, other than an standard response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to outlaw image stripping technology. Under a upcoming criminal offence, anyone who supplied such technology would "receive a jail term and significant fines".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their platforms, but did not specify whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' prompts when they reference it in a message.
It is frequently used to offer commentary or more context to other posters' remarks, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for allowing users to produce pictures and videos with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of public figures.
"X has been enabling the creation and sharing of these pictures for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.
It said services such as X were obligated to take "necessary steps" to "reduce the danger" of users coming across illegal material on their services and to delete it promptly when they become aware of it.
