Grok Is Being Used to Depict Horrific Violence Against Real Women ⇥ futurism.com

Jason Koebler, 404 Media:

Over the last week, users of X realized that they could use Grok to “put a bikini on her,” “take her clothes off,” and otherwise sexualize images that people uploaded to the site. This went roughly how you would expect: Users have been derobing celebrities, politicians, and random people—mostly women—for the last week. This has included underage girls, on a platform that has notoriously gutted its content moderation team and gotten rid of nearly all rules.

Ananya Bhattacharya, Rest of World:

On January 2, India’s IT ministry issued a 72-hour ultimatum to X over its artificial intelligence chatbot generating “obscene” content — specifically, sexualized or manipulated images of women and, in some cases, minors.

Maggie Harrison Dupré, Futurism:

In addition to the sexual imagery of underage girls, the women depicted in Grok-generated nonconsensual porn range from some who appear to be private citizens to a slew of celebrities, from famous actresses to the First Lady of the United States. And somehow, that was only the tip of the iceberg.

When we dug through this content, we noticed another stomach-churning variation of the trend: Grok, at the request of users, altering images to depict real women being sexually abused, humiliated, hurt, and even killed.

It is extraordinary yet, sadly, predictable to me that xAI is not treating this as a problem. Setting aside the fundamentally offensive capability that any user can tell Grok to generate photorealistic images based on someone else’s likeness — something it should be prohibited from doing — there appears to be no rush to fix this high-res misogyny on demand. A basic corporate response would be to turn off image generation capabilities until better safeguards are in place. Yet I just opened the Grok account on X, switched to the Replies tab, and it took almost no scrolling at all to find it generating images like these mere seconds ago.

The kinds of images Grok is generating should be criminal, and the people who oversee it should be held liable as should those who are prompting Grok. In my eyes, though I am not a lawyer, it is no less a case of harassment when a robot is directed by a user than when the user does it themself. I know A.I. is a nascent industry and there are things people are worried about over-regulating, but this should not be one of them. xAI is clearly overseen by adolescent-brained people who think it is sufficient to make it against company policy for users to make heinous requests, rather than sufficiently restricting its own software.