San Francisco City Attorney David Chiu has issued a formal cease-and-desist letter to Meta, demanding that the company immediately stop allowing paid advertisements containing AI-generated child sexual abuse content. The action follows investigative findings revealing hundreds of policy-violating ads running across Facebook, Instagram, and Threads.
Investigation Reveals Widespread AI Ad Misuse
According to reports by researchers at the Tech Transparency Project, Meta’s platforms ran more than 350 advertisements transforming still images of minors into short video clips depicting sexual acts. These ads directed users to third-party generative AI applications capable of producing nonconsensual intimate imagery and digital undress features.
Despite Meta stating that it maintains zero tolerance for child exploitation, investigators found that hundreds of these harmful advertisements appeared after initial warnings were raised. Many of the ads successfully bypassed automated and manual review systems, reaching thousands of accounts globally before being flagged and removed.
City Attorney Inquiry and Meta’s Jurisdictional Defense
The City Attorney’s Office has requested detailed explanations regarding how Meta’s advertising systems operate, how unsafe ads evade detection, and how repeat advertisers are handled. The inquiry also addresses reporting protocols concerning the National Center for Missing and Exploited Children.
Meta responded by arguing that the disputed advertisements were not proven to have displayed within San Francisco jurisdiction. Representatives for the City Attorney’s Office rejected this argument, noting that local consumers can access the unlawful ad content through online spaces within the city.
Growing Challenges in Generative AI Content Moderation
The ongoing controversy underscores the severe safety and moderation challenges facing major social media networks as generative AI tools become more accessible. Industry watchdogs continue to press platform operators for stronger safeguards against the misuse of synthetic media technologies.
Key Takeaways
- Cease-and-Desist Issued: San Francisco City Attorney David Chiu demanded Meta stop running paid ads featuring AI-generated child sexual abuse content.
- Widespread Violations: Over 350 ads on Facebook, Instagram, and Threads promoted third-party apps capable of generating nonconsensual intimate imagery.
- Systemic Moderation Failures: Unlawful ads bypassed automated and manual reviews, remaining active even after initial warnings were provided.
- Jurisdiction Dispute: Meta claimed no proof the ads reached San Francisco, a point strongly rejected by local officials due to online accessibility.
Join our community by subscribing to our Weekly Newsletter to stay updated on the latest AI updates and technologies, including the tips and how-to guides.
Also, follow us on Instagram (@tid_technology) for more updates in your feed and our WhatsApp Channel to get daily news straight to your Messaging App.
