A picture of my children at the beach, my son playing soccer or even my entire family, smiling in front of our house are all examples of photos I've posted to social media. I know I'm not alone, considering just how often I've seen similar images highlighting the goings-on of modern lives.
But AI has created a disturbing new risk that allows the use of ordinary photos of real children like the ones I mentioned above, to be manipulated into sexual abuse imagery without the child — or their parents — ever knowing.
To be clear, this is not a theoretical threat.
332 AI-generated child sexual abuse ads slipped through
A new investigation from the Tech Transparency Project (TTP) found hundreds of ads containing AI-generated child sexual abuse material (CSAM) running across Facebook and Instagram. Researchers identified four real children whose ordinary photos had been manipulated, according to WIRED, including two teenage influencers, a preteen whose image appeared on a stock-photo site and a young member of a European royal family.
In total, TTP found 332 ads containing AI-generated child sexual abuse material, or CSAM. Most depicted a child whose photograph had been manipulated using AI to make it appear they were performing a sexual act.
The findings are an alarming reminder of how generative AI has changed the risks around sharing photos of children online. Creating convincing fake imagery no longer requires an explicit photograph in the first place. An ordinary picture can provide the raw material.
How ordinary photos become AI-generated abuse
Generative AI image tools can take an existing photograph and modify parts of it while preserving recognizable features of the person in the original.
That's what researchers say happened to several of the children identified in the Meta ads.
According to WIRED, researchers were able to trace images of four minors back to their original sources. Two were teenage influencers with public social media accounts. Another was a preteen whose photograph was available through a stock-photo website. The fourth was a young member of a European royal family.
None of those original photographs needed to contain sexual material for them to be misused.
"AI apps are making it far too easy to create realistic criminal imagery of child sexual abuse, creating the new risk of everyday photos of children being taken and transformed into sexual content," Emma Hardy, director of communications at the Internet Watch Foundation, told WIRED.
Once an ordinary image is publicly accessible, someone can potentially download it and feed it into an AI tool designed to alter photographs.
Meta's AI didn't catch hundreds of the ads
The scale of the latest investigation also shows how difficult the resulting content can be to stop.
In August, TTP researchers initially uncovered 53 ads containing child sexual abuse material across Meta's platforms. Meta subsequently said many of those ads had appeared before the company introduced new AI technology intended to improve its ability to detect violating ads when they were uploaded.
However, researchers kept finding them. In fact, TTP says it ultimately identified 332 ads containing AI-generated CSAM, with hundreds appearing after Meta had announced its improved detection technology.
That is particularly significant because these weren't simply Facebook or Instagram posts that escaped moderation. They were paid advertisements.
Meta says ads are reviewed before they're allowed to run and that automated systems play a major role in that process.
Meta spokesperson Tracy Clayton told WIRED that the company does not tolerate nudify apps or child exploitation, whether the imagery involves real children or is generated by AI. Meta said bad actors deliberately attempt to evade its detection systems and that some of the videos were harder to detect because images of children appeared only briefly or were blurred.
The company said many of the ads researchers identified had already been removed, most received fewer than 200 impressions and it received less than $5,000 in payments from the ads. But researchers documented instances in which exposure increased after an ad was reported.
TTP says one ad involving a real teenager had reached four European accounts when it was reported to Meta. Several days later, when Meta removed it, the ad had reached more than 330.
The investigation shows the tools are mainstream
Researchers found ads directing people toward nearly 50 apps available through Apple's App Store and Google's Play Store. Around 300 of the roughly 350 ads examined in the broader investigation promoted AI image or video generators, according to WIRED.
Apple told WIRED that some developers initially submitted compliant versions of their apps before subsequently adding prohibited nudification capabilities. Apple said it removed 17 of 20 apps identified by researchers and took action against the remaining three, as well as seven additional apps WIRED identified.
Google said it removed all apps included in the report and suspended hundreds of other apps for violating its policies.
Still, these finding reveal a problem that extends beyond any single platform because AI can be used to create the imagery. An app store can distribute the tool used to make it. An advertising platform can then distribute ads promoting it.
Each layer may have its own safety systems, but abusive material only needs to slip through those systems once to reach users.
What to do if you find an AI-generated image of your child
If you discover sexualized AI imagery involving your child, don't repost or widely share the image in an attempt to warn others.
Instead, preserve information that can help identify where the material appeared, such as the account name, page address and other surrounding details, and report the content directly to the platform.
In the U.S., suspected child sexual exploitation can also be reported to the National Center for Missing & Exploited Children's CyberTipline.
Parents can also use NCMEC's Take It Down service, which is designed to help stop the online sharing of nude, partially nude or sexually explicit images or videos involving someone who was under 18. The system creates a digital fingerprint, or hash, from an image without requiring the image itself to be uploaded.
Since AI-generated imagery creates newer and more complicated scenarios, parents who aren't sure whether a manipulated image qualifies should still report what they've found rather than trying to determine that themselves.
The takeaway
Parents have been warned about "sharenting" and children's digital footprints for years. Sharenting is a term that combines "sharing" and "parenting" and is the practice of parents sharing images, videos and personal details about their children on social media.
Although parents can't entirely eliminate the risk of AI abuse on the images or videos they post, but reducing the number of publicly accessible photos of your children and being more selective about what you share, can reduce the amount of material available to people who might misuse it.
Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok.
More from Tom's Guide
