Microsoft's new tech scans online chats to catch child predators

Microsoft
(Image credit: Shutterstock)

Good news for children and parents: Microsoft announced yesterday (Jan. 9) that it will hunt down online sexual predators using artificial intelligence to scan chats in search of potential child grooming. 

Child grooming is a method used to lure potential victims. The predator simply talks with a targeted child over a long period of time to make the child feel safe and comfortable. If successful, the grooming can lead to sexual abuse online, which may involve forcing kids to send sexual videos and meeting them physically.

How does Microsoft's approach work?

Project Artemis uses artificial intelligence to continuously monitor chats with kids to detect conversation that could be interpreted to be grooming.

The technique, Microsoft says, “evaluates and rates conversation characteristics and assigns an overall probability rating. 

"This rating can then be used as a determiner, set by individual companies implementing the technique, as to when a flagged conversation should be sent to human moderators for review.” 

Human moderators could then evaluate the contents and identify “imminent threats for referral to law enforcement, as well as incidents of suspected child sexual exploitation to the National Center for Missing and Exploited Children (NCMEC)”. 

According to Microsoft, the “NCMEC, along with ECPAT InternationalINHOPE and the Internet Watch Foundation (IWF), provided valuable feedback throughout the collaborative process.”

Privacy concerns

Of course, this human moderation factor raises a concern for privacy. It wouldn’t be the first time that tools allegedly used for our security have been misused. On the other hand, such sensitive matters can’t be all left in the hands of an AI algorithm. 

Testing for years

Microsoft says that the new tool, called Project Artemis, has been developed for the last 14 months in collaboration with The Meet Group, Roblox, Kik and Thorn, beginning with the November 2018 Microsoft “360 Cross-Industry Hackathon“, an event co-sponsored by the WePROTECT Global Alliance and the Child Dignity Alliance

The software giant says it has successfully used Project Artemis’ underlying techniques in Xbox Live “for years”. Now it’s looking to incorporate the Project Artemis tool set into Skype, its multi-platform chat system.

Even better, Project Artemis is now available to any company that wants to incorporate its software. Developers who are interested in licensing these technology could contact Thorn starting today, January 10.

'By no means a panacea'

Microsoft warned that Project Artemis would not end online child abuse. 

“Project Artemis is a significant step forward, but it is by no means a panacea,” the company said in its announcement. “Child sexual exploitation and abuse online and the detection of online child grooming are weighty problems. But we are not deterred by the complexity and intricacy of such issues.” 

Earlier this week, Apple announced at a CES 2020 privacy roundtable that it scans user accounts for known images of child pornography and child abuse. Apple chief privacy officer Jane Horvath said that if Apple finds any such images, the user accounts are automatically flagged, the (London) Telegraph reported.

Apple didn't specify exactly how it does this, but its own description of the process seems to match a technology jointly developed by Microsoft and Dartmouth College  called PhotoDNA, which The Telegraph said is also used by Google, Facebook and Twitter. (PhotoDNA is also used to track terrorism-related content.)

PhotoDNA compares new images to a database of known child-abuse images that have already been detected and flagged by authorities. It also works with audio an video to an extent. But PhotoDNA can't prevent grooming and future abuse, as Project Artemis is designed to do.

TOPICS
Jesus Diaz

Jesus Diaz founded the new Sploid for Gawker Media after seven years working at Gizmodo, where he helmed the lost-in-a-bar iPhone 4 story and wrote old angry man rants, among other things. He's a creative director, screenwriter, and producer at The Magic Sauce, and currently writes for Fast Company and Tom's Guide.

Latest in Online Security
Windows
240 million Windows 10 users are vulnerable to six different hacker exploits — protect yourself now
Victims of Identity Theft
FTC says Americans lost $12 billion to scams last year and these were the worst ones — here's how to stay safe
Apple iPhone 16 Plus Review.
Apple just released an emergency security update for a flaw used in an ‘extremely sophisticated attack’ — update your devices right now
A person trying to set up a new Wi-Fi router
Thousands of TP-Link routers have been infected by a botnet to spread malware
An image of a CAPTCHA
Hackers are using reCAPTCHA to trick users into infecting their own PCs with malware — how to stay safe
A smartphone screen displaying the Android name and logo next to a sign reading 'MALWARE'.
Fake Google Play Store pages are spreading Trojan malware that can steal your financial data
Latest in News
NYTimes Connections
NYT Connections today hints and answers — Sunday, March 16 (#644)
Nintendo Switch 2
New Nintendo Switch 2 FCC filing suggests this beloved Nintendo controller could make a comeback
(From L to R) Rohan (Nik Dodani), Josh (Brandon Flynn), Dorothy (Edie Falco), John (Dean Norris), and Liddie (Lisa Kuthrow) in The Parenting
Max top 10 movies — here’s the 3 worth watching right now
NYT Strands on a cellphone
NYT Strands today — hints, spangram and answers for game #378 (Sunday, March 16 2025)
Samsung Galaxy Tab S10 FE renders
Samsung Galaxy Tab S10 FE price leak is bad news for budget-conscious buyers
Google Assistant
Gemini to kill off Google Assistant on most Android phones — here's what you need to know