ChatGPT can analyze your iPhone screen with iOS 18.2 — here's how it works

ChatGPT analyzing screen in iOS 18.2
(Image credit: Future)

Apple has released the first developer beta for iOS 18.2, which includes this fantastic new ChatGPT feature.

When Apple first announced its AI features at WWDC, much was said about Apple Intelligence, including ChatGPT's integration with Siri. Sadly, like many other features, it was held off until a later update for compatible devices.

Apple has since released the first beta for iOS 18.2, which includes the upgraded Siri. One of the new features is that Siri can now scan a screenshot and use OpenAI's program to find information about what it sees.

With ChatGPT, Siri can relate certain questions to OpenAI's chatbot, but it can also read what is on the screen of an iPhone, iPad or iMac. It works similarly to Google Lens, with users being able to ask Siri about something on their screen. Siri will then take a screenshot and upload it to ChatGPT, which can use its database to find relevant information to relay back to the user.

This feature will likely be more useful for anyone who hasn't upgraded to an iPhone 16, as only the newer models have the Visual Intelligence feature. While Visual Intelligence performs a similar action, it can do it live through the camera rather than with a screenshot.

Closeup of iPhone 16 Pro Max Camera Control button.

(Image credit: Future)

Apple has laid out how the data is handled with ChatGPT integration to help comfort anyone with data security concerns. When communicating with OpenAI, only the image and location data are sent to ChatGPT; no other information, like your Apple Account, is shared. OpenAI can not use the data to train its model; however, if you have a paid account with OpenAI, the data and requests can be logged.

Plenty more are coming with the iOS 18.2 beta, including Image Playground and Genmoji. The latter allows users to create new custom emojis based on several factors, including people in your contacts. Meanwhile, Image Playground uses AI to generate images based on text prompts or can use built-in suggestions and concepts. iOS 18.2 includes a redesigned Mail App with an upgraded visual design and AI-backed organization.

For now, these features are locked to the developer beta, but a public beta is likely not far away. All these changes will help keep the iPhone one of the best phones on the market.

More from Tom's Guide

Josh Render
Staff Writer

Josh is a staff writer for Tom's Guide and is based in the UK. He has worked for several publications but now works primarily on mobile phones. Outside of phones, he has a passion for video games, novels, and Warhammer. 

TOPICS