Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
We knew Apple Worldwide Developers Conference It will be full of AI, but I didn’t expect to see an image feature that makes me think: “This is wild.”
during WWDC At Monday’s keynote, Apple showed off some new editing features in its Photos app that I think will be really useful. In addition to the existing cleaning tool, which can remove unwanted distractions from the image, we will also be able to enlarge the edges of the image.
But it was the spatial reframing feature, a feature that lets you adjust the composition of an image to reflect where you wanted to stand to take it, that really caught my attention.
All these features are used Generative artificial intelligenceand will be included in the category of new tools in the editing environment in the Photos app. First beta version for developers iOS 27 Now available to registered developers.
More AI, less regression
Generative AI is a technology that photographers are shying away from (or should be), thanks to all that exists. AI slope It is produced everywhere. And yes, that includes Apple’s creations Photo playground application, Image generator Which the company also showed during WWDC Keynote.
But generative AI doesn’t need to mean entire images generated from text prompts. When applied to selective areas, such as scanning a piece of trash next to a subject’s foot, generative AI can do some of the menial work of replacing pixels that photographers might spend time retouching in an app like Photoshop. Google Pixel phones Includes a similar Magic Eraser tool.
Spatial reframing is a great example of how you can use technology to enhance the real photos you take.
How would spatial reframing work?
Apple Spatial images The technology uses artificial intelligence to determine depth in a flat image, giving it a 3D effect that responds when you tilt your phone or view it in landscape mode. Vision Pro headphoneeven if it is not captured as a spatial image. It can give depth iPhone Lock screen images too.
Honestly, the effect quality is pretty good. The separation between the subject and the background is usually not jarring, nor does it have a “cropped” appearance. But it’s mostly an iPhone gimmick (I don’t have an account Vision Pro to try it out in that environment).
Spatial reframing takes that technology and makes it useful. As shown in the main demo, you’ll be able to drag the image to adjust the perspective of the shot. The background will adjust as if you had taken a physical step to the side or repositioned the camera to get a better angle.
The original image (left) gets a change in perspective (middle) and the spatial recast fills in the background details using generative AI (note the building in the distance that wasn’t visible in the original image.
Image editing software like Adobe Lightroom allows you to level the entire image, rotating it around a central axis to perform limited reframing, but at the cost of distorting the image.
After you reframe the shot, the Photos app uses Creative AI to fill in any areas around the edges.
Apple says it uses on-device spatial modeling to determine depth, and also uses a private cloud computing infrastructure to handle image creation.
“It just produces new content to fill in the gaps where perspective has changed,” said Alok Deshpande, Apple’s director of camera and photo programs. “This ensures that the reframed image remains consistent with the original scene.”
The result is an image of the location you wanted to go to when you took the photo.
It remains to be discovered whether or not the edited images are as clean as they were presented. Sometimes I use existing cleaning An advantage in images, but it can be very hit or miss in terms of the quality of the pixels generated. With the new photography templates promised in iOS 27, I hope the edited images will be ones that any photographer would be proud to share – perhaps not at a gallery or competition, but informally with friends or on social media.
