Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Sometimes I sit and draw strange scribbles in the air that dangle, hover, and spin if you tap them. I draw them in midair using a thick black spatial pen made by Logitech Moses calledwhile wearing it See Apple Pro Headset.
This process seems like magic, yet Apple often seems indifferent to the opportunity. Professional-focused 3D creative tools that make 3D art like this don’t really live in Vision Pro. Apple didn’t even bother to create a spatial version of its Pencil. This feeling of incomplete frustration is exactly what makes the Vision Pro, now more than two years old and in its second hardware iteration, sometimes feel dead.
But it is not. In fact, Apple is planning more of them. Smaller and lighter, but not for at least a few years, according to Latest reporting by Mark Gorman. Meanwhile, Apple’s AI smart glasses may not even arrive, at least Late next year…and later for display-enabled devices that may get some vision-like features.
But as we move towards Apple WWDC Software Conference Next week will be the perfect time for Apple to unleash the Vision Pro and its many hidden capabilities. Not just for this device itself, but for all the things that come after it. AI included.
Apple’s characters use Gaussian splatting, a technology that remains an untapped part of VisionOS’ potential future capabilities.
More power under the hood than is being exploited
I’m fascinated by the idea of spatial computing — specifically, screens and apps floating around me on demand. But what Apple has offered so far is a limited subset of what could be. Apple An increasingly real character Avatars are only one aspect of that latent potential.
I know the possibilities because I look at and test products like this all the time and talk to people who are exploring solutions that don’t exist yet. The Vision Pro is seen as Apple’s biggest product failure of the Tim Cook era – who do you know who owns one? – but it is also widely known as the most advanced VR/AR device. The M5 processor it uses, the accuracy of its eye tracking, the quality of its farthest-closest motion sensor, and the cameras that blend views of the world around you into scrolling video are all the better.
What’s bad is how the Vision Pro fails to do all of this to deliver professional tools that are actually useful, and how it fails to explore the ideas Apple will have to work around in an expected range of AI-enabled wearables that don’t yet exist.
I expect Apple to make glasses, AirPods with cameras, and maybe even some sort of compatible necklace or pin. But in the meantime, the Vision Pro is a very real product that already has a lot of that potential, if only Apple would unleash it.
And no, an endless supply of 3D video movies and sporting events is not the answer for a $3,500 device. But what about ultra-realistic 3D scans, which can be done via Gaussian spraying and displayed on Vision Pro? Or artificial intelligence that can learn about your worn-out worlds and even guide you through multiple projects?
Samsung’s Galaxy XR plays with Gemini Live ahead of the smart glasses coming this fall. Apple can do the same.
Visual intelligence can and should happen in Vision Pro first
Camera-aware AI, or multi-modal AI, is a growing field that most major AI platforms are already involved in. Meta has already integrated it into it Smart glassesGoogle and Samsung It is mixed in Mixed reality headsets that already exist, And in glasses Coming by the end of this year.
Google and Samsung have released the Vision Pro-like device Galaxy headset last fall to explore ideas, such as an always-on Gemini Live mode that can visually recognize the space you’re in and the apps you’re using. Now that Apple has a partnership with Google To integrate Siri with Gemini, the way seems open for similar explorations in Apple devices as well.
I’m curious and concerned about how AI will mediate our senses in wearable devices, and how it can be a tool… Violation of privacy For both the wearer and anyone in the wearer’s vicinity. But in the Vision Pro, Apple has a great way to test this, using all the sensors and processors in the headset to explore these ideas before Smart glasses, smart pins, or smart AirPods Release you.
There’s another small aspect of AI that doesn’t relate to agents, known as Gaussian splatting, which Vision Pro OS should explore much more. Unlike Apple’s stereoscopic, immersive videos captured with multiple camera lenses, Gaussian specks can create full 3D images and videos using artificial intelligence to stitch them together. Apple has introduced more spatial 3D layers into Vision, with characters being the biggest part. But Vision Pro should be part of the entire 3D scanning studio system with apps that Apple creates on iPhones and in headphones, as extensions of the camera app on existing phones.
Apple Watches and iPhones should be pieces of the Vision Pro and iPad connectivity matrix, too.
Connect all things Apple to Vision Pro
The Logitech Muse is like the Apple Pencil for the Vision Pro, which Apple never bothered making. Likewise, Sony PlayStation VR 2 controllers Can connect with Vision Pro To play games in ways that aren’t possible, since Apple doesn’t make its own spatial controllers either.
Some of this is understandable, since the Vision Pro is an experimental product, and Apple often relies on companies like Logitech to explore peripheral ideas it hasn’t made yet (this happens with iPad keyboard cases, too).
However, the Vision Pro still doesn’t connect seamlessly with other Apple products that have been around for years. While AirPods connect to them, and Macs can extend screens or even stream Mac apps to headphones, iPhones, iPads, and watches are strangely left out, except for streaming iPhone/iPad screens via AirPlay.
I want to share apps and extend screens from nearby iPhones and iPads, allowing the Vision Pro to act as a shared type of computer, much like a Mac. There’s no reason they can’t. iPhone-level chips can now run MacOS, e.g MacBook Neo Proven. Sure, they can share screens and expand apps too, or let me magically manipulate all of them with one headset that recognizes them all.
The Apple Watch is perhaps the most egregious piece to be left out: it’s literally a wrist-worn trackpad with motion tracking, and could be a great interface with the Vision Pro if Apple allows it.
Apple’s creative apps aren’t all in the original Vision models yet. They should be.
Open the way for professional applications, one way or another
The biggest flaw with the Vision Pro, for me, isn’t its price or size. That’s how the headset still lags behind Macs and even iPads in being a true pro PC befitting the “pro” name, even with an M5 processor under its belt now.
What do I mean by supporter? I mean video editing suites, music creation tools, 3D graphics software — anything that can and should be on VisionOS to allow the kind of creative work that Apple prides itself on.
It’s weird to me because apparently filmmakers use Vision Pros, often as monitors to watch footage or 3D models. They should be complete gateways to do everything. The massive virtual screen and 3D interface should enable emulation and creativity that I’ve seen even in headsets like the much less powerful Meta Task 3.
If convincing developers to create powerful professional apps is too difficult and Apple doesn’t want to make them (which seems to be the case after the new version) Creator Studio suite of applications No improvements included in Vision Pro), scaling up and streaming from Mac and iPad may help. VisionOS can display 3D content from Macs and even… Improve it with your favorite streama trick that provides high accuracy only where your eyes are looking directly.
Opening up a path for developers to extend VisionOS-ready Mac apps, or doing so for iPad and iPhone, is a necessary step. Otherwise, when it comes to getting serious work done, I’ll probably put the Vision Pro down.
