Exploring the Latest Innovations of VisionOS 2 Showcased at Apple’s Worldwide Developers Conference: Insights and Curiosities

Screenshot by David Gewirtz/ZDNET

It’s been a year since Apple announced the wildly ambitious, insanely expensive, deeply imperfect , and incredibly impressive Vision Pro , and about five months since Apple shipped it.

Since shipping in February, Apple only released one small update to VisionOS, while Meta has been busy actually shipping functional update after functional update for the far less expensive Quest 2 and Quest 3 devices.

Last week, I itemized ten VisionOS features I wanted to see Apple announce at this week’s WWDC. As expected, Apple today announced a series of updates to VisionOS.

Also: Everything Apple announced at WWDC 2024, including iOS 18, Siri, AI, and more

Did they improve the overall quality of life and usability of Apple’s most quixotic moonshot? I’ll tell you this: Apple only addressed two of the ten items I asked for. On the other hand, Apple did announce a bunch of additional features. I’ll discuss them here in the order Apple presented them during the keynote.

Disclaimer: This post includes affiliate links

If you click on a link and make a purchase, I may receive a commission at no extra cost to you.

1. Turn 2D images into spatial photos

I’ve never been terribly impressed by Apple’s implementation of spatial photos, which are photos that have a 3D feel to them. The little square box and the fairly minimal lens separation of the iPhone 15 Pro resulted in something that seems more novelty than noteworthy.

Now, Apple has announced that it’s using machine learning to derive a left eye and right eye’s view of all your 2D photos, allowing you to view your flat photos as if they were 3D images.

Also: Every iPhone model that will get Apple’s iOS 18 (and which ones won’t)

Now, here’s the thing. I tried watching a couple of Marvel 3D movies via Disney+ on the Vision Pro. All the Marvel movies were filmed in 2D and digitally remastered into 3D movies. The 3D effect is pretty underwhelming. Sometimes an object seems to be pulled from the background, but otherwise, it’s not much to write home about.

I’m guessing that Apple’s 3Difying of a 2D image will be at about the same level.

2. Watch those images together using SharePlay in the Photos app

And here’s another feature that I can’t imagine any users clamoring for. Now, if you have multiple Vision Pro users, they can all see and discuss the same image together while using the Photos app.

So, yeah, if Grandma Sadie in Seattle has a Vision Pro, Cousin Marty in Miami has a Vision Pro, and you have a Vision Pro in Yuma, you can all stand around the virtual photo screen and ooh and ah over pictures of baby Alice.

3. Gestures for opening Home View and Control Center

While it’s easy enough to open Vision Pro’s home view (the field of app icons) by pressing the Digital Crown on the top of the unit, getting to Control Center is a pain. VisionOS puts a tiny little arrow waaay up at the top of the viewing space, and you have to glance at it and tap your fingers together to get it to open.

In VisionOS 2, there are gestures. Holding your hand palm up and tapping fingers opens the app view. Flipping your hand over will give you time and battery level. A finger tap in this position will open up Control Center.

Unlike the previous two features, this is a quality of life improvement.

4. Mac virtual display improvements

The Mac virtual display feature of the Vision Pro has been nice, but limited. In VisionOS 1, you could only show one 4K display. Personally, I’m waiting for Apple to ditch the entire display paradigm and just show windows, but that’s not going to happen this year.

Screenshot by David Gewirtz/ZDNET

On the other hand, Apple “later this year” will enable you to create displays with more resolutions. The most interesting is the full wrap-around display, which will be the equivalent of two 4K monitors.

5. Choo-choo train mode

To be fair, Apple didn’t call it that. We’ve talked about the Vision Pro’s travel mode , which was mostly intended for airplane travel.

In VisionOS 2, Apple is announcing that there is dedicated support in travel mode for train travel. We know people have already been successful using the Vision Pro on the train and even subway , but in this case, Apple presumably tweaked performance somewhat.

Click here

6. New developer APIs

There are a ton of VR and AR APIs in Apple’s platforms. But this week, at WWDC, Apple is showcasing three new APIs developers can use to gain access to Apple technology.

Also: Inside VisionOS: 18 things developers need to know about coding for Apple Vision Pro

• Volumetric API: Allows complex 3D apps to run side-by-side. A volumetric API is at the core of most 3D content, because it allows developers to create, manage, and present volumetric content. My read on this announcement is not the introduction of a Volumetric API, but that it can allow multiple 3D apps to run concurrently.
• TabletopKit: Helps developers create apps that anchor to flat surfaces. One obvious example is board game kits, but this could apply to interior design apps, architectural and construction projects, gaming, navigation, and more.
• Enterprise APIs: These were described as enterprise-specific APIs, so we can assume they’re somewhat special purpose. Examples given were surgical training and manufacturing.

7. New Canon spatial lens for creating spatial video

While iPhone 15 Pro users can create spatial video, there are limitations to using an iPhone as a camera. Apple announced today that Canon will be introducing a spatial lens for its EOS R7 digital camera .

Screenshot by David Gewirtz/ZDNET

This allows more commercial and prosumer projects where the photographer or videographer needs more control over the complex range of settings available in a more traditional camera. Those videos can then be edited in Final Cut Pro, Apple’s only mention of Final Cut during WWDC. There’s also a new Vimeo app for VisionOS, that will allow spatial videos to be shared.

8. Third-party immersive video workflow

One of the truly breathtaking experiences of the Vision Pro is what Apple calls Immersive video. This is a very big step beyond the typical 3D video we’ve seen in movies and TV shows. The entire experience wraps around you 180 degrees. When it’s done right, it’s astonishing.

Also: I watched Apple’s newest Immersive Video on the Vision Pro, and it was my ‘aha’ moment

Apple is opening up the format for third-party creators to create immersive video. The approach is to use cameras and DaVinci Resolve Studio from Blackmagic Design . There’s no doubt that Blackmagic makes excellent gear and that DaVinci Resolve is a fine application. It’s just a little surprising that Apple didn’t bake this capability into its own high-end video production tool, Final Cut Pro.

Even so, I think this is great. Those immersive videos are fantastic.

Other new features

Apple didn’t cover all the new features during its announcement. At the end of the Vision Pro segment, the company put up one of its feature grid graphics (shown at the beginning of this article). Here are a few that weren’t mentioned:

• Mouse support: I mentioned this in my wishlist. Apple’s presenters didn’t mention it, but it’s on that grid of features. Hurray!
• Keyboard breakthrough: This was another feature I felt it was important for Apple to introduce, and apparently it will be available. No mention was made of it, so we have no idea if it’s limited solely to Apple’s own keyboards (but that’s probably the case).
• Bora Bora environment: I wanted to see more environments and we get…one. Personally, I think all the work on spatial photos and spatial videos could have been replaced with the ability for third parties to create 3D environments, but Apple didn’t go there. But hey, Bora Bora, right?
• Rearrangeable Home View: This was also not described but I’m guessing this means you can move around the Home icons.

What do you think?

I’ll admit, I’m fairly underwhelmed. I definitely would have thought there would be a bunch of new environments (at least one or two indoors), as well as third-party options.

Apple did announce that more than 2,000 apps were built specifically for the Vision Pro and more than 1.5 million iPad apps are compatible. But I’m still surprised that Apple didn’t create spatial versions of its remaining non-spatial apps (i.e., Books, Calendar, Clock, Home, Maps, Podcasts, Reminders, Shortcuts, Stocks, and Voice Memos).

And, I’m shocked that Find My support is still missing from VisionOS (or at least wasn’t mentioned).

What do you think? Is VisionOS 2 enough to get us through another year of this astoundingly expensive device? Is there anything in VisionOS 2 that wasn’t in VisionOS 1 that would inspire you now to go out and buy one? If you have a Vision Pro, will any of these features improve your quality of life? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter , and follow me on Twitter/X at @DavidGewirtz , on Facebook at Facebook.com/DavidGewirtz , on Instagram at Instagram.com/DavidGewirtz , and on YouTube at YouTube.com/DavidGewirtzTV .

Featured

We’ve used every iPhone 16 model and here’s our best buying advice for 2024

20 years later, real-time Linux makes it to the kernel - really

My biggest regret with upgrading my iPhone to iOS 18 (and I’m not alone)

Want a programming job? Learn these three languages

• We’ve used every iPhone 16 model and here’s our best buying advice for 2024
• 20 years later, real-time Linux makes it to the kernel - really
• My biggest regret with upgrading my iPhone to iOS 18 (and I’m not alone)
• Want a programming job? Learn these three languages

Also read: