Apple says its Vision Pro is the result of decades of collective experience on Mac, Apple Watch and iPhone (Photo: Apple)

Apple has announced a new virtual and augmented reality headset called Apple Vision Pro. Set to cost $3,500 it is likely to become a strong competitor for the Microsoft HoloLens in the enterprise space. Announced during the company’s WWDC developer conference, it includes a new operating system called VisionOS and works with existing iOS and iPad apps.

Apple says its Vision Pro is the result of decades of collective experience on Mac, Apple Watch and iPhone. (Photo courtesy of Apple)

Bringing back the “one more thing” section which Apple founder Steve Jobs used to reveal new innovations, CEO Tim Cook used his WWDC keynote to introduce Apple Vision Pro, describing extended reality as a “profound technology”. The device looks like ski goggles but with similar styling to the Apple Watch, including a crown-type ring on the side. Cook said it was “a new type of computer that blends the virtual world with the real world” and the “first Apple product you look through and not at”.

Vision Pro is controlled through hand gestures, voice and eye movements. It includes a separate battery rather than one housed on device. This, according to Apple, spreads the weight rather than have it worn on the head. The device has multiple cameras and a lightweight frame machined from a custom aluminium alloy that also houses a high-performance computer that is air-cooled. It includes a seal inside that can be customised to suit the face shape of the user.

“Use your apps anywhere, make them any size you want, capture photos and video and relive your most important memories any way you want,” Cook enthused. The device blends digital content into the real space, which Cook said is a “new dimension” of personal computing which he called spatial computing.

As well as the AR-based initial setup, Apple has launched “environments” that allow you to enter a full virtual reality world rather than overlay digital on the real world. Even if fully immersed, Apple says if someone comes into view they can be shown overlaid on your virtual world.

Apple intends the Vision Pro to be used in the office or for remote working. This could include working across the same document or using Facetime and other video conferencing software. It also has the same document-sharing features and apps as the iPhone and macOS.

For video conferencing, the Vision Pro features an on-device neural network to produce a virtual avatar that mimics the way you look, as there are no cameras looking back at you directly.

The headset is powered by an M2 Apple Silicon chip that runs in parallel with a new chip called R1 designed for “real-time sensor processing” that can take input from 12 cameras, six microphones and multiple sensors to eliminate lag, streaming new images to displays within 12 microseconds. This, the company claims, helps to minimise nausea, a common problem experienced by users wearing XR headsets.

It comes with a new operating system called VisionOS designed “from the ground up for spatial computing”, as well as a number of developer kits. This includes all of the existing frameworks used in iOS and macOS so all existing iPhone apps will be available from day one.

Apple’s roadmap: software updates and AI

Throughout the keynote, Apple highlighted end-to-end encryption and “on-device” processing of data, rather than sending information to the cloud for processing. This even includes allowing the downloading of specific map areas so they are available offline.

Also announced during the conference were major updates to Apple’s existing product lines including the Mac and software like iOS, macOS and iPadOS. This includes a range of AI features such as “adaptive audio” that adjusts volume based on background noise. Individual apps, including the Phone, Notes and Messages apps are also being updated. The new version of MacOS is being called Sonoma and will include many of the new iOS features.

One of the biggest updates is coming to the Phone app. It is getting a new AI-powered feature called “live voicemail” that offers a live transcription in real-time as people leave a voice message. You can opt to pick up the phone mid-transcription if you want to take the call, or leave it to voicemail. This is created using the on-device neural engine built into the Apple Silicon processor.

Without mentioning AI, Apple confirmed several AI features. These are included through its apps including live check-in for messages, the live voicemail feature and search filters. Within images, the foreground object of photos taken on the phone can be extracted and turned into stickers. This also works with “live photos” to create animated stickers.

AirDrop, the feature that lets users send a file between devices, has also been updated and now includes the ability to easily swap contact details through “name drop”, that works like a smart business card. Autocorrect is also getting a refresh to include grammar and sentence correction that uses the built-in neural engine to learn behaviour – so predictions match the words you want to use not the words it thinks you want to use. The company is also using transformer AI models to update dictation to make it more accurate.

On the Mac there will be a range of new features including AI-powered video effects available to any video conferencing tool. This includes gestures to trigger background effects such as a thumbs up to create fireworks behind the person in the frame. Safari, the built-in browser on the Mac is also getting “private browsing” that blocks all windows when not using them. It will also remove URL trackers.

Apple is hoping to turn its TV box into a conferencing tool, allowing users to use the phone camera and live sync with any video conferencing app on the big screen. This includes Zoom and WebEx. It also allows app sharing with any iPhone or iPad app.

WWDC: Mac and Apple Silicon

The company announced an update to its small form factor “pro studio” desktop machine and a new version of the Mac Pro desktop with expandable PCI slots in tower and rack mount versions. Both will come with all versions of the M2 processor.

Apple also announced a new high-performance M2 Ultra processor. It has a 76-core GPU and a 32-core neural engine. The company claims the M2 Ultra chip is capable of training large foundation models on the device, describing it as the “most powerful chip on a personal computer”. Both the Pro and Pro Studio will work with the M2 Ultra.

Ben Wood, chief analyst at CCS Insight said: “The benefit of familiar apps and other Apple products such as the iPhone and AirPods working seamlessly with a Mac is very appealing to consumers. This is a hugely worrying trend for PC makers as consumers that buy a Mac end up being even more tightly locked into Apple’s ecosystem.

“The new Mac computers underline that the investments made in Apple Silicon are delivering valuable returns. Apple’s vertically integrated approach across semiconductors, software and hardware continues to give it an advantage over rivals. It’s a daunting situation for other chipmakers, most notably Intel, and for rival PC makers.”

When it comes to the Vision Pro, Leo Gebbie, principal analyst for connected devices at CCS Insight. said the familiarity of the Apple operating system could help the success of the device. “Apple had adopted a super-premium design for the Vision Pro headset which is clearly reflected in the price,” he said. “You can see strong influences from the Apple Watch with the Digital Crown and aluminium housing – even the head strap looks like one of the high-end watch bands it offers on Apple Watch. It certainly seems to be in a different league to the plasticky headsets from rivals – but it is [probably] ten-times more expensive!”

“The real question is what comes next. We predict that Apple’s entry into the spatial computing market will lift the fortunes of all players, but the Vision Pro is some way away from being a mass market offering given its price tag, clearing the way for some rivals for now.”

But, he added: “Apple stepping into spatial computing is an exciting vote of confidence for the category, and we expect to see more from the company in coming years.”