Oculus Expanding Quest Mixed Reality Capabilities With Enhanced Developer Tools

https://ift.tt/2PwAH8n

Oculus plans to further open up the mixed reality capabilities of Quest with new tools that will allow developers to build apps which more intelligently integrate with the user’s real room. In the near future developers will also be permitted to distribute mixed reality apps to customers via the Quest store or Oculus App Lab for the first time.

Oculus first began unlocking Quest’s mixed reality capabilities with the Passthrough API which allowed developers to tap into the headset’s pass-through video view for the first time earlier this year. Now the company is announcing a more advanced set of tools, which it calls the Presence Platform, which will allow developers to build more advanced mixed reality applications.

The Presence Platform includes the Insight SDK, Interaction SDK, and Voice SDK.

Insight SDK

The main building block of the insight SDK is the Passthrough feature, which developers previously had access to in an experimental form. That feature is moving out of its experimental form and into general availability starting with the next developer update.

Additionally, the Insight SDK includes Spatial Anchors which gives developers the ability to place virtual objects in the scene and allow them to persist between sessions. For instance, a piano learning app could allow you mark the location of your piano, and the app could then remember where the piano is any time you open it.

The Insight SDK further includes Scene Understanding, which Oculus says allows developers to build “scene-aware experiences that have rich interactions with the user’s environment.” This includes geometric and semantic representation of the user’s space, meaning developers can see the shape of the room and get a useful idea of what’s in it. For instance, the Scene Understanding feature will allow developers to know what parts of the scene are walls, ceilings, floors, furniture, etc all of which can be used as a surface on which virtual content can be naturally placed.

Oculus says the developer will see a “single, comprehensive, up-to-date representation of the physical world that is indexable and queryable.” You can think of this like the headset building a map of the space around you that developers can use as a guide upon which to build a virtual experience that understand your physical space.

However, users will need to do some work on their end in order to generate this map for apps that need it, including marking their walls and tracing over their furniture.

Crucially Oculus says that the Insight SDK will enable developers to build feature-rich mixed reality apps “without needing access to the raw images or videos from your Quest sensors.” We’ve reached out to the company to further clarify if Oculus itself will send the raw sensor footage off of the headset for any processing, or if it will all happen on-device.

The Scene Understanding portion of the Insight SDK will launch in an experimental form early next year, according to the company.

Interaction SDK

Another part of the Presence Platform is the Interaction SDK which will give Unity developers a ready-made set of simple interactions for hands & controllers, like poking buttons, grabbing objects, targeting, and selecting. This saves developers time in building their own versions of these commonly used interactions in their apps.

Oculus says the goal of the Interaction SDK is to “offer standardized interaction patterns, and prevent regressions [in tracking performance of specific interactions] as the technology evolves,” and further says that the system will make it easier for developers to build their own interactions and gestures.

The company says that the Interaction SDK (and the previously announced Tracked Keyboard SDK) will become available early next year.

Voice SDK

The Voice SDK portion of the Presence Platform will open up voice-control to Quest developers, which Oculus says can drive both simple navigation functions (like quickly launching your favorite Beat Saber song with your voice) and gameplay (like casting a voice-activated spell).

The system is based on Facebook’s Wit.ai natural language platform which is free to use. Oculus says the Voice SDK will arrive in an experimental form in the next developer release.

Mixed Reality Apps on the Quest Store and App Lab

While not all of the Presence Platform SDKs will arrive at the same time, as of the next Quest developer release, devs will be allowed to ship mixed reality apps via the Quest store or App Lab. That release is expected next month.

The World Beyond Sample App

Early next year Oculus says it will make available a sample project called The World Beyond which developers can use as a starting point for building atop the Presence Platform features. The app will also be made available to users.

The post Oculus Expanding Quest Mixed Reality Capabilities With Enhanced Developer Tools appeared first on Road to VR.

from Road to VR https://ift.tt/3vRC8mK
via IFTTT

Facebook Boosts AR Developers Tools for Forthcoming AR Glasses

https://ift.tt/3GvED3e

Today Facebook announced that it plans to expand the capabilities of its Spark AR platform to give developers more tools to build experiences that leverage existing smartphone-based AR and the company’s upcoming AR glasses.

Facebook has made no secret that its Ray-Ban Stories camera glasses are just its first step toward building full blown AR glasses. But just as the hardware continues to be refined, the software must evolve too.

Facebook’s Spark AR Studio toolset was initially built largely around AR face filters but has been steadily expanding with more features. The latest updates improve the platform’s understanding of things beyond the user’s face, like their entire body and the world around them. The company says it’s “working toward fully-featured AR glasses” and hopes the new capabilities of Spark AR will “illuminate the path to AR glasses.”

The updated toolset will soon support geo-mapped experiences for public spaces which will allow for location-based AR experiences that will be accessible from specific locations. This could enable experiences like a scavenger hunt, guided tour around a city, or even a murder mystery where you have to go find clues in the real world.

Image courtesy Facebook

Facebook says it’s testing this geo-locked AR capability with select partners, but will open it up to everyone in 2022.

Beyond geo-awareness, the company says it’s adding body and hand-tracking to Spark AR Studio which will allow developers to easily anchor AR effects to a person’s body. The capability will support “20 different 2D key points” which can be applied to a single person or multiple people.

Hands can be tracked specifically as well, and Facebook says a hand bounding-box will be the first function of a “multi-phased capability rollout” which will ostensibly add additional hand-tracking features (like, perhaps, finger-tracking) in the future.

Facebook says that body and hand-tracking in Spark AR Studio will be available “soon.”

The company is also building out what it calls a “virtual objects pipeline” for AR, which will make it easy to place virtual 3D objects in the scene which can include text, characters, GIFs, stickers, and more, all of which will support an understanding of depth, occlusion, and plane tracking to convincingly existing in the space. This developer feature will come in 2022, according to Facebook.

The post Facebook Boosts AR Developers Tools for Forthcoming AR Glasses appeared first on Road to VR.

from Road to VR https://ift.tt/2ZsImxi
via IFTTT

Web & WebXR Apps Are Coming to the Quest Store With Multitasking

https://ift.tt/3jIxUsY

Oculus today announced that web apps and WebXR apps will soon be allowed into the main Quest store and App Lab. Apps served this way will support multitasking, allowing users to run them side-by-side for increased productivity. Developers will also be able to charge for these experiences just like any other app on the headset.

Oculus recently updated Quest to support multitasking for flat apps, but only a few of its first-party apps were available, making the feature only minimally useful.

That’s about to change as the company today announced that it will allow web apps based on the Progressive Web App framework—like Slack, Dropbox, and Facebook—to be distributed through the main Quest store or App Lab.

Obviously one could just as easily have pulled up Slack, Dropbox, or Facebook through the built-in Oculus Browser, but when served this way, these ‘flat’ web apps will support multitasking within the Quest interface, making it easier to run productivity apps side-by-side in the headset.

Apps installed this way will also be included in the headset’s regular app library, which should make them easier to access compared to first launching the browser and then entering a URL or navigating to a bookmark.

Developers could also ostensibly modify their app to run differently when in the headset compared to running on a standard monitor; this could include tweaks for ease-of-use (like bigger buttons and text or an email-based login to spare the user from typing their credentials with the slow virtual keyboard). As an incentive to customize their app specific to the headset, developers can also choose to charge for their web apps, according to Oculus, though in-app subscriptions and DLC functionality isn’t supported.

Oculus says it will launch the first batch of web apps in the Quest store tomorrow, starting with Facebook, Instagram, Smartsheet, and Spike, followed soon by Dropbox, Monday.com, MURAL, My5 (UK), PlutoTV, and Slack. In the “near future,” the company says others will be able to submit their web apps for distribution through the Quest store or App Lab.

Oculus says that apps delivered this way can also make use of WebXR, meaning they can easily go from being flat apps to fully immersive experiences. One could imagine a 360 video player that has a flat video browser but can then play back the video in a sphere around the user.

While WebXR apps could, of course, always be accessed through the Oculus Browser, putting them in the Quest store could make them much easier for users to find—an issue which has plagued WebXR content since its inception. Having them in the headset’s usual app library will also make WebXR apps feel much closer to ‘first-class citizens’ compared to native apps.

Given that Oculus will be hosting these apps in the Quest store or App Lab, we expect that any web apps distributed this way will be subject to the same content and technical guidelines required of native apps. But hey, they can’t stop you from pulling up even “rejected” web apps right through the Oculus Browser.

The post Web & WebXR Apps Are Coming to the Quest Store With Multitasking appeared first on Road to VR.

from Road to VR https://ift.tt/3CpRCkq
via IFTTT

Quest’s Home Environment is Finally Getting Social Capabilities, Becomes ‘Horizon Home’

https://ift.tt/315xPZH

One of our longstanding critiques of Quest’s social features is the lack of a simple, first-party place to meet up with friends. Finally that’s set to change thanks to ‘Horizon Home’ which will enhance the existing Quest home environment with social features, allowing you to invite party members to visit, hang out, or launch into VR apps together.

For a long while now its been possible to invite Oculus friends on Quest to a party to easily voice chat with them, but not being able to simply appear in the same space as them (at least, not without both having the same third-party app) seems like a huge miss considering the power of VR to connect people at a distance.

That’s finally due to change “soon” when Oculus introduces Horizon Home which will finally imbue the Quest home environment with social features.

For now the company seems to indicate that Horizon Home won’t really look much different than the existing home environments on Quest, except for the underlying addition of friends being able to join the space, and some capabilities like the ability to watch videos and launch from the space into VR apps together.

But in the future Oculus says users will be able to customize their home. That feature is “a little further out,” and while the company hasn’t said much about what kind of customizations will be supported, it sounds a lot like the Oculus Rooms app which was available on Oculus Go and Gear VR.

The Horizon Home naming scheme aligns with Facebook’s newly envisioned branding for their metaverse-y experiences, alongside Horizon Worlds, Horizon Workrooms, and now Horizon Venues (a slight rebrand of the company’s Venues social app). The company says users can expect that Horizon experiences will feature their familiar avatars and broadly include similar interfaces and interactions for a cohesive experience between apps.

More Social Improvements on Oculus Quest

Image courtesy Oculus

In addition to the Quest home space finally getting social features, Oculus today announced a handful of other improvements that will hopefully make the Quest social experience more seamless.

For one, the company says that social invites will appear in more places (your headset, your smartphone, and even the web), making it easier for people to see when they’ve been invited to do something in VR regardless of where they are. Similarly, Facebook Messenger voice calls are coming to Quest, making it easy to voice chat with friends whether they’re in-headset or out.

Oculus is also giving developers more tools to make social actions easier to integrate into their games. For instance, a new ‘Ask to Join’ feature will allow users outside of a game to ask a friend (that is already inside of a game) to join the session. Additionally, developers will be able to build invite functionality directly into their app’s interface (if they choose), instead of asking players to use the Quest menu.

Along with the recently introduced ability to create a VR sessions that’s sharable and joinable from a simple web link, these new capabilities, if fully adopted, ought to go a long way toward making the Quest social experience more complete and easier to use.

The post Quest’s Home Environment is Finally Getting Social Capabilities, Becomes ‘Horizon Home’ appeared first on Road to VR.

from Road to VR https://ift.tt/3Cs9LxX
via IFTTT

Facebook’s Latest Avatar System is Finally Rolling Out to All Quest Developers in December

https://ift.tt/3aFIIUb

At Facebook Connect today the company announced it’s finally making the SDK for its latest Oculus Avatars available starting in December, which will allow any developer to integrate the company’s avatars into their third-party apps and games.

Facebook began rolling out its latest avatars in the Oculus Avatars 2.0 update in March, which creates new default avatars for the company’s first-party social VR platform, Facebook Horizon.

In addition to being more lifelike and visually appealing than its prior releases, Avatars 2.0 is also positioned to unify the Oculus and Facebook ecosystems somewhat by bringing them to the full swath of Facebook properties including the Facebook app, Messenger, Instagram, and more.

Over the past year a number of third-party applications have worked with Facebook to adopt the system. You can already see the new avatars in Epic Roller Coasters, PokerStars VR, and Topgolf with Pro Putt, Synth Riders and ForeVR Bowling. This SDK release will allow all Quest developers to do the same thing.

In comparison to previous avatar systems created for the Oculus platform, Avatars 2.0 offers up more possibilities for customization, including customizable skin tone, hair style, face shape/markings/lines, eye shape, eyebrows, eye makeup, and more. You can also choose clothing, glasses, and body types—something Facebook says makes for one quintillion possible combinations.

The company hasn’t said exactly when to expect the Oculus Avatar 2.0 SDK, although we’ll be glued to the company’s developer blog then to give you the heads up.

The post Facebook’s Latest Avatar System is Finally Rolling Out to All Quest Developers in December appeared first on Road to VR.

from Road to VR https://ift.tt/3vS0Fbs
via IFTTT

Blade And Sorcery Is Coming To Oculus Quest 2

https://ift.tt/eA8V8J

Warp Frog’s VR hit, Blade And Sorcery, is coming to Oculus Quest 2 next month.

The physics-driven VR combat simulator launches on the standalone headset on November 4 for $19.99 as a separate, standalone version called Blade & Sorcery: Nomad. The game still features a sandbox mode in which you can experiment with both melee and magic-based combat in hilarious and often very violent ways. Blade And Sorcery was one of the first games to really explore what physics-driven VR combat could feel like, and it’s still a hugely popular release on SteamVR as a result.

Nomad will also include the new Dungeons mode, which was recently added to the early access version of the game on PC as part of a free update. This is a more linear mode in which you visit randomized rooms and clear out enemies. Currently, this mode is in its early stages on PC, with more options set to be added in future updates.

Blade And Sorcery was already rumored to be coming to Quest 2, with Warp Frog noting that, if it happened, it would likely be exclusive to Quest 2. The most recent standalone headset is more powerful than the original Quest, and other games like the recently-released Resident Evil 4 are exclusive to the new headset too.

What do you make of Blade And Sorcery: Nomad coming to Oculus Quest 2? Let us know in the comments below!

from UploadVR https://ift.tt/3moyIEZ
via IFTTT

Horizon Workrooms Will Let You Customize The Room Later This Year

https://ift.tt/30ZHxN9

Horizon Workrooms is set to expand, offering users the ability to customize rooms and add custom logos and graphics. 

These new features won’t be available until later in the year, but will let users add some personal touches to the Workrooms, including a custom logo graphic on the wall and custom posters. Users will also be able to switch between “a wider variety of environments”, one of which is pictured below.

In its current state, Horizon Workrooms only offers one room with no customization beyond the seating arrangement. Users can only choose between conversation, boardroom table and presentation settings, which alter the direction of the seats but nothing about the room itself. 

On a call earlier this week Facebook’s Product Management Lead, AV/VR for Enterprise Jill Campbell said that these new customization options are only the beginning of future personalization plans for Workrooms. “This is one step we’re going to take in that direction with lots more to come.”

The image embedded above shows a brand new environment set among city skyscrapers, while the image below shows the default countryside setting that launched with Workrooms. The images also show the custom logo and poster integration, exampled here by Zoom (whose video conferencing service is soon to receive native integration with Workrooms). 

Workrooms launched earlier this year as Facebook’s remote work and collaboration solution for VR. It allows teams to meet in VR and work together with a sense of presence as if everyone is in one room. It achieves this by harnessing a combination of existing and emerging technologies, such as spatial audio, keyboard and hand tracking, passthrough, remote desktop and Oculus Avatars. 

from UploadVR https://ift.tt/2ZruqUi
via IFTTT