When it comes to hand tracking games on Quest, nothing really comes close to Unplugged.
Developed by Anotherway and published by Vertigo Games in late 2021, Unplugged is an air guitar game, inspired by Guitar Hero and many others, that lets you shred in VR with a virtual guitar and your real hands.
As I’ve said elsewhere, Unplugged leverages Quest’s hand tracking technology to breathe life into the imaginary act of air guitar. In doing so, it takes hand tracking to a whole new conceptual and technological level, surpassing everything else available on Quest.
“From the very beginning, our obsession was to understand how the technology is limited and try to polish that stuff,” says studio director and Unplugged creator Ricardo Acosta. “That was the very first thing. Not the graphics, not even the gameplay.”
After speaking with Acosta in our virtual studio (full video interview embedded above), it’s clear that creating a polished and tangible experience was always the goal. “I think that hand tracking is here for good,” he tells me. “I wanted to create something that worked for real. It wasn’t just another demo.”
Such strong commitment to this new form of input is a big call, especially for Acosta, who spent years as a hand tracking skeptic while working on the HoloLens team at Microsoft. “When I was at Microsoft, I was like an advocate for controllers,” he says with a laugh. “At Microsoft, they are all about hand tracking, but I was like, ‘No guys, we need controllers. Controllers are great.’ And now I’m saying the exact opposite thing.”
“On the first version of the HoloLens … you have hand tracking, but just like the blob. It’s just the hand, not the fingers.” Without full and reliable finger tracking, Acosta came away disappointed and skeptical. “With the HoloLens 2, it was a bit better, but the lag between your movement and the hand was very big, for a lot of technical reasons.”
Even so, Unplugged was first conceptualized in 2015 — well before the advent of any modern VR’s hand tracking functionality. “I remember being in a concert in Prague and I was just like doing air guitar,” he recalls. “And at some point I was like, oh, this is an interaction that could work in VR.”
“As soon as I went back home, I prototyped something … and it totally worked. It was like, oh, this is good. This is something that we could actually turn into a game.” The original idea developed into something akin to Rock Band but for VR, using controllers and the first Vive headsets and Oculus SDKs. Acosta said he quit his job at Microsoft to work on the prototype, titled Rock the Stage, over the course of four months.
“I think that it was pretty good,” he says of the Rock the Stage prototype, of which videos still exist online. “The best thing it was that it made you feel like you were there.” But Acosta soon ran into a road bump — music games, and particularly the associated licensing, are complicated work. “You need a lot of money. You need a team of people handling all that music licensing. And I didn’t have all that back in the day. So I decided, at some point, to go back to my job.”
After continuing in Microsoft’s VR/AR division for another few years, Acosta revisited the concept in 2020 while bored at home during the pandemic. “Oculus [had] just released the hand tracking system [for Quest] and suddenly it came to me like, ‘Oh my god, I could actually rescue that…prototype and try [see] if it works using hand tracking.'”
Even in the early stages, hand tracking felt like a turning point for the previously controller-only experience. “It worked so well. . .Back in the day with the controllers was nice, but with hand tracking was exactly what it should be.” Acosta adapted his original prototype into something new, ditching controllers for something much more freeing and immersive. “When I put [hand tracking] on the prototype, it wasn’t perfect, but it was good enough for me to start polishing the experience. I knew that with a bit of work and a few algorithms on top of the hand tracking, I could make it work.”
Acosta created a video showcasing the new prototype game and posted it to social media. It soon exploded and attracted a lot of interest, especially from publishers offering funding. After discussions options with a few different publishers, Acosta signed with Vertigo Games. “They offered the best deal. And also they were bigger, and they really had a super nice vision about what the game should be.”
“At first I was a bit scared about it, because it was a super big project. We didn’t have a company together. It was complicated.” What started as a one-man show had to turn into a burgeoning team. Acosta’s wife joined as a project manager and they were then joined by a few others to make up the small studio now known as Anotherway.
“We are six people now, which is not a lot,” he says. “Very recently, we had the opportunity to grow a little bit, but we decided to stay small. I’ve been working in Microsoft for most of my career. That is a very big company and it’s amazing, but I really like working with just a very small amount of people. It’s a very creative environment.”
Working alongside Vertigo, Unplugged quickly developed into a project with bigger ambitions than Acosta had ever imagined. “I’m very conservative in terms of adding features, because I know that anything you add to a project, it will create a lot of problems, a lot of bugs, a lot of things.”
“They pushed for more staff. They wanted more music, they wanted more venues, they wanted more quality on the game and they’ve been always pushing for that. And I think that, in general, the game would have been way smaller without Vertigo,” he says.
In particular, working with Vertigo opened up opportunities when it came to the proposed tracklist. “In the very beginning we were just going for small bands. And then when we signed up with Vertigo they were like ‘No, like indie bands are cool and we will have a few. But we need famous bands.’ And we were like, oh, but that’s going to be super complicated.”
Vertigo sent Anotherway a Spotify playlist and asked them to add any songs they might want in the game. “And we were like ‘Wait, whatever music?'” It was a big mental shift.
The Offspring’s The Kids Aren’t Alright was the first major song that Vertigo and Anotherway secured the rights to. “We were just like jumping, like, ‘Oh my god, we made it.'” The final selection included some massive artists — The Clash, T. Rex, Weezer and Steel Panther, to name a few. “[Music licensing] is a very time-consuming process, and I knew that. So not even in my wildest dreams I would have dreamed about having Weezer or Tenacious D, The Offspring, or Ozzy…”
The inclusion of Tenacious D’s Roadie is particularly special to Acosta — not only is the band one of his favorites, but he had used the song all the way back in 2015 in the very first prototype. However, the song almost didn’t make it into the final game at all.
Vertigo and Anotherway initially struggled to make contact with Tenacious D to secure the rights. However, Vertigo had a trick up its sleeve — Guitar Hero legend Mark Henderson had been brought on board to assist with the game. “He was like, ‘Guys, leave it up to me. I’ll make it happen.’ So somehow he contacted the manager of Tenacious D and started talking to them.”
With Henderson’s help the rights to the song were secured. But another problem emerged — with a PEGI 12 rating, Roadie’s explicit and frequent F-bombs weren’t going to cut it. “So at another point we were like, ‘Okay, we have the song now, but we cannot use it because we are PEGI 12, so we have to take it out from the list.'”
Acosta made his peace with leaving the song off the tracklist but, in his words, “maybe the stars were in a particular position that night.” Henderson was able to get Tenacious D back into the studio to re-record a clean version of Roadie, specifically for Unplugged, excluding all the swearing.
“It was insane,” says Acosta. “Knowing that my favorite band re-recorded a song just for the game. It’s insane. It’s just amazing. And a lot of people have complained about the fact that it’s a different version of the song, without the swearing. But I’m so proud of that. To me, it’s even better because it’s our song.”
With a solid tracklist secured, Acosta and the team at Anotherway set to work on creating an unforgettable and reliable hand tracking experience. “I am a UX designer, so for me, the main important thing on anything is user experience. If the experience is not good, the whole game won’t work, or the whole experience will be shit, and we didn’t want that.”
As a result, the gameplay itself was adapted and designed to work with, not against, hand tracking. Even tiny changes mad a big difference — the size of the guitar in Unplugged, for example, is a bit smaller than a regular, real-life guitar, which helps keep your hands in view of the cameras.
“In the beginning, with hand tracking 1.0, we had to be very aware of your movements,” he explains. “We had to create the mapping so that the music charts in a way that is always aware of the limitations of the technology.”
That meant that at launch, the mapping in Unplugged didn’t always completely follow the music, leading some players to complain that the music and the notes didn’t always line up. “And we knew why, but we couldn’t do anything about it, because the hand tracking was very limited and you couldn’t move your hand that quickly,” he said.
Nonetheless, Acosta remains proud of the experience offered at launch. “In the first version, it was absolutely playable. Obviously it wasn’t perfect, but it was playable. And I think that we proved that you can actually create a hand tracking game that uses hand tracking very intensively.”
Skip forward a few months after launch and the release of Meta’s Hand Tracking 2.0 software offered huge gains for Unplugged. Not only was the technology more reliable than ever, but it was so good that Anotherway went back and re-mapped the entire tracklist for increased accuracy and challenge. “We want the game to be fully accessible for everyone, obviously. But I think that for 98% of people, the game works very well.”
Nonetheless, Anotherway are still implementing algorithms and workarounds to account for error and improve the experience — the latest being an AI system. “We’re using deep learning in order to see where your hands should be or what’s your pose or what’s your intentions. We made all that stuff so [that] when there is a problem with the hand tracking, there is another layer trying to help and trying to make the experience as smooth as possible.”
There’s more to come too. In the short term, Anotherway just released a new DLC pack — featuring songs by metal band Pantera — and are working on an upcoming accessibility update adding new features and “another thing” that is top secret but will be “really big.”
In terms of song selection, there’s definitely more on the way. “We are working to add more music all the time. We want to add free music [as well], not just DLC. Also, I want to add more indie music because I think that there is a lot of really good indie music out there.”
But what about the long term? What does the next year or more look like for Unplugged? “I cannot talk too much about it because Vertigo will kill me,” Acosta says with a laugh. “But our plans are very big. Unplugged is going to become bigger, at least in terms of features…”
“I would be very excited about Unplugged if I knew what’s going to happen. Probably like in a year, Unplugged will be very different. It will have way more stuff. That’s it. That’s all I can say.”
For a game that has already pioneered a new technology on a cutting edge piece of hardware, there could be a lot of interesting developments in Anotherway’s future.
“Unplugged is going to move forward,” Acosta said. “That is for sure. We are not staying still.”
Unplugged is available on Quest headsets and hand tracking-enabled PC VR headsets on Steam. You can read our full and updated 2022 of the game here.
from UploadVR https://ift.tt/r7yODzV
via IFTTT
No comments:
Post a Comment