Every Friday afternoon at AppAttic we have experimental hack time, which basically means we get to work on whatever we want, so long as it has some potential benefit to the company. During October and November we’ve been experimenting with VR technology, controllers and tracking movement. Many Friday afternoons have been spent crawling around the office floor trying to pick up small virtual boxes, much to the amusement of our office buddies. Read on, and check out the video at the end of the post to see what we hacked together.

Friday afternoon's at AppAttic!

Friday afternoon’s on the floor at AppAttic!

Source: SlashGear

HTC Vive

Headsets

Oculus Rift with the new Oculus Touch Controller

Oculus Rift with the new Oculus Touch Controller

We’ve been using with the HTC Vive which comes with 2 hand controllers and have been fortunate enough to experiment with the latest Oculus Rift and their even newer Touch controllers. The jury’s still out as to whether VR is going to see a Apple/Windows divide, but one thing is for sure; there are going to be a lot more options on the market next year.

During the first couple of weeks we also experimented with a few mobile VR headsets, including the Samsung Gear VR, Google Cardboard and the Merge VR but decided the HTC Vive was the most suitable option for our first experiment.

So after playing with the tech (and a variety of games of course – hey, it’s research!) we got to work experimenting and seeing what we could hack together and what we could track.  

Using Unity made the first steps a fairly simple process; we got a basic HTC Vive demo up and running in no time.

Tracking and Controlling

We wanted to track movement and muscle activity, ideally with something that was simple to use so we opted for the Myo EMG band that would not only provide real-time data, but potentially enable us to control the game in an AR environment* (we couldn’t use the Myo alone within a VR environment…because the person playing is fully immersed and cannot see their arms or hands. AR, yey! VR, ney!). Anyway, the Myo is super-easy to use; no clips or clasps – just slide it on.

We then added a Mio Fuse for realtime tracking of heart rate.

Myo Band (EMG) with the Mio Fuse (heart rate)

Myo Band (EMG) with the Mio Fuse (heart rate)

* AR is most certainly on our agenda, however with HoloLens being $3,000 USD and the only current viable option, we decided to stick with VR for now. A new wave of much more affordable AR headsets are due in 2017 so watch this space!

Meanwhile we needed to control gameplay whilst in VR, that meant using the HTC Vive controllers or something else.

The Vive controllers are great, but we wanted to experiment and so opted for the Leap Motion, a device which we could strap to the headset and which would allow the player to see their ‘own’ hands within the game. Unlike the Kinect, the Leap Motion provides much more granular gesture control at close proximity, such as the ability to grab and pinch and a high frame rate as required for VR.

Bringing it all together…

As we entered November it was time to pull our chosen tech together and what we – well, mainly Richard Gillen to be fair – managed to come up with was rather promising; a basic VR demo which collects, processes and displays muscle movement via EMG data in real time and is controlled using the Leap Motion (ie. the player is not required to hold any physical controller).

Leap Motion control and Myo EMG visualised in VR

Leap Motion (control without physical controllers) and Myo EMG visualised in VR

Watch the video here:


Leap Motion with Myo EMG graph charting real time in HTC Vive VR / Virtual Reality from AppAttic on Vimeo.

A few take-aways

  • The Oculus Rift with the Touch controllers is a much lighter and more portable solution than the HTC Vive, however it did not feel as immersive. The Oculus headset was not such a snug fit and the sound from the inbuilt headphones was far inferior to the £10 set we use with the Vive.
  • The controllers for the Rift and Vive are very different. We didn’t have enough time to make a concrete decision about which we preferred but we did conclude that any decision we might make could easily change depending on the game. Ideally the controller should resemble the in-game device so as to increase the immersiveness. We imagine a range of innovative controllers emerging in 2017.
  • Both headsets would be much more user-friendly without the Predator-esq wires coming out of the back of your head (there are harness available for dealing with these cables and HTC Vive have a battery add-on for tethered use – Oculus also coming soon we hope!)
  • A high-end mobile headset (we tested the Samsung Gear VR and Merge VR) combined with a quality mobile phone (we tested the Samsung Galaxy S6 and the iPhone 6) was pretty impressive. Nothing compared to the Vive/Rift but surprisingly good when high quality material was viewed. The problem is that there is a lot of low quality material available for mobile VR and the likes of Youtube and Facebook apply a lot of compression to 360 videos. Although we are very interested in testing the Google Daydream which has just become available.
  • Whilst the Leap Motion is promising, lack of any physical feedback felt odd. More experimentation will be required.
  • The Myo provided data with relative ease, our next consideration is to test it on persons with rehabilitation requirements, whereby the EMG signals may be weaker than ours.

Overall this was a great hack that gave us some insight as to what is possible in terms of getting several pieces of commercial tech to work together, however there’s a long way to go in terms creating a quality and engaging user experience.  We’re looking forward to 2017 and all the new goodies yet to come.

Forthcoming Hacks

Our next plan is to incorporate smart toys into our hack. Something like the Sphero, ideally in an AR environment where the ‘player’ can interact with both virtual and physical elements that respond. We just have to wait for a wave of affordable consumer AR headsets. C’mon Santa, you know you want to!

Sphero - smart toys for rehab?

Image Source: Sphero.com