Evolution of VR controllers: from finger tracking to hand tracking

As we all know, AR/VR is a product with strong interactive form, and the most widely used peripheral with intuitive interaction is the VR controller. Its interaction methods are mainly reflected in: 6DoF positioning tracking, button/touch interaction, tactile/vibration feedback, and finger track.

Since consumer VR began to gain popularity in 2016, the interaction of consumer VR controllers has not been significantly upgraded. Although a number of VR glove products have been born, there are basically none that can achieve consumer-level applications. They are expensive and large in size with good experience. Moreover, there is no unified way to develop VR gloves as third-party peripherals other than VR headsets. Restricted by standards, the ecology is extremely chaotic.

As for the differences in VR controllers, one of them is that Oculus Touch has built-in capacitive sensors that can simulate the pinching action of the thumb and index finger, but this is not perfect enough, and at the same time, its other products have almost no such function.

But it is undeniable that finger tracking does make the VR experience better.

Until the Valve Index controller was released in May 2019, it can be said that it officially brought the five-finger tracking function to the mass consumer-level VR peripherals. Currently, the Index five-finger tracking controller is suitable for any VR headset of the SteamVR solution, mainly Index and HTC products.

Compared with Oculus Touch, the Index VR handle can provide a complete five-finger tracking mode, which is subdivided into the movements of each finger, and it is very smooth. This will make the experience of grasping in VR more immersive, and it can also Make the original VR viewing interactive experience richer and more interesting, and the experience is greatly improved.

But this is not enough. We know that Oculus Quest already provides hand tracking function. Although it is still in the beta testing stage, its stability and user-friendliness are better than many official products.

Currently, the gesture recognition function in Oculus Quest has been integrated into the system, and the system will automatically switch to gesture control mode when the controller is not connected. In some third-party hand tracking test apps, we can see that the Quest hand tracking effect is extremely smooth. Even if the hands have overlapping areas, the finger movements can basically be accurately restored.

More interesting VR interaction methods: wristband/bracelet

It is reported that the gesture technology in Quest originally originated from the Facebook Reality Labs department (hereinafter referred to as: FRL), which focuses on the research and development of VR technology and hardware. This department was the earliest Oculus Research. It was later renamed after Facebook's strategic adjustment was divided into the AR department and the Protal department. Qingting.com has previously reported on other research conducted by the FRL department, such as tactile feedback wristbands, VR gloves and input delay, virtual avatar systems, etc.

Today we are going to talk about a “smart bracelet” from the FRL department: Tasbi. Of course, strictly speaking, it is a tactile feedback wristband. This project was first announced at the Visual Tactile Conference in July 2019. Recently, Facebook published a detailed research paper on the solution. The biggest improvement is the addition of two-handed mode.

So you may have questions, since hand tracking is available and it works, do you still need a wristband? Of course it's possible.

But this is not enough. We know that Oculus Quest already provides hand tracking function. Although it is still in the beta testing stage, its stability and user-friendliness are better than many official products.

Currently, the gesture recognition function in Oculus Quest has been integrated into the system, and the system will automatically switch to gesture control mode when the controller is not connected. In some third-party hand tracking test apps, we can see that the Quest hand tracking effect is extremely smooth. Even if the hands have overlapping areas, the finger movements can basically be accurately restored.

More interesting VR interaction methods: wristband/bracelet

It is reported that the gesture technology in Quest originally originated from the Facebook Reality Labs department (hereinafter referred to as: FRL), which focuses on the research and development of VR technology and hardware. This department was the earliest Oculus Research. It was later renamed after Facebook's strategic adjustment was divided into the AR department and the Protal department. Qingting.com has previously reported on other research conducted by the FRL department, such as tactile feedback wristbands, VR gloves and input delay, virtual avatar systems, etc.

Today we are going to talk about a “smart bracelet” from the FRL department: Tasbi. Of course, strictly speaking, it is a tactile feedback wristband. This project was first announced at the Visual Tactile Conference in July 2019. Recently, Facebook published a detailed research paper on the solution. The biggest improvement is the addition of two-handed mode.

So you may have questions, since hand tracking is available and it works, do you still need a wristband? Of course it's possible.

According to FRL research results, combining the three sensory feedbacks of vibration feedback, pressure feedback, and visual feedback can bring a more immersive and realistic experience to AR/VR content.

In addition, Facebook also acquired CTRL-Labs, a brain-computer interface solution provider, in September last year. The latter's core advantage is to implement hand tracking functions through wrist-worn devices based on electrical pulse signals. There is no doubt that the goal of future Facebook research and development is to realize AR/VR brain control or other interactions non-invasively, and there is a huge room for imagination.

It can be seen that the Tasbi wristband provides more possibilities for future AR/VR interactions, especially after the popularization of optical hand tracking solutions such as Oculus Quest, there is indeed a certain gap in the experience without a handle, and combining it with a wristband may It is the way of experience in the future, and it is even conceivable that Quest 2 will launch a version with a single head display and no handle, so that consumers can choose by themselves.

Of course, Tasbi is still in the research stage and currently has many shortcomings. For example, the demonstration showed that it needs to be connected with multiple cables, and has not yet reached the consumer-level trend of wirelessization; at the same time, the current Tasbi entry and exit experience still requires a short period of calibration and cannot be used out of the box, and needs to be optimized.