How Virtual Reality Motion Tracking Works

The virtual reality field is rapidly becoming one of the most fascinating fields of technology. For most people who use VR gadgets regularly, the immersive experience that comes with it is indescribable! If you are into VR, you would likely agree with those words.

Virtual reality motion tracking needs to be understood on how it works. To understand how it works, it would be easier if you own some basic VR gadgets. This equipment is crucial to the immersion you experience.

The most important gadget being the VR headset, other vital gadgets are a pair of controllers and a good audio device.

These are the things you need at the basic level.

You can’t be an avid user of VR without ever coming across these two words “motion tracking”. Do you understand how motion tracking works in virtual reality?

Without much ado, let’s delve into it.

CONTENTS:

• What is motion tracking
• How does motion tracking work
• Active optical tracking
• Non-optical methods of motion tracking in VR
• Degrees of freedom (DoF): 3-DoF vs 6-DoF for VR headset selection
• Which one should I buy? 3-DoF or 6-DoF headset
• The tracking system of various virtual reality headsets
• How to deal with VR motion sickness
• What does the future hold for VR motion tracking

What is motion tracking?

In simple terms, motion tracking has to do with tracking the movement of objects within a given space and transmitting the data to a computer system for detailed processing. Motion tracking is essential to feel the immersion that comes with VR. Without it, you would become stuck in the virtual reality world, virtually unable to do anything.

Motion tracking can at times include recording object motions and conforming it with already stored data.

This innovation has been applied to different fields such as the military, entertainment, sports, and so on.

In some circles, motion tracking is equally known as motion capture. The usefulness of motion tracking is not only employed in virtual reality. It is also used in the video-making industry to track the exact location of an object with the use of a camera.

How does motion tracking work?

To understand the basics of motion tracking, we need to first comprehend the concept of “degree of freedom (DoF)”. You might be wondering what is the degree of freedom. The degree of freedom is the concept that makes real-life movements possible in the virtual reality world. On the other hand, DoF could as well mean the various ways a particular object can move through 3D space.

Furthermore, an object is only capable of moving only in 6 total directions:

• 3 translational movement (movement around x y z axis)
• 3 rotational movements (forward or backward, left or right, up or down movement)

For example, the human body can move in three directions on 3 perpendicular axes. This sort of movement is otherwise called 3-DoF. When translational movement is coupled with rotational movement, it gives rise to 6-DoF. The term 6-DoF is essential to provide a quality and complete motion tracking system.

The ultimate goal of virtual reality is to make what we do in real life seemingly possible. For instance, when driving a car, it is normal to place your hands on the steering to control the car. The goal is to mimic the same action in the virtual world. The level of immersion experienced with virtual reality depends on various factors. One of the key factors is the quality of motion tracking.

Let’s delve into the basics of motion tracking. Motion can be tracked in a variety of ways. For us not to complicate it, we’ll classify it into two major ways: optical and non-optical tracking.

Optical Tracking in VR

In simple terms, optical tracking detects the position of an object or person using visual information.

In virtual reality, different cameras are placed on the headset to monitor the position and direction based on the processing and analysis done by a computer.

In optical tracking, the individual being tracked in most cases has optical markers. The optical marker system involves employing the use of video technology to monitor the person or object being tracked.

Markerless tracking is not as complex as using markers to track. You don’t need any pre-placed targets, features of the natural surroundings are used to detect position and orientation.

For optical tracking to be done, markers are required.

A variety of markers can be used. The markers can either be visible or invisible. An example of a visible marker is QR Codes. But from observation, we noticed that the use of infrared light is normally chosen as a marker.

Tracking with markers involves setting up a known target point to serve as a point of reference.

The cameras target these markers and then employ the use of different algorithms to find out the exact location of the object.

This can become problematic when two cameras stationed at two different angles see different dots or markers but think they are both the same.

Of course, this leads to poor motion tracking and inaccurate tracking.

In broad terms, markers can be classified into passive and active markers. Passive markers are also called reflective markers.

We have another type of marker called an active marker. Passive markers only reflect light, while active markers are computer-directed LEDs that make room for accuracy and upgrade over the passive marker system.

On a general note, active markers are better and perform well over passive ones.

However, there are still some obvious limitations to the active marker methods. The most obvious limitation is that the actor has to put on a power supply or be connected to the system in a way. This can be uncomfortable.

Active optical tracking

Active optical tracking is quite expensive. It is not easy producing it on a grand scale for commercial purposes.

For example, when an actor’s body motion is needed to be captured for the sake of making animations, the actor’s body is covered with a lot of markers. But this is not sustainable in the commercial world, virtual reality companies use only a few strategic markers or sometimes no markers at all for their products.

One good example of active tracking in a consumer gadget is the PlayStation Move controller.

This controller has a bulbous and glowing sphere at the top of it. The bulbous part on top of this device works in tandem with Sony’s Playstation Eye camera.

The ball allows for 3D tracking in real-time. There’s an RGB LED inside the ball.

The main role of the ball is to provide a visual reference for the camera. The PlayStation Eye (camera) has been programmed to detect the exact shape and size of the ball.

Immediately the ball on the controller is detectable to the camera, the exact location of the ball is affirmed in 3D space. Tracking the shape and size of the ball helps the Playstation camera to know the exact location of the ball at all times.

Microsoft Kinect is a pacesetter among consumer motion tracking devices. successful motion. It is primarily made for the Xbox 360 and one console.

Quite a few Windows computers can as well perform complex motion tracking without employing the use of any markers at all.

The inbuilt camera of the computer only needs to locate the position of the subject. The newest version of the Microsoft Kinect can track a maximum of 6 people at the same time. Microsoft took its time to use intelligent software methods integrated with unique infrared and high-sensor cameras. Despite the feats of the Microsoft Kinect system, it still can’t be compared with the precision offered by expert marker-based systems.

The Leap Motion is another wonderful illustration of the marker-free tracking system. This system does not make use of full-body tracking.

The Leap motion produces a high-resolution scan of objects in real-time.

When the leap motion system is applied in the virtual reality context, this is done by attaching the leap motion device to the front of a head-mounted display.

The attached device then creates a digital rendition of the subject’s hands which automatically makes room for interaction.

Commercial virtual reality has not attained the level in which full-body tracking would compete with professional systems in the aspects of movement freedom and precision & accuracy.

Most virtual reality products we have available for consumers today are still below par.

The VR gadget that comes close to offering the Leap Motion experience is the Oculus Rift.

This virtual reality headset still doesn’t offer the perfect leap motion experience. The Manufacturers of this product recommend not standing up and wandering around while wearing the headset. Because you may likely fall and injure yourself.

The Lighthouse tracking system is a laser-based inside-out positional tracking system manufactured by Valve for SteamVR and HTC Vive.

This tracking system tracks the position and orientation of the user’s head-mounted display and controllers in real-time.

This tracking system gives users the freedom to move in any direction and reorient themselves in any position within the possible range of the SteamVR base stations. It is the technology that allowed SteamVR to produce a full-room experience in virtual reality. The Base stations are vital to the Lighthouse tracking system.

The Base stations are portable rectangular objects placed in tracking areas.

The purpose of the Base stations is to serve as central points for tracked devices such as controllers and head-mounted displays.

Base stations serve as a reference point by engulfing the whole room with non-visible light. The sensors on the tracked devices intercept the non-visible light, the base station would then be able to detect the location of these devices. Multiple Base Stations allow the tracked devices to figure out where they are in the 3D space.

The major drawback for Base stations is that they are highly susceptible to occlusion. To deal with this issue, two Base stations can be placed at strategic points of the room. Usually, multiple Base stations are needed to improve and increase the tracking span. Valve has promised to make this technology revolutionary and available to all hardware manufacturers. We can only wait!

One distinct advantage of optical tracking is that it is not too expensive to set up. The major issue is that it’s not easy to calibrate. When using optical tracking, it is vital not to obstruct the flow of light, otherwise, the right data may not be received.

Non-optical methods of motion tracking in VR

This method of tracking does not make use of cameras. This method of motion tracking makes use of micro-electromechanical sensors such as accelerometers, gyroscopes, and magnetometers.

For example, the accelerometers measure linear acceleration. For acceleration to be measured, time and velocity must be taken into account.

The output of the accelerometer could be integrated to find the velocity and then integrated again to find the position relative to some initial point. In simple terms, accelerometers measure movement along XYZ axes.

Gyroscopes on the other hand measure angular velocity. Angular velocity can be integrated as well to determine angular position relative to the initial point. Gyroscopes are always used for rotational tracking 360-degree rotation.

A magnetometer measures the strength and direction of the magnet field. It acts as a compass by detecting magnetic North.

The Google cardboard for instance uses a magnetometer, a magnet ring moves on top of another magnet in a downward and upward manner. The fluctuation in the field is then recorded as a button click.

The science behind magnetometers can be applied in different ways. Some devices use electromagnets while others use permanent magnets. Regardless of the type of magnet used, when a magnetic field disturbs the material inside a magnetometer, changes occur. The direction and magnitude can then be measured.

These various electromechanical sensors were very expensive before the giant stride of development in industries such as aeronautics, automobile, and computer fields happened. They are now more affordable and smaller in size compared to the older versions.

The revolution in the field of science and technology has also accelerated the development of these sensors.

The mobile device market has driven up the demands for them. It is undeniable that the relative success of the VR industry today owes its success to technologies primarily designed for mobile devices such as tablets and smartphones.

When a device employs the use of these three sensors altogether, it creates a device with high precision motion data and low latency. The three sensors can be used together with the optical method of motion tracking.

If you want a premium motion tracking experience, infrared tracking or passive reflector tracking is simply the best. Moreover, these sensors can work well on their own without being connected with an optical camera for motion tracking.

Various virtual reality headsets available prove this to be true.

Direct electro-mechanical sensor: this is another example of non-optical tracking technology. The sensor primarily detects body parts movement. Some VR gadgets developers are investing quite a lot of resources in this technology.

An example of this is VR gloves and some other haptic products. All haptic VR products make use of a direct electro-mechanical motion tracker. Take for example, when you wear a VR glove and bend your fingers, the sensors inbuilt in the glove automatically get activated.

The sensors in the gloves somehow interpret these hand movements and then convert the hand movement into electrical signals. A good example of this type of glove is GloveOne. Additionally, Salto is another wonderful choice.

Another promising approach that can be applied to virtual reality sometimes in the future is the Myo armband. It has to do with placing a device called Myo around the arm. After being tied, the Myo deciphers the electrical impulses from muscles in the upper limb for movement-based controls of computer software. One unique advantage of the Myo device is that it is easy to use and is not obstructive.

The Myo device is meant to be worn all day. As earlier explained, this technology could be extended to virtual reality in which direct sensing would be accurate and more precise. It’s a promising technology. It might later replace other tracking methods. We will surely keep an eye on it.

Mechanical Exoskeleton System: This motion tracking method is somewhat more mechanical than other methods. The mechanical exoskeleton system can be used for dual functions. It can be used to capture motion and at the same time to provide feedback. A classic example of a VR product that uses this system is the Gypsy motion capture suit.

This suit measures the user’s movements with potentiometers. Another example is Dexmo F2. It is a glove-like hand-worn exoskeleton that provides information about positional tracking, hand motion capture, and force feedback.

Omnidirectional treadmill: This is another type of tracker that tracks mechanical movements. The earliest form of this tracker was primarily made for soldiers to give them the freedom to run infinitely in any direction they choose. The consumer version of this mechanical movement tracker is now available for purchase.

The most popular VR tracker using this technology is Virtuix Omni. It is a VR treadmill you can choose to use in the comfort of your room. This VR tracker can detect body movements such as crouching, running, and jumping in a limited fashion.

Degrees of freedom (DoF): 3-DoF vs 6-DoF for VR headset selection

The degree of freedom is an essential concept of VR. You may not fully understand what the differences are between 3-DoF and 6-DoF headsets. We’ll highlight the differences between the two. Let’s start with the 3-DoF headset.

3-DoF Headset

In simple terms, a 3-DoF headset only allows the tracking of rotational motion but not translational. With a 3-DoF headset, the following are the things you can track:

• Looks right or left
• Rotation of the head ( up and down)
• Pivots right or left

A 3-DoF headset can never decipher whether a translational movement has occurred. The following are the examples of VR headsets making use of 3-DoF:

• Oculus Go
• Google cardboard
• Merge VR
• Samsung Gear
• Google Daydream

How does 3-DoF work?

The headsets making use of 3-DoF provide the basic form of subject tracking in VR. It functions using inbuilt sensors such as gyroscope, magnetometer, and accelerometers. Smartphones equally use these sensors to measure movement. It is not as sophisticated as the 6-DoF.

6-DoF headsets

6-DoF headsets function in a way that allows users to track translational and rotational movements at the same time. This kind of headset can determine whether users have changed the direction of their heads.

This type of VR motion tracking gives a whole lot of freedom to users. To a good extent, it helps users to perform their real-life activities in VR. The movements it can detect include:

• Up or down
• Forward or backward
• Laterally or vertically

Most modern VR headsets we have available today make use of this technology. Although full-body tracking can not be achieved by these headsets, they offer a good immersive VR experience. Examples of VR headsets with 6-DoF include:

• HTC Vive
• Oculus Quest 2
• Oculus Rift
• Windows Mixed Reality

How does 6-DoF tracking work?

We have various ways to achieve 6-DoF tracking. The pioneer versions of headsets with 6-DoF used positional tracking sensors. This involves placing two devices in strategic locations within the room to track the movement of the headset.

A typical example of this is the Oculus Rift which is integrated with infrared LEDs. The infrared LEDs are controlled by two 30cm tall sensors which are not far apart (1.5cm).

The modern versions of 6-DoF headsets have been upgraded over the older generation.

Modern 6-DoF inside-out tracking to achieve 6-DoF. You might be wondering what inside-out tracking is? Inside-out tracking is a method of positional tracking used in VR to track head-mounted displays (HMDs) and motion controller accessories.

The major difference between inside-out and outside-in tracking methods is the location of sensors and cameras.

For an inside-out display, the cameras and sensors are located on the device being tracked (e.g HMDs). But for outside-in tracking, the sensors are placed in a specific location.

The early 6-DoF headsets used the outside-in tracking method while the newer versions use the inside-in method.

Which one should I buy? 3-DoF or 6-DoF headset?

This can be somewhat tricky especially if you are new to virtual reality.

Many factors may be involved in your decision-making. But the key factors that should influence your decision are your needs (business and personal) and budget. Other factors are secondary. We’ll briefly highlight the strong points or the application of both the 3-DoF and 6-DoF headsets.

3-DoF

• Suitable for watching 360-degree videos or images
• Practicing giving a lecture at a conference.
• Viewing the interior of a house in VR before purchasing

6-DoF

• Suitable for action and adventure VR games.
• Driving simulator training the user on how to drive.
• Inspecting the engine of a vehicle.

The tracking system of various virtual reality headsets

Virtual reality is different from regular 3D displays in that they have a wide field of view and they are also tracked. Some VR headsets are currently equipped with positional VR tracking that enables users to bend, crouch, and even jump. We’ll explain the tracking system being used by some popular VR headsets.

Oculus Rift (formerly known as Constellation)

When Oculus began developing the Rift headset, they faced an enormous challenge. The challenge was providing a VR headset with a quality track that is similar to the one provided by OptiTrack but at a cheaper price. Oculus made various tracked devices that have a predefined “constellation” of infrared LEDs concealed under an external plastic. The user won’t be able to see the IR light because it’s invisible to the human eye.

The sensors (cameras) within the headset can sense the Infrared light. Frames of infrared light are then sent to the user’s computer via a USB cable at a frequency of 60 HZ. The PC analyzes each frame by locating the exact position of each infrared LED and the relative position of each object.

The software integrated with this tracking system can recognize each LED because it knows the exact shape of the “constellation”. The software can recall where the object was in the previous frame. The accelerometer helps it to know the direction of acceleration. Gyroscope helps with rotation. To enable the software to recognize each infrared LED appropriately, each IR LED gives a signal at a particular frequency to identify itself.

To support fast-tracking, the controllers and headset communicate wirelessly with a chip in the sensor when it is about time to be identified. Oculus Quest and Rift S no longer use Constellation for headset tracking. But it is still being used for onboard headset cameras to track controllers.

PlayStation VR

The Playstation VR headset also makes use of cameras. What makes the PlayStation VR headset unique is that it uses a visual light spectrum for tracking.

For example, the PlayStation 4 is equipped with two spaced cameras. These cameras are connected to the PlayStation. Both cameras use the image data to track the different light strips found on the headset and the controllers.

SLAM VR

Immediately after the Oculus Rift and HTC Vive were released, many potential buyers were put off by having to mount base stations or sensors in the room. So instead of placing base stations or sensors at a stationary location, some companies started using inside-out tracking methods for their products. This involves using inbuilt headset cameras to track the exact location of the headset using a computer vision algorithm.

The type of computer vision algorithm used is called SLAM which means “Simultaneous Location and Mapping”. The SLAM algorithm simply works by taking note of special static features in the room. It compares the data from the gyroscope and accelerometer to detect how these features appear to move. This helps in locating the position of the headset.

Many VR companies have released headsets using SLAM technology. Some companies are also planning of releasing theirs soon. Some companies who have released SLAM enabled headsets include:

• Facebook: Oculus Quest and Oculus Rift A
• HTC: Vive Focus
• Microsoft: Windows MR headset
• Google: Lenovo Mirage Solo

It is worthy to note that Google calls their SLAM tracking algorithm ‘WorldSense’ and Facebook calls theirs “Oculus Insight”. These tracking systems are similar to the tracking system used by Constellation. The subtle difference lies in the type of light used– either visible or Infrared light. They all have one unique similarity; the cameras found on the headset track LEDs under the controller plastic.

SteamVR ‘Lighthouse’ (HTC Vive)

This tracking system has been previously discussed under ‘Active Optical Tracking’. This headset is unique in the VR market. Unlike many headsets we have today, it does not employ the use of cameras. You also don’t need your PC to process or analyze any data. It was majorly designed for room-scale positional tracking without any need to integrate it with a user’s PC. The standout feature of the Lighthouse is that it is relatively simple to use.

How to deal with VR motion sickness

It is not uncommon to feel dizzy or uneasy after having a VR session. It is well documented that long hours of VR can cause nausea and dizziness. Those who have been into VR for a long time are not immune from these conditions. We’ll share some tips with you on how to deal with VR motion sickness. Here are the tips:

Take Ginger Beforehand

Research has made us know about the benefits of ginger. It is highly medicinal. According to some studies, ginger can protect you from feeling nauseated after using VR. You may not like to take it raw, you can choose to take a ginger supplement instead.

Place a Fan Near Yourself

You may not have heard about this before, but having a fan blowing near you while playing VR games can protect you against developing motion sickness. Sweating while putting on a VR headset might contribute to developing motion sickness. You can give it a try during your next VR session.

Wear an Acupuncture Wristband

If you are the sort of person who doesn’t like taking medications, you can decide to wear an acupuncture wristband. A wristband known as Seaband might be all you need. The band works by putting acupuncture pressure which is known as the Nei-Kuan point in your wrist. This wrist band would not completely stop you from having motion sickness, it will reduce your susceptibility to it.

Take Frequent Breaks

One of the best ways to deal with VR motion sickness is to take intermittent breaks. It is even essential to take breaks for any type of video game. Doing this will guard against eye strain and muscle aches. A practical way to do this is to set a reminder on your mobile device. When you take a break, don’t stay idle, walk around to take in the fresh air. VR manufacturers recommend taking a break every 10-15 minutes.

Don’t use VR when You’re Sick

When you’re, don’t even think of using VR. It can make your sickness worse. For example, if you have a cold and catarrh, the worst thing is to opt for VR because it will likely make it worse. And more importantly, if you have any medical condition affecting your balance, consult your doctor before using VR.

What does the future hold for VR motion tracking?

We have discussed in detail the optical and the non-optical methods of tracking. If you are a big fan of virtual reality, you might begin to wonder: what does the future hold for VR?

Since no one can accurately foretell what the future holds, we can only wait. Let’s examine some likely factors that may influence VR motion tracking in the future.

From the look of things, our thought is that optical and nonoptical methods of motion tracking will still be in force for the foreseeable future. The giant strides that have been taken in other areas of science and technology will soon impact the VR industry in a way we’ve not seen before. For example, in the field of neuroprosthetics, people with total limb paralysis can now make use of robots to get complex tasks done.

Currently, the injured nerve endings of an amputee can now be rewired so that the robotic prosthetic limbs can allow the amputee to receive signals from the rewired nerve endings. To reroute the nerve endings of an amputee, invasive surgery is needed to accomplish.

In the nearest future, there may be no need for invasive surgery to reroute a nerve ending or to implant a chip. Our minds might be the only things we need to move our limbs. The advances in sensor technology offer a promising future for VR motion tracking.

That would be all from us for today. Hope to see your thoughts about this topic in the comments below.

Until next time.

---

YOU MAY ALSO LIKE:

How To Set Up A VR Room – Everything You Need To Know

Best Full-body VR Tracking Sets

Best VR Grips and Hand Straps of 2021