Augmented reality or commonly referred to as AR, is the result of combining the user’s surrounding environment in real time with superimposed information, such as sound or images. In fact, if you have a smartphone and use basic apps, then you have probably used AR without even realizing it.
Have you seen those trendy pictures where the person has puppy ears and a puppy nose plastered all over Facebook, Snapchat or Instagram? That’s done with AR. Layers of digital data (the nose and ears) appear over reality. You don’t really have dog ears, but with this tweaked reality, you do. In other words, there is a detection system in the app to find your face and its orientation in the picture and the ears and nose are laid in the correct positions.
Have you used the Ikea app for furniture shopping? You can take a picture in your house and see what the furniture looks like in that space. Or to see what it will look like. If so, you’ve just augmented reality. Exciting!
History of AR
What we see today in AR is the result of many iterations of trial and error and the consolidation and extension of less sophisticated technologies. It wasn’t until the last half of the twentieth century where we began to build the essential building blocks to the AR in which we know and love today.
AR in the 1950s.
- 1955, Morten Heilig filed for a patent for The Sensorama. It is the first known machine using immersive, multi-sensory technology.
AR in the 1960s.
- 1968, Ivan Sutherland created a first head-mounted display. A device that is worn on the head or as part of a helmet, that have a optical displays.
AR in the 1970s.
- 1975, Myron Krueger created Videoplace, which allows users to interact with virtual objects. His idea with the Videoplace was the creation of an artificial reality that surrounded the users, and responded to their movements and actions, without being encumbered by the use of goggles or gloves. In essence it’s a artificial reality laboratory.
AR in the 1980s.
- 1980, Steve Mann developed a wearable computer designed to be worn in front of the eye called EyeTap.
- 1987, Douglas George and Robert Morris create a working prototype of an astronomical telescope-based “heads-up display” system. Overlays the actual sky images, multi-intensity star, and celestial body images, and other relevant information on the eyepiece. This is the first case where we saw superimposed images.
AR in the 1990s.
- 1990, Thomas Caudell a researcher from Boeing had coined the term augmented reality to describe how the head-mounted displays that electricians used when assembling complicated wiring harnesses worked.
- 1992, Louis Rosenberg of the US Air Force created the AR system called “Virtual Fixtures”.
- 1998, One of the first commercial applications of AR technology was the yellow “first down” line that began appearing in televised football games.
- 1999, a group of scientists led by Frank Delgado and Mike Abernathy reported successful flight test of LandForm software video map overlay. It generated runways, taxiways and street data from a helicopter video.
AR in the 2000s.
- 2000, Bruce H. Thomas develops ARQuake, the first outdoor mobile AR game.
- 2004, Trimble Navigation presented an outdoor helmet-mounted AR system.
- 2008, Wikitude made the AR Travel Guide for Android mobile devices.
- 2009, the MIT Media Lab’s Fluid Interfaces Group presented SixthSense, a device that combined the use of a camera, small projector, smartphone and mirror. The device hangs from the user’s chest in a lanyard fashion from the neck.
- 2009, ARToolkit was ported to Adobe Flash (FLARToolkit) by Saqoosha, bringing augmented reality to the web browser
- AR in the 2010’s
- 2013, Google Glass hit the markets. A moving augmented reality to a more wearable interface; in this case, glasses. It displays on the user’s lens screen via a small projector and responds to voice commands, overlaying images, videos and sounds onto the screen
- 2015, Microsoft presented two brand new technologies: Windows Holographic and HoloLens (an AR goggles with lots of sensors to display HD holograms).
- 2016, Niantic launched Pokemon Go game for mobile devices. The game went viral worldwide and integrated AR even further to the general public.
How does it work
To be considered AR, the simulation needs to be live. Unlike VR, you are not fulling immersed in a virtual world, but instead usually see a slightly altered depiction of current reality.
There are several types that you may come across. One type may not be overlaid directly what the user is seeing in real-time, but watching. This is most commonly seeing in sports games. An example being in football when there is a reply and they want to see if the player stepped out of bounds and they draw the players foot distance from the line.
Another form of AR allows the user to see superimposed data on what they are viewing. An example of this could be google glasses over-laying objects on top of what the user is seeing in real time.
What technology makes this overlay possible? Computer vision. Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world. Basically, a way for the computer to understand what is happening in real life. Once the computer understands what is happening it can overlay data on the real-world environment adequately. An example being the Vivino app, where you can take a picture of a wine menu and see an overlaid image with reviews and information for the particular wine.
What’s needed
As simply and seamless as AR seems on the user side, there are a lot of complexities and intricacies that bring a flawless experience. This includes the proper device to show you the augmented reality such as a headset or mobile phone, as well as different sensors to understand the environment. As well as computer vision which was mentioned previously, which does quite a bit of the processing and computer behind scenes
Device
Most smart devices built in the last several years support the functionality for Augmented reality. Whether it’s a mobile phone, tablets or specific device that is tailored for AR, there are no shortages in ways to experience Augmented Reality. These devices typically have the hardware components needed, such as the required sensors, cameras, accelerometer, gyroscope, digital compass, GPS, CPU, displays etc.
Here are typically devices people use for the most optimal AR experience:
- Mobile devices (smartphones and tablets) – Most commonly used by the general public for mobile games (think Pokemon Go).
- AR specific Device, designed specifically for AR functionalities. An example is head-up displays (HUD), generally used to train military fighters pilots, that also extend to applications in aviation, automotive industry, sports, etc.
- AR glasses (or smart glasses) – Display an overlay on the world you are seeing around you. Could be things such as mobile phone notifications or to access content hands-free. The most popular are the discontinued Google Glasses.
- AR contact lenses (or smart lenses), taking the glasses one step further are contact lenses that actually lie directly on top of your eyes. Manufacturers like Samsung and Sony have announced the development of AR lenses.
- Virtual retinal displays (VRD), creating images by projecting laser light into the human eye. Aiming at bright, high contrast and high-resolution images, such systems yet remain to be made for a practical use.
Hardware required
For the previously discussed devices to properly run AR applications, the correct sensors need to be in place. Since the real time environment is being read and displayed the objects need to be mapped out (S.L.A.M. or simultaneous localization and mapping) as well as have the distance and size calculated (depth tracking).
Here are the required accessories the devices need in order to do so:
- Cameras and sensors. The surrounding world being the main source of information for augmented reality is collected through the camera. So, the better the camera, the better-quality experience. Cameras on devices are scanning the surroundings and with this info, a device locates physical objects and generates 3D models, using S.L.A.M.
- Processing. To be able to process the imagery and overlay data on top AND display to the user, the AR devices need to act like little computers. They require a CPU, a GPU, flash memory, RAM, Bluetooth/WiFi, a GPS, etc. to be able to measure speed, angle, direction, orientation in space, and so on.
- Reflection. In some Headsets there is reflection technology used to assist human eyes to view virtual images. Some have an “array of small curved mirrors” and some have a double-sided mirror to reflect light to a camera and to a user’s eye.
Types of AR
As of today, there are 4 types of augmented reality today. With the development and progression of technology, these types will begin to diverge, but for now it is just 4.
Marker-less AR
Position-based augmented reality, utilizes the devices built in sensors — GPS, a compass, a gyroscope, and an accelerometer to calculate the user’s position and exact location. From this data, there will be certain features available. An example being on Instagram you can overlay the location on your photo. This data then determines what AR content you find or get in a certain area. Another example could include events and information, business ads pop-ups, navigation support that show up on your mobile phone.
Marker-Based AR
When there is a specific marker in the visuals, then there is a trigger for the next part. It’s only when a specific marker is visible to the device that the user can interact with it to complete the AR experience. Once registered, the augmented reality device might overlay information from that marker directly on the screen or open a link, play a sound, etc. Augmented reality apps are written in special 3D programs that allow the developer to tie animation or contextual digital information in the computer program to an augmented reality “marker” in the real world.
Using computer vision, once the object, such as a QR code or face is recognized there will be a next reaction such as overlaying dog ears on the face. In other words, the digital information from a known marker (your face), it begins to execute the marker’s code and layer the correct image or images (dog ears).
Projection-Based AR
This is one of the least developed types. Think Star Wars or star trek when there was a hologram that showed up in the middle of the room. Projecting synthetic light to physical surfaces, also known as a hologram. In some cases these holograms can be interacted with. Currently there are holographic keyboards that could be used in case of a typical physical keyboard.
Superimposition-Based AR / layered
Replaces the original view with an augmented, fully or partially. Object recognition plays a key role, without it the whole concept is simply impossible. Wonder what you would look like with tattoos? You can take a picture of yourself and place the desired tattoo and the desired location to see what it would look like. Going for a different hair style? Take a picture of yourself and select the desired hairstyle to get a preview before you make the change. This type of AR uses a device uses to recognize a physical space and then overlay virtual information on top of it
Use cases
Now that you know the history of augmented reality, how it works, and different types we can go over all of the different industries augmented reality impacts. There are many advantages to using augmented reality across many industries.
Games and Apps
With the increasing popularity of applications such as Instagram, snapchat, and pokemon go augmented reality is becoming a more integral part of our everyday lives. Many people don’t actually realize that they have used an application that utilizes augmented reality.
AR games provide much better experiences to players than traditional games and are much more engaging. There is something exciting about merging the physical and virtual worlds. Drew Lewis of Touchstone Research (2016) states that 30% of smartphone users are using AR apps at least once per week.
Some promote a more active lifestyle, such as Pokemon Go, which requires you to walk in order to experience more of the game. The game allowed users to see Pokémon characters bouncing around in their own town. The goal was to capture these pocket monsters, then use them to battle others, locally, in AR gyms. Pokemon Go was an international sensation, which had almost 100 million monthly active users at its peak.
Harry Potter Fan? There’s an AR app for that too. The game allows the user to immerse oneself in the world of Hogwarts, whether its casting spells, using potions or learning from teachers at Hogwarts. This game was wildly successful with an average rating of 4.7 in the apple store.
Education
People have different types of learning styles and unfortunately traditional schools that people are forced to go to don’t always tap into their particular style. This might make it difficult for them to learn, since it is not in the way for them to understand it. Learning with augmented reality can making learning fun and engaging, tapping into a person’s unique learning style.
You can feed your interest in astronomy with Vito Technology Star Walk app. The user just needs to point the camera of their device at the sky. After which there will be names of the planets and stars super imposed on the image. This sounds very simply, but many sensors in the device are used in order to create this unique experience. You need location data to see what’s visible that time of night in the particular part of the world you are located. Next, the position of the phone and where it’s pointing to in the night sky.
Ever heard of an AR coloring book? No? Disney’s got you covered. You color as you would in an old fashion manner and open the app and scan the drawing with your camera. The app will then detect which character you had drawn and recreate the drawing in a 3-D depiction.
Google Translate recently released a new augmented reality feature. You can speak and the language you know and translate in real time to a different text to display text or read in out loud in the other language. You can scan text in a language you don’t understand, and it will translate it for you in real time. Simply Amazing!!!
Navigation
Augmented Reality in Navigation can have several faucets. For instance, instead of using a mobile phone or built-in navigation system, you can have your directions displayed direction on the windshield. This method is still very expensive, which is the main road block for everyday adoption. Today there is one company commercializing this product WayRay, who developed a holographic GPS system that overlays directions right on the road in front of you
Another method for bicyclists would be to display directions or car warnings through a headset, so the biker doesn’t need to keep looking backwards.
Pilots might have speed and altitude displayed on either a headset or windshield, so they don’t need to look down at the controls constantly.
Miscellaneous
- Retail: Can act to bring better customer engagement and retention, as well as brand awareness and more sales. Some features may also help customers make wiser purchases – providing product data with 3D models of any size or color.
- Real-estate: Display 3-D tours of apartments and houses, which can be changed with different furniture or constructions, to see what it would look like in real life before a person buys the place.
- Medicine/healthcare: to help diagnose, monitor, train, localize health issues
- Military: Navigation for pilots or doing training
- Art :Think the coloring book previously mentioned and how it was brought to life in a 3D version
- Tourism: data on destinations, sightseeing objects, navigation, and directions.
Is it different than VR?
Yes. Virtual Reality and Augmented reality are often confused, but the distinguishing difference is that in VR you are completely immersed in the environment. You have a headset on that bring you into a completely different world – visually and auditorily. In Augmented reality you may or may not have an auditory factor, but there is an element of real visuals going on. In many cases AR has an overlay of what is really going on, but in VR you enter a different world.
Related: What is VR?
AR is more widely available to the public, where you need a smart device to get started. In the case with VR, you need some kind of headset as well as a smart device with more advanced processing power.
Simply put — You interact with the virtual reality simulation, but it is a computer-generated environment in which you are completely immersed. Augmented reality, adds (or superimposes data) to the reality you would ordinarily see rather than replacing it.
No Responses