Seeing Things Like You’ve Never Seen Them Before

The augmented-reality
head-up display – a new dimension in safety

A vehicle’s cockpit is a fundamental design element for safe mobility. In the future, the driver will receive an even better supply of information and will constantly have an overview of traffic – thanks to augmented reality, the computer-based advancement in how we perceive reality.

The human–machine interface is becoming increasingly important

Distraction at the steering wheel is the number one cause of accidents. Whether this is due to using a smartphone (forbidden for good reason), spending too long looking at the speedometer or adjusting the ventilation – just a brief moment of inattentiveness can lead to dangerous situations. Taking your eyes off the road for too long and engaging in operating procedures that require demanding hand–eye coordination is therefore best avoided. Yet at the same time, there are more and more features in a vehicle cockpit and connectivity with the Internet, music and apps is increasing. Meanwhile, new drive systems provide new items of information, which have never before played a role in the history of the automobile. One example of this is coasting in the hybrid vehicle, an energy-saving method used in electrified driving. All of this increases the operating spectrum and the driver's need for information. It is precisely here that the new augmented-reality head-up display from Continental comes into play. It converts information into intuitive understanding and places valuable virtual items of information in the driver’s field of vision.

The new variable in augmentation

Due to the normally expansive mirror systems with which items of information can be projected onto the windshield, large-scale head-up displays have previously been reserved for the premium class of passenger cars. This is not as it should be. So when we designed the new display, we had to think small. At the Consumer Electronics Show 2019 in Las Vegas, one of the world's largest trade fairs for entertainment electronics, Continental will be showcasing an automobile-friendly augmented reality head-up display based on waveguide technology. The new milestone was developed together with DigiLens Inc., a leading Silicon Valley-based company in projection technology based on holographic lattice structures. The spectacular new waveguide technology means that expansive mirror systems are no longer required and, for the first time, the particularly large-scale augmentation (support) in the driver's field of vision with compact device dimensions.

While the conventional mirror technology for augmented reality requires a volume of around 30 liters (picture on left), the new waveguide technology (picture on right) facilitates compact device dimensions with a volume of only 10 liters – with no change in the size of the design surface.

In the augmented-reality head-up display, we have overcome the greatest hurdle by reducing the volume of the instrument. As such, this development will soon be ready for production.

Dr. Frank Rabe,
Head of Instrumentation & Driver HMI at Continental

Dr. Richter, you are the principal expert for optical technologies at Continental. Why are you using waveguide technology in the augmented-reality head-up display?

We want our solution to open up the benefits of the AR HUD for as many drivers as possible, including those in the mid-size and compact classes. The conventional mirror technology in an AR HUD requires a volume of around 30 liters. In most vehicles, this amount of space simply is not available.

How exactly does this technology work?

Instead of mirrors, our demonstrators use three flat waveguides stacked on top of each other for the RGB color spectrum. Light beams are then coupled from below into the waveguide, diverted within the waveguide, and finally decoupled upward to project the virtual image onto the windshield in reflected form.

And how are the pictures generated?

While conventional head-up displays – in other words, head-up displays without augmented reality – are based on backlit TFT displays, for example, we use a high-quality laser technology to generate better picture qualities.

How much augmented reality can the driver cope with?

We want to complement people's viewing habits, but not overwhelm them. Our general rule therefore is: as much as necessary, but as little as possible. Previous tests have already resulted in a high level of acceptance because virtual information arrows dramatically reduce the stress of navigating in an unknown large city, for example.

What specific challenges have you come up against?

One of the numerous challenges definitely lay in the fact that our development partner has previously implemented solutions for helmet visors for pilots or motorcyclists, for example. However, the full-surface application for a windshield has an entirely different dimension. All the more reason for us to celebrate this impressive solution.

What further potential do you see for this technology?

This is a broad field. In addition to enhanced safety, a driving force here – and increasingly in automated driving – is convenience. As such, location-specific information or films can be projected onto the windshield or side windows by means of augmented reality. This will definitely give rise to new business models as well.

When will vehicles with augmented reality be ready for the market?

Initial prototypes will be available as soon as 2019; based on current projections, production could start in 2022. We want to optimally fulfill current and future requirements in human-vehicle interaction in state-of-the-art cockpit architectures.
 

Augmented reality – already a reality in many places

Augmented reality has already made inroads into many different parts of our lives. While it originated in military aviation and was designed for use in helmet visors to assist fighter pilots, there are now a wide range of civilian applications and it is an integral part of our everyday life. One of the most popular examples can be seen in football. A whistle, a free kick, the wall is in position – but is the distance between the shooter and the defense correct? The distance is shown here by means of augmented reality. But augmented reality is also setting new benchmarks in many other areas, too.

AR in medicine and surgery

Doctors will soon be able to use augmented reality to enhance their sight during operations. For example, it will be possible to show computer tomography scans taken prior to the operation or up-to-date image data from ultrasound devices above a patient's body during an operation, allowing medical operations to be conducted more efficiently and with greater precision. Augmented-reality applications are also being used as a teaching tool for medical simulators.

AR in industry

Automobile manufacturers are increasingly using augmented reality – and not just in the cockpit. Because servicing a modern vehicle is becoming increasingly complicated – due in no small part to the increasing complexity of the systems installed in the engine compartment – manufacturers are working on augmented-reality glasses to support mechanics during their work; for example, by providing them with a handy step-by-step guide while they service an engine.

AR in smartphones

The Pokémon Go game, which quickly became all the rage all around the world, was just a foretaste of the possibilities of augmented reality in smartphones. For example, the location-specific information can be superimposed on the camera image by means of a browser. The user can then instantly look up, for example, an address, background information about a tourist attraction, the price of a piece of property, the location of the nearest clubs or the shortest route to a destination.

AR in architecture

Augmented reality is ideal for visualizing buildings in public spaces. In this way, historic buildings that have long since disappeared or architectural projects yet to be built can be conjured up and viewed in the context of the current surroundings.

AR in education

Augmented reality has great potential when it comes to learning, particularly with mobile applications. A good example of this is the “Timetraveler Berlin Wall” app, which offers a fascinating window into the past. While the user is making their way along the Berlin Wall, this app allows them to accurately relive the historical events from bygone eras in the form of videos, photos and texts on their smartphone's display.

AR in the working world

Do you like the idea of meeting up with people without having to take the time to travel? Augmented reality will soon revolutionize conference calls. If virtual avatars are created of all conference participants, the faces of the participants can be placed onto the avatars in real time by means of special glasses and camera systems.

AR under water

The Fraunhofer Institute for Applied Information Technology (FIT) has developed an underwater simulation with augmented-reality technology. In the future, this method could help professional divers responsible for carrying out hazardous underwater repairs in situations where their vision is limited, such as on oil rigs or dams.

Augmented reality is the perfect merger of data and real life: This enhanced view of reality offers greater knowledge, greater comfort and greater safety – including when on the road with the augmented-reality head-up display.