Augmented Reality in medicine

HealthcARe: Augmented Reality in Medicine

There is hardly any other area in which mixed reality has as much potential for application as in healthcare. In a study conducted in 2019 on the acceptance of Augmented Reality Smart Glasses in medicine, we asked healthcare professionals and patients/relatives, in which areas they could imagine AR being used in healthcare – and where they see potential problems. The following article provides an overview of the participants’ and whether and how they are already practically feasible.

Possible Applications of AR in Healthcare

Data Collection

Using augmented reality for the collection and connection of patient data in a virtual medical record turned out to be the most imaginable application for AR in healthcare. It allows access to and management of patient and examination data such as X-ray, CT and MRI images. Doctors and nursing staff thus have an overview of patient data during examinations, treatments or surgical procedures. If the corresponding data is stored in the cloud or in the memory of the Smart Glasses, this imaginary application can be implemented without any problems.

However, the connection with other medical devices, for example for monitoring vital functions during surgical procedures, is more difficult than the participants imagine (Glauser, 2013).

AR-assisted Diagnosis

The survey participants could also imagine using augmented reality for support or partial automation of the diagnosis: The Smart Glasses record the patient’s symptoms via automatic visual recognition or input by the attending physician, evaluate them and suggest one or more possible diagnoses based on online databases. By providing additional information from suitable databases, the Smart Glasses could also support the user in the diagnosis process.

While an automated diagnosis has not yet been used, the general information support has already been tested: The Augmented Reality Smart Glasses identified the complex medical terms correctly about half the time. The provided information was deemed as overall useful for clinical decision making (Muensterer, Lacher, Zoeller, Bronstein, & Kübler, 2014).

Live instructions via AR

Another frequently mentioned AR application for the medical field are step-by-step instructions. It could be used to assist in the maintenance and repair of medical equipment, during examinations, treatments and surgical procedures. Actual AR solutions, however, only exist in the area of maintenance and service. Instructions for working on patients are not yet in use.

In this area, another application mentioned by our participants has proven itself: the cooperation of medical professionals over distance. With the help of Augmented Reality Smart Glasses, it is possible to start a live transmission during any process to connect with colleagues or experts in a kind of video conference, who can thus actively support the user in their task (Medical xpress, 2013; Muensterer et al., 2014).

Augmented Reality in Medical Education

This feature is also used in the training of healthcare professionals. For example, a live transmission as described above enables medical students to follow a procedure in detail without having to be present in the operating room themselves – and possibly miss important details (Muensterer et al., 2014).


Training, analysis and evaluation of various surgical procedures with AR applications on a simulation basis (Barsom, Graafland, & Schijven, 2016), but also AR tools such as the “mirracle” application (Blum, Kleeberger, Bichlmeier, & Navab, 2012) for visualizing the human body, are also used in medical education. Such simulations are also being used in virtual reality (Willaert et al., 2012).

Further applications of AR in medicine

However, applications for visualization are also being developed separately from the training: A mini-projector-based AR enables “positionally correct projection of visual information in real time directly onto the surface of a patient” (Kobler, Hussong, & Ortmaier, 2010).

Furthermore, the use of Augmented Reality Smart Glasses for photo, audio and video documentation is imaginable. This function has already been tested successfully in forensic medicine (Albrecht et al., 2014). In this context, the operation of the Smart Glasses without hands by voice command was particularly convincing.

In psychotherapy, Augmented and Virtual Reality are mainly used to treat phobias. While VR has proven itself in the therapy of the fear of heights or flying, the potential of AR lies above all in its decisive difference to virtual reality: the integration of digital elements into the real world offers great opportunities for the therapy of phobias like arachnophobia, the fear of spiders (Juan, Alcaniz, Botella, & Banos, R.M., Guerrero, B., 2005).

Our participants could imagine a whole range of medical AR applications for patients. Concrete examples are indoor navigation within a medical facility or the prevention of medical problems, for example by providing additional information on food in a supermarket. Remote treatment would be possible by means of simple video calls between doctor and patient, language translation would work with a variety of AR apps, and in the field of logistics and organisation, augmented reality is already being used.

Augmented Reality in der Medizin: Studienergebnisse 2019

Problems of Augmented Reality in medicine

The second part of the study focuses on possible problems of augmented reality in medicine. The participants saw the greatest risks in

  • Occupational safety/ health effects of Smart Glasses on the users,
  • information overload, excessive demands, and distraction of the users and
  • Loss of independence and know-how of healthcare professionals.

According to our participants, the following requirements must be met in order to enable the successful implementation of Augmented Reality in the health care system:

Despite modern technical support, the focus must lie on the relationship between doctor and patient.

Medical personnel must be appropriately trained in the use of Augmented Reality Smart Glasses.

Appropriate data protection must be guaranteed.

The latter is still the biggest barrier in practice. While testing Google Glass in a hospital, the recorded data was synchronized with the Google servers. This means that confidential patient data is stored on potentially insecure, external servers (Muensterer et al., 2014).

The legal situation is the following: Health data is special personal data according to Art. 9 DSGVO, therefore its protection requires special attention. Data processing when using Augmented Reality Smart Glasses would be carried out in the cloud and is therefore subject to § 11 BDSG regarding the collection, processing or use of personal data on behalf of the customer. The paragraph requires the relevant medical institution to regularly check the suitability of the cloud provider for storing and processing sensitive patient data in accordance with Art. 32 DSGVO. If the cloud provider’s servers are located abroad, Art. 44 ff. DSGVO regarding data transmission also apply.

Cloud computing with Augmented Reality Smart Glasses is therefore not prohibited by law. However, it poses a major challenge, especially for smaller institutions that do not have the necessary legal and technical know-how.


Albrecht, U. V., Jan, U. v., Kuebler, J., Zoeller, C., Lacher, M., Muensterer, O. J., & Hagemeier, L. (2014). Google Glass for documentation of medical findings: evaluation in forensic medicine. Journal of Medical Internet Research, 16(2).

Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016). Systematic review on the effectiveness of augmented reality applications in medical training. Surgical endoscopy, 30(10).

Blum, T., Kleeberger, V., Bichlmeier, C., & Navab, N. (2012). mirracle: An augmented reality magic mirror system for anatomy education. IEEE Virtual Reality Workshops (VRW).

Glauser, W. (2013). Doctors among early adopters of Google Glass. Canadian Medical Association Journal, 185(16), 1385.

Juan, M. C., Alcaniz, M., Botella, C., & Banos, R.M., Guerrero, B. (2005). Using augmented reality to treat phobias. IEEE Computer Graphics and Applications, 25(6), 31–37.

Kobler, J., Hussong, A., & Ortmaier, T. (2010). Mini-Projektor basierte Augmented Reality für medizinische Anwendungen. CURAC.

Medical xpress (2013). First US surgery transmitted live via Google Glass (w/ Video). Retrieved from

Muensterer, O. J., Lacher, M., Zoeller, C., Bronstein, M., & Kübler, J. (2014). Google Glass in pediatric surgery: an exploratory study. International journal of surgery, 12(4), 281–289.

Willaert, W. I., Aggarwal, R., Van Herzeele, I., Cheshire, N. J., & Vermassen, F. E. (2012). Recent advancements in medical simulation: patient-specific virtual reality simulation. World journal of surgery36(7), 1703-1712.

Photo credit:



Leave a Reply