XR Blog

The Communication between Autonomous Vehicles and Pedestrians

By 9. January 2020 January 22nd, 2020 No Comments

The Communication between Autonomous Vehicles and Pedestrians in Ambiguous Situations

Summary: In this article, Sebastian Stadler discusses the compensation of the missing communication between Autonomous Vehicles (AV) and pedestrians in ambiguous situation with external Human-Machine Interfaces.  Since real-life experiments potentially involve safety issues, Virtual Reality (VR) technology is used for quantitative and qualitative data collection. The key aspects are:

  1. Study context: The missing communication between AVs and pedestrians
  2. The usage of VR for usability testing
  3. Study setup in VR
  4. Insights and implications

by Sebastian Stadler, January 2020

Study Context: The missing communication between AVs and pedestrians

The communication between pedestrians and car drivers is currently mainly carried out via eye contact and gestures. This applies particularly to ambiguous situations such as at zebra crossings or at car parks. Since in level 5 autonomous vehicles (AV) a driver is not present anymore, this communication cannot be carried out in the same manner anymore. This fact can lead to potentially unclear situations, and thus, safety hazards. Therefore, the designers of TUMCREATE evaluated explicit Human-Machine Interfaces (HMIs) with the help of usability tests. The concepts consisted of display-based HMIs that were attached to the front of the AV as well as laser-projections in front of the AV and on the infrastructure such as the sidewalk.

Virtual Reality Applications for Usability Testing

Since tests in real-life conditions involve a high safety risk for participants, immersive Virtual Reality Technology was used to conduct the tests. A further advantage of using Virtual Reality for this study was the possibility to create immersive and realistic scenarios that led to authentic behaviour from participants. Moreover, Virtual Reality constitutes an inexpensive and timesaving alternative compared to studies that are carried out in real-life conditions.

Study Setup in Virtual Reality

The designers used Virtual Reality Glasses (i.e. Head-Mounted Display) with six degrees of freedom (HTC Vive Pro). Thus, beyond head movements, also spatial movements could be tracked. This fact highly increased the immersion for the participants since their actual movements resembled their movements in the virtual environment. The task for the participant during the test was to repeatedly cross a one-way road by using a zebra crossing. Every time the participant approached the road, an AV with one specific HMI concept approached. Thus, ten HMI concepts could be evaluated. Additionally, the participants were approached once by an AV that was not equipped with any kind of explicit HMI. This constituted the baseline scenario for the subsequent data analysis. By comparing the baseline scenario with the HMI scenarios, the decision times for participants to cross the road as well as the error rates could be derived. Furthermore, subsequent to the Virtual Reality test the participants subjectively rated each HMI concept.

Insights and Implications

The quantitative and qualitative data collection in Virtual Reality led to the following insights:

  • Decision times decrease when the AV is equipped with an explicit HMI concept
  • Error rates decrease when the AV is equipped with an explicit HMI concept
  • Participants feel supported by explicit HMIs when attempting to cross the road
  • Explicit communication channels are required to clarify ambiguous situations
  • Clear symbols including red and green colour coding decrease the risk potential for pedestrians
  • Compared to laser-projections, display-based HMIs lead to increased awareness of the traffic situation

The overall aim of the project was to demonstrate that the implementation of explicit communication channels onto AVs increase safety for pedestrians but also for passengers inside the AVs. Questions regarding a potential standardization as well as training measures for citizens remain open for clarification. These questions will be addressed in future studies.


Stadler, S., Cornet, H. and Frenkler, F. (2020), “Towards User Acceptance of Autonomous Vehicles : A Virtual Reality Study on Human-Machine Interfaces”, International Journal of Technology Marketing, Vol. 13 No. 3, available at: https://doi.org/10.1504/IJTMKT.2019.10025612 (in press).

Stadler, S., Cornet, H., Novaes Theoto, T. and Frenkler, F. (2019), “A Tool, not a Toy: Using Virtual Reality to Evaluate the Communication Between Autonomous Vehicles and Pedestrians”, in tom Dieck, M.C. and Jung, T.H. (Eds.), Augmented Reality and Virtual Reality, Springer Nature Switzerland AG, Cham, available at: https://doi.org/10.1007/978-3-030-06246-0_15.

Stadler, S., Cornet, H., Kong, P. and Frenkler, F. (2017), “How can communication between Autonomous Vehicles & Humans be improved by using Virtual Reality?”, Asia – Design Engineering Workshop 2017, Seoul, Korea.

May I cite this posting? If yes, how?


If you enjoy this posting and want to reference it in your own (academic or professional) work, feel free to cite!


Our suggested citation is:

Stadler, Sebastian (2020): The Communication between Autonomous Vehicles and Pedestrians in Ambiguous Situations; xrealitylab-article from Jan. 09, 2020; URL: http://xrealitylab.com/virtual-reality-autonomous-vehicles-human-machine-interfaces/


Please make sure you have the permission to re-use external content that we included in this article.