Multimodal Augmented Reality Guidance for Object Localization and Wayfinding in Visually Challenging Environments

Abstract

Object localization is difficult in cluttered environments, even more for people with visual impairments like tunnel vision. Augmented reality and spatial audio offer promising avenues for assistive cueing, yet little is known about how auditory, visual, and combined cues compare in guiding object search in real-life conditions. Using a video see through head-mounted display and camera-based object localization, we compared these conditions in a user study. Six participants searched objects in an everyday live environment. Our system provides distance-adaptive, spatially localized guidance toward the objects. We simulated tunnel vision and revealed the potential of dual-modality spatialized cues in supporting daily tasks.

Hannah Schieber

Doctoral Candidate

I am interested in computer vision and extended reality. I research 3D scene content creation using neural rendering and guidance of people in 3D.

Julian Kreimeier

Senior Researcher,
Research Manager

I research human-computer interaction, accesibility and extended reality.

Daniel Roth

Director

Assistant professor at TU Munich and Director of the HEX Lab