Augmenting the Dark: Exploring Assistive Micro-guidance in Sonified Mixed Reality

This thesis proposes a series of evaluations of sonification methods rendered as AR via three user experiments in simulated and real-life scenarios. We aim to promote the use of spatialized sonification in situated Augmented Reality (AR) to model next-generation micro-guidance aids for low-visibility and vision-impaired (VI) scenarios. The solutions being proposed are compared in interactive sonifications of two- and three-dimensional positions in a series of hand-navigation assessments with visually impaired and blindfolded sighted users, to validate different approaches in environments without any visual feedback. In a user experiment (N=47) in 2D hand-navigation, results outlined that methods based on sound spatiality had the most promising performance in average time taken and distance from target. In a follow-up experiment (N=19), assessing vertical guidance in a 3D task, results indicated a significantly higher accuracy for our novel method of representing height through pitch - Target-based Dynamic (TbD) pitch. Regarding workload, users initially self-reported that methods that exclusively used a single pitch signal to indicate a 2D distance as the least demanding, followed by the spatiality-based methods. When targeting multiple ways to express vertical distance as a third axis, users self-reported the TbD method as the least demanding in workload. For our final user experiment, we gathered sighted people’s self-reported and perceived empathy with the BVI (Blind and Visually Impaired) community via an online survey, from both sighted (N = 77) and BVI people (N = 20) respectively. This highlighted a significant disparity between the two. Later, blindfolded sighted participants went through a blindness embodiment experience in AR (N = 15), while relying on a proposed assistive technology. By re-testing sighted people’s empathy, we found that their empathetic and sympathetic responses towards said community significantly increased. Furthermore, survey results suggest that the BVI community believes the use of these empathy-evoking embodied experiences may lead to the development of new assistive technologies. Ultimately, this thesis evaluates how sonified AR can help users to have better and safer performances in a micro-navigation context, particularly in day-to-day manual tasks, such as handling a hot stove and discarding trash in a garbage can.