HEX @ Springer VR
New paper: Evaluating visual guidance techniques for AR-supported medical assembly
We’re excited to share that our latest research paper, “Evaluation of visual guidance techniques for augmented reality-supported assembly tasks in the medical context,” has been published open access in Virtual Reality.
Read the paper on Springer Nature
How should augmented reality guide users through an assembly task?
Augmented reality has great potential to support people when performing complex manual tasks. In medical contexts, this could be particularly useful for the assembly and preparation of surgical instruments, where clear and reliable guidance is essential.
But providing instructions in AR is not simply a matter of putting information into the user’s field of view. Where instructions are placed, and how they relate to the physical object, can have a major impact on the user experience and task performance.
In this study, we investigated three different approaches to visualizing instructions in AR:
- In-situ guidance, where visual instructions are displayed directly on the physical assembly;
- Field-of-view-fixed guidance, where the visualization remains fixed relative to the user’s view; and
- World-fixed guidance, where the instructions remain anchored in the surrounding environment.
We compared these approaches with conventional paper-based instructions using reproducible 3D-printed assemblies representing surgical instruments.
Bringing experts and non-experts together
For our within-subject user study, we recruited 32 participants: 16 scrub nurses and 16 laypersons. Participants completed step-by-step assembly tasks involving different objects with varying shapes and levels of complexity.
Our AR system used gaze-based interaction, allowing participants to interact with the instructions without requiring conventional controllers.
We looked at both objective measures, such as task completion time and errors, and subjective measures, including usability, user experience, and workload.
What did we find?
Overall, the AR-based approaches performed comparably to paper-based instructions in terms of objective task performance. At the same time, AR achieved comparable or better results for several subjective measures related to usability, user experience, and workload.
One of the most interesting findings concerned where the AR information was displayed.
Across our experiments, world-fixed and field-of-view-fixed visualizations generally performed better than the in-situ approach. With in-situ guidance, users need to process information at different locations directly on the assembly, and the approach can also be more sensitive to inaccuracies in object pose estimation.
Our results also highlight that there is no one-size-fits-all solution for AR guidance. Participant feedback revealed opportunities to further optimize and personalize how instructions are presented.
Why is this interesting?
A central aspect of our work is the combination of markerless object pose estimation, augmented reality visualization, and real hands-on assembly tasks.
Rather than evaluating AR guidance in an abstract environment, we wanted to investigate what happens when digital instructions have to be aligned with and support interaction with real physical objects.
We believe these findings can contribute to the development of future AR-based assembly assistance systems, particularly in medical and other safety-critical contexts where users need clear, precise, and timely guidance.
We were also encouraged by the positive feedback on our gaze-based interaction approach. At the same time, the differences in feedback across participants emphasize the importance of considering individual experience and expertise when designing future AR assistance systems.
What’s next?
For us, this study is one step toward AR systems that can understand what physical object a user is working with, what state it is currently in, and what kind of assistance the user needs — and then provide the right information at the right time and in the right place.
There are several interesting directions to explore from here, including more robust pose estimation, adaptive visualization strategies, and personalization based on user expertise and interaction behavior.
We’re very happy to see this work now published and openly available, and we look forward to building on these results in our future research.
Paper
Kreimeier, J., Prasad, P., Li, S. et al. Evaluation of visual guidance techniques for augmented reality-supported assembly tasks in the medical context. Virtual Reality (2026).
DOI: 10.1007/s10055-026-01491-3
