HEX Lab

About Us

Imagineering novel languages for the communication between human(s) and machine(s)

Established in 2021 and relocated to the Technical University of Munich (TUM) in 2023, the Human-Centered Computing and Extended Reality Lab aims to pioneer new technologies to enhance medical care and healthcare. We pursue fundamental, translational, and radical blue-sky research to improve medicine for patients and healthcare professionals.

We adopt a holistic perspective on machine intelligence, emphasizing AI-driven technologies that enable meaningful collaboration between humans and machines. Therefore, we operate at the intersections of human-computer interaction, artificial intelligence, extended reality, robotics, and medicine. Exemplary research areas include AI-based assistance systems for surgical procedures, systems for diagnosing diseases and disorders, telepresence procedures, and robot-assisted rehabilitation measures.

We perform our research integrative as part of TUM’s School of Medicine and Health, TUM’s School of Computation, Information, and Technology, and the TUM University Hospital Department Clinical Medicine, affiliated with the Clinic for Orthopedics and Sports Orthopedics.

People

Head

Avatar

Daniel Roth

Head

Human-Centered Computing, Machine Intelligence, Extended Reality, Robotics

Senior Research Scientists

Avatar

Julian Kreimeier

Senior Researcher

Extended Reality, Artificial Intelligence, Interfaces & Interactivity

Researchers

Avatar

Constantin Kleinbeck

PhD Candidate

Extended Reality, Artificial Intelligence, Rendering, Interface & Interactivity

Avatar

Shiyu Li

PhD Candidate

6D pose estimation, 3D object tracking, AR assembly guidance

Avatar

Victor Schaack

PhD Candidate

Robotics, Artificial Intelligence, Exoskeletons

Avatar

Hannah Schieber

PhD Candidate

Computer Vision, Neural Rendering, Deep Learning, Extended Reality

Avatar

Luisa Theelke

PhD Candidate

Digital Health, Machine Learning, Human-Computer Interaction

External Affiliated Researchers

Avatar

Judith Friedl-Knirsch

PhD Candidate

Cross Reality, Empiric Evaluation, Human-Computer Interaction

Avatar

Kristoffer Waldow

PhD Candidate

Mixed Reality, Avatar-based Communication, Artificial Intelligence, Motion Capture

Team Support

Avatar

Hafize Nur Şahbudak

MSc Student

Computer Vision, Extended Reality, Human Computer Interaction, BCI

Visitors

Avatar

Dimash Mukashev

MSc Student

Robotics, Haptics, Tactile Sensing, Extended and Mixed Reality

Projects

Our research is centered around the understanding, design, development, and evaluation of computing solutions and human-machine interfaces that improve the way we interact with, work with, and communicate through machines. A majority of our research considers AI driven Extended Reality user interfaces in the context of physical and mental health, such as Virtual-, Mixed- or Augmented Reality systems. Examples of these are assistive technologies for medical procedures or systems to assess and diagnose pathologies and disorders.

To do so, we strive for foundational, translational, and radical scientific contributions with high methodological and experimental quality. Our research is inspired by multiple disciplines, including but not limited to Computer Science, Psychology, Design, Medicine, and Neuroscience. We incorporate methods from these disciplines, such as computer vision, computer graphics, artificial intelligence, and user-centered design in a broad methodological bandwidth to support our research goals. We closely interact with partners from medicine and industry to incorporate different perspectives.

KARVIMIO Project
KARVIMIO is a three-year BMBF funded project with a total volume of 1.73 Million Euros.
KARVIMIO Project
Medical Visualization in Mixed Reality
Realistic visualization of medical and surgical data in 3D
Medical Visualization in Mixed Reality
Smart Remote Extended Reality Teleconsultation and Product Guidance
Neural rendering and reconstruction for guidance
Smart Remote Extended Reality Teleconsultation and Product Guidance

Teaching

Topic Areas

We offer theses in various research areas. Applications for thesis are considered to the dates 1st of April for the Summer Semester and 1st of October for the Winter Semester. We offer topics in the following areas:

  • Human-Centered Computing / Human-Computer Interaction
  • Machine Intelligence / Artificial Intelligence
  • Computer Vision / Neural Rendering / Deep Learning
  • Extended Reality
  • Rendering, Interfaces & Interactivity
  • Robotics & Exoskeletons
  • Digital Health

If you are interested, please send your transcript of records, CV and motivation to the respective supervisor with CC to hex-thesis.ortho@mh.tum.de.


Lectures and Seminars

Exergames in Medicine and Health (SS)
This course deals with the theory, design, engineering and development of exergames. In the course, students will be provided with theoretical game-design, gamification and medical foundations and work in small groups to realize working exergame prototypes. Exemplary project themes could be:

  • Designing exergames for rehabilitation and physiotherapy
  • Designing gamified approaches for performance optimization in sports
  • VR supported simulations for patients with motor impairments

The course is designed in an interactive project format. Based on initial discussions and theoretical input, students research, design, develop, and evaluate solutions in the form of projects and studies in small groups following user-centered design and agile software engineering principles.

More information can be found here.

XR: Virtual-, Mixed-, and Augmented Reality (WS)
In this lecture, students will expand their knowledge of advanced topics in virtual, augmented, and mixed reality. Key topics include among others:

  • Real-time interactive systems, avatars, and virtual embodiment
  • Social XR, collaborative environments
  • Advanced 3D interfaces and visualization techniques
  • Cross-reality and environment reconstruction

In an accompanying practical exercise, ythey will gain practical experience in developing advanced XR application.

Registration via TUMonline

Computer Vision in the Operating Room (WS)
This seminar focuses on the latest methods in computer vision and deep learning for applications in the operating room. The course includes individual paper reviews and interactive presentations, helping students not only to develop technical expertise but also to build scientific communication and discussion skills.

More information can be found here.

Open Positions


Student

We are currently not looking for student lab assistants. Please check back later or reach out to us.


PhD / Postdoc

If you have a strong research / study background in the areas of XR, HCI, Computer Vision, Medical Engineering, or Robotics please drop us a mail and we’ll get back to you.

Contact

Prof. Dr. Daniel Roth
Technical University of Munich
Klinikum rechts der Isar
Machine Intelligence in Orthopedics
Human-Centered Computing and Extended Reality Lab

  • Trogerstrasse 10, Munich, 81675
  • Find us on the third floor.
    Please direct postal deliveries to c/o Fritz Seidl, Ismaninger Str. 22
  • Twitter