Surgical Interventions and Training

Imagine a flight simulator for surgeons. By using augmented reality to create an immersive three-dimensional medical environment, we could help train and assess medical providers on the intellectual and physical abilities needed to take care of patients in an array of settings and situations. In addition, such an environment can be tailored to train providers in rare but important events, like mass casualty scenarios, or difficult operative dilemmas. Beyond simulation, this technology will transform the practice of medicine, especially in fields like trauma surgery. Our current work focuses on recreating real events for training, but the technology being developed will allow for real-time transmission of 3-D images. Furthermore, it will lay the groundwork for software which will enable smart eyewear to transmit, receive, and even interpret important clinical information: lab results would come directly to the eyes of the provider.

In another area of interest, the use of computerized tomography(CT) scans and Magnetic Resonance Imaging (MRI) has expanded significantly for image-guided interventions, and has led to procedures that have the potential to improve resection of primary brain tumors. Given the wide variety of contrast that is possible from these imaging modalities, and that each of the techniques provides unique information, there is a great desire to be able to overlay the imaging information over the surgical field. We will use the projection-based augmented reality in the Augmentarium to experiment with 3D-printed phantoms of the head with multiple image modalities superimposed on the surgical field. This will allow us to examine calibration, registration, and latency issues in a controlled environment.