3D Simulation

3D Simulation in neurosurgery starts with imaging. We work closely with the Department of Radiology at the Icahn School of Medicine at Mount Sinai (ISMMS) to ensure that the best quality imaging is done for our patients, which accurately reflect their particular pathology or disease state. Named “simulation protocol,” this type of advanced imaging is now standard in the Department of Neurosurgery at ISMMS. Typical image modalities are high-resolution, isotropic, thinly sliced CT and MRI scanning sequences, together with appropriate applications of more advanced imaging, which are used as input to all downstream simulation. Two key steps follow: the production of 3D-structural models of the brain from imaging data, and the interpretation of that data.


During the generation of 3D data, structures in the head that are most important for the patient’s pathology are identified within the radiological imaging and “segmented” from its surroundings. For example, a brain tumor is identified and modeled from the 2D imaging data set to produce a 3D model of the tumor. In almost all cases, the brain’s blood supply, or vascular network, and bony anatomy (skull) are modeled, in addition to more complex structures such as cranial nerves. The process of segmentation takes many forms and can vary depending on a host of external factors, but the result is a set of models that accurately reflect the relationship between structures of the brain that are critical for the planning of the operation.

Surgical Navigation Advanced Platform (SNAP)

Once 3D data has been generated, it can be used in many ways. A typical pipeline will involve the use of Surgical Theater’s Surgical Navigation Advanced Platform (SNAP) to display the data pre-operatively to the patient, for improved understanding of their pathology and confidence in the approach, and to the surgeon, for procedural planning.


Intraoperatively, the SNAP and other simulation platforms, such as Synaptive’s BrightMatter system, are used to navigate in 3D virtual and augmented reality, the patient’s anatomy in real time. This is an invaluable toolset for the surgeon; for the first time, surgeons are able to use virtual reality and heads-up display technology to literally see around corners and through tissue, identify so-called “no-fly” zones, and significantly augment their situational awareness.

Preoperative Planning

In addition to virtual and augmented reality visualizations, 3D printing has become a significant effort in the Neurosurgery Simulation Core for preoperative planning. The same models produced for virtual reality, in collaboration with our partners, the Mount Sinai Institute for Next Generation Health Care and the newly formed Medical Modeling Core, can 3D print resources to enable a tactile approach to pre-operative planning—one that our physicians have found invaluable.