PEORIA — Dr. Matthew Bramlet has spent his career in medicine building three-dimensional hearts. At first, only inside his head, but now he’s focused on bringing them to life.
As a pediatric cardiologist specializing in MRI, Bramlet expertly reads the series of 2-D images produced by MRI and CT scans for every patient at the Children’s Hospital of Illinois with a congenital heart defect. He mentally constructs the complex anatomy of each, diagnosing the unique malformations to help patients and doctors select treatments for the best possible outcomes.
“I spent years of initial training learning how to draw a 2-D heart, and I actually prided myself in my ability to draw these,” he said, sketching an anatomically correct heart in about 20 seconds on a scrap of paper.
In the early years of his career, this was perhaps his best tool to explain to families what doctors saw when they looked at their newborn child’s complicated medical images, but with new tools in education and simulation, Bramlet saw an opportunity to give those images more dimension.
“I was a doc who had an idea that I thought was valuable,” Bramlet said. Now, he’s primarily focused on proving that value by creating an interactive program to examine anatomical lesions in virtual reality.
Virtual reality is the latest frontier in what’s been a nearly constantly evolving industry of medical imaging.
When Dr. Mark Plunkett began his pediatric cardiology practice in the mid-’90s, the main diagnostic tool he used was cardiac catheterization, then echocardiography.
More recently, CT and MRI imaging have advanced to the point that the data could be reconstructed into three-dimensional images — a development that led to a partnership between Jump Trading Simulation & Education Center and the National Institute of Health to create a digital exchange where doctors and hospitals can freely access these images.
Then those images were 3-D printed, first at Jump using plaster composites and more recently in partnership with Caterpillar Inc. using more flexible polymers.
But making these images not just three-dimensional, but truly interactive in the virtual reality format is an innovation that could have far-reaching impact, not only to diagnose patients, but to prepare for surgery and potentially save lives if the prototype that Bramlet and a team of engineers are building pans out.
“This is new and innovative technology that I would never have dreamed of at the beginning of my career, or even more recently,” said Plunkett, who has operated on thousands of hearts, some of them as small as a grape.
Plunkett has used the still-under-development virtual reality platform to assess two patients’ cases: one a congenital heart defect that has since been surgically repaired, and the other a complicated case of a child who never developed the arteries that carry blood to her lungs. Instead blood travels to her lungs through a complex series of smaller blood vessels, a part of the anatomy much more difficult to evaluate through traditional, 2-D imaging.
“It gets a little confusing even if you have an MRI and you do a 3-D rendering of the MRI. It’s not the sort of thing that lends itself to this sort of modeling,” said Plunkett, holding a 3-D printed heart.
“What you’re really trying to do is take a large number of blood vessels that are running everywhere and define exactly what their path is and how they are related to each other or connected to each other. The rendering in VR (virtual reality) was great because you could just pick one blood vessel and follow its path.”
To create those intricate images, a pair of engineers has developed a process to meticulously designate the parts of 2-D images they want to render into a 3-D object. This process remains labor intensive and can take anywhere from half an hour to several weeks depending on complexity of the anatomy and experience of the user.
A second pair of engineers has been tasked with designing the software to use with the Vive, a commercially available virtual reality system. One of those engineers is Justin Drawz, a simulation engineer with a background in bioengineering and an off-hours interest in playing and designing video games.
From a technical standpoint, there’s no existing medical or educational tool that would provide any usable comparison to what they were trying to create; instead, they started with the digital images and improvised.
“From there, it’s just like developing a game,” Drawz said.
For Drawz and the other engineers, that required a user-centric paradigm. They had to think like the highly trained and specialized doctors who would be using the technology.
Surgeons want to be as intimately familiar with a patient’s anatomy as possible before undertaking a major surgery.
“They’re looking for really being able to pick up and hold the thing, move it around — see it like they’re going to see it on the operating table,” Drawz said.
An up-close, in-depth interaction with that heart, blood vessel network or tumor can help doctors achieve a level of familiarity that previously would have required decades of surgical experience and operations on hundreds of hearts. With a virtual reality platform, surgeons could achieve that level of mastery quicker and with minimal risk to patients undergoing major surgery.
“Many times surgeons will say, ‘When we got in it wasn’t exactly like what we thought it was.’ Most of the time, surgeons are able to improvise without an appreciable difference, but how many times do they maybe sit there and the further it gets away from what they expected, the more on-the-fly decision making it requires?” Bramlet said. “Which is time on a cross-clamped heart where every second counts.”
Among their challenges ahead, Bramlet explained, is deciphering where the technology will have the greatest value.
“We had the neurosurgeons up here, and they are just chomping at the bit for us to put cases in here before they operate,” Bramlet said.
While practicing clinicians are eager to use the new tech, all but the most complex cases can usually be adequately assessed and treated using existing imaging techniques. Virtual reality may be more widely applicable to the less seasoned residents and medical students.
“I’m putting all my chips down saying I bet once we start cranking this information out we’re going to be able to discover tremendous options for how to make it sustainable, and I think medical education is a significant opportunity for that,” Bramlet said.
Laura Nightengale is the Journal Star’s health and lifestyle reporter. She can be reached at 686-3181 or lnightengale@pjstar.com. Follow her on Twitter @lauranight.
