Pragmatic Enthusiasm for AI
By setting realistic project expectations, a small data science team found they could achieve measurable results.
As a radiologist, balancing the potential of AI with the current capabilities and practical considerations for implementing successful projects can be a struggle. However, once this balance is realized, there is real potential to find ways to create change within an institution. We started with a small team focusing on the right problems and with buy-in from leadership.
Developing a Program
Penn State Health is an academic radiology department with 50 radiologists subspecialized by division. Our residency programs graduate six diagnostic and one IR resident per year. As in many health systems, our hospital IT department controls much, if not all, of our institutional data. IT’s goals are determined at the highest level of the organization, and its priorities are well-defined. These objectives include storing information about patient clinical encounters, maintaining efficient billing and processes for covering insurance and other business needs, and protecting all of the data. Until recently, IT priorities have not included making data available and enabling data analysis at the departmental level.
When we began our AI journey, we came to a critical realization: We needed to collaborate with IT to enable an easier and more flexible approach to data access and analysis. Since structures in our hospital were originally put in place to efficiently use and safeguard data — rather than for data analysis — gaining access to data for other purposes was no easy task. We faced hurdles including a lack of clarity in data governance, poor or insufficient underlying data structure, and a disconnect between physicians’ data needs and IT’s abilities and priorities. Despite these hurdles, the enormous potential benefit of using our data to extend Penn State Health’s AI capabilities and resources made it worthwhile for radiologists to get involved in data science beyond their other duties.
Building a Team
Our team was named Division of Radiology Innovation and Value Enhancement (DRIVE) and was envisioned as a bridge between radiology and IT. Two radiologists serve as the co-directors of DRIVE. One director is “operationally” focused and the other “research” focused. However, deliberate coordination between the two areas creates a tightly integrated team. DRIVE is also supported by three full-time data scientists (employed by the department of radiology) with varying skill sets, including full-stack programming and statistical analysis.
For practical reasons, such as the desire for short-term deliverables, we began by using data readily available to us for AI projects. At the same time, we worked to overcome longer-term institutional data governance and cybersecurity concerns. Our DRIVE team simultaneously began focusing on non-interpretive data analysis initiatives designed for automating manual processes, enhancing the patient experience, and streamlining operational efficiencies.
Time-consuming manual processes are a good starting point for new AI projects. Because our institution uses the traditional “resident coverage with subspecialist over-reading” model of overnight staffing, we were interested in assessing discrepancies between residents’ preliminary interpretations and final staff’s interpretations for adult trauma patients. Previously, this was a manual analysis process of querying the trauma database to identify patients and followed by a manual review of radiology reports.
Using a rapid iterative process of development, we retrieved preliminary resident and final attending radiology reports for a specific time period and performed text analysis. The results: The time required for analysis of discrepancies between residents’ and radiologists’ interpretations dropped from several hours to 10 minutes per week. We were also able to demonstrate a low incidence of major discrepancies — those with the potential to change clinical management or incur patient harm.
With this more timely data analytics capability, we now can demonstrate the high quality of overnight resident interpretation, preserve this crucial educational experience, and maintain the value of subspecialty interpretation expertise.
Technology can also enhance the patient experience, which was the goal of our second major automation initiative. We were able to use chatbot technology to improve patient education prior to them undergoing breast biopsy.
Our initial pilot phase showed that the chatbot enhanced patient understanding of the procedure in 86 percent of patients. By accessing analytical data developed as part of the chatbot, we also determined that patients wanted to access the chatbot outside of the hospital setting and via their smartphones. We are now launching a full clinical implementation of the chatbot — enhanced with pictures and video — to all visitors to our breast center.
On the operational front, we have begun using data analysis to guide the operational performance of our specialty divisions. These analyses include assessing the overall staffing needs of the department as well as daily staffing requirements to meet clinical needs. These tools are beginning to change the culture from a reactive, assumptive process to one that is proactive, predictive, and based on hard data.
While there are hurdles to overcome, IT departments are beginning to recognize the benefits of providing departmental users with easier access to data. As we continue to collaborate on additional data science projects, they can see the value of granting access to radiology departments to meet our business needs. At the same time, they can better see the important role of data analysts in increasing efficiency and improving the patient experience.
By Michael M. Moore, MD, associate professor of radiology and pediatrics, and Nabeel I. Sarwani, MD, professor of radiology, at Penn State Health System