A new study finds that artificial intelligence (AI) is able to aid physicians in interpreting X-rays accurately after injuries that involve suspected fractures (Radiology 2021; https://doi.org/10.1148/radiol.210937). Citing missed fractures as a common cause of diagnostic discrepancy between initial radiographic interpretation and the final read by board-certified radiologists, a team including researchers from Boston University School of Medicine and Harvard Medical School sought to assess the effect of assistance by AI on diagnostic performances of physicians for fractures on radiographs.
Fracture interpretation can represent up to 24 percent of harmful diagnostic errors seen in the emergency department, noted the authors, adding that inconsistencies in radiographic diagnosis of fractures are more common during the evening and overnight hours (5 p.m. to 3 a.m.), likely due to non-expert reading and fatigue. In patients with multiple traumas, the proportion of missed injuries, including fractures, can be high on the forearm and hands as well as the feet, according to the researchers.
The study used the multi-reader, multi-case methodology based on an external multicenter dataset of 480 examinations with at least 60 examinations per body region-foot and ankle, knee and leg, hip and pelvis, hand and wrist, elbow and arm, shoulder and clavicle, rib cage and thoracolumbar spine-between July 2020 and January 2021. Fracture prevalence was set at 50 percent, according to the authors, who noted that "the ground truth was determined by two musculoskeletal radiologists, with discrepancies solved by a third."
Overall, 24 readers-radiologists, orthopedists, emergency physicians, physician assistants, rheumatologists, family physicians-were presented the whole validation set, with and without AI assistance, with a 1-month minimum washout period, according to the study. The authors pointed out that the primary analysis "had to demonstrate superiority of sensitivity per patient and the noninferiority of specificity per patient at -3 percent margin with AI aid. Stand-alone AI performance was also assessed using receiver operating characteristic curves."
The researchers trained the AI algorithm on X-rays to detect fractures, ultimately finding that the sensitivity per patient was 10.4 percent higher with the aid of AI than without the use of it. The specificity per patient with AI aid was non-inferior to that without the aid of AI, with a difference of +5 percent, with AI shortening the average reading time by more than 6 seconds per examination.
The sensitivity by patient gain was significant in all regions but the shoulder, clavicle, and spine, according to the authors. Overall, AI assistance improved the sensitivity and "may even improve the specificity of fracture detection by radiologists and non-radiologists without lengthening reading time," the researchers wrote.
The AI algorithm used in this study "can quickly and automatically detect X-rays that are positive for fractures and flag those studies in the system so that radiologists can prioritize reading X-rays with positive findings," noted Ali Guermazi, MD, PhD, Chief of Radiology at VA Boston Healthcare System, Professor of Radiology and Medicine at Boston University School of Medicine, and the study's corresponding author.
With the AI algorithm, X-rays are taken and automatically forwarded to a server that has the algorithm installed on it. The software analyzes the images, coming to one of three conclusions: fracture, no fracture, or doubt. The algorithm is capable of providing certainty of a fracture in most assessments, showing with a fixed line where the fracture is located. In cases where a fracture is suspected, the software will mark the fracture with a dotted line.
"The system also highlights regions of interest with bounding boxes around areas where fractures are suspected and provides an instant second reading to reduce errors," added Guermazi. "This can potentially contribute to less waiting time at the time of hospital or clinic visit before patients can get a final diagnosis of fracture and should help guarantee optimal patient management."
Looking forward, Guermazi foresees AI and this algorithm serving as a valuable tool to help radiologists and the rest of the radiology team improve diagnostic performance.
In this study, AI assistance helped reduce missed fractures by 29 percent and increased readers' sensitivity by 16 percent, and by 30 percent for exams with more than one fracture, while improving specificity by 5 percent, he noted.
Given such results, "this tool will be helpful to prioritize radiologists' readings for patients with a definitive or suspected fracture by the AI as it provides a summary at the examination level. It should also shorten the interpretation time by highlighting regions of interest directly on the images, thus improving radiologist's reading workflow and acting as a safety net to ensure diagnostic accuracy," Guermazi stated. "Indeed, missed fractures are the second most common diagnosis leading to medical claim in the United States, and account for 24 percent of harmful diagnostic errors in the emergency room."
In the future, Guermazi believes that "AI will assist radiologists more and more to improve and standardize diagnostic performances and improve constrained workflows, so that radiologists can focus on higher value and more complex tasks.
Mark McGraw is a contributing writer.