Artificial intelligence passes radiologists’ specialist exams, just about!

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers. Radiologists in the UK must pass the “Fellowship of the Royal College of Radiologists” (FRCR) exam as part of their training. In one part of this exam, the trainee radiologists are asked to interpret 30 X-rays within 35 minutes and need to get >90% correct to pass.

A publication in the BMJ Christmas issue, led by Dr Susan Shelmerdine at GOSH and Dr Jonathan Weir-McCall at University of Cambridge, shows for the first time, that an artificial intelligence (AI) tool could pass this exam too.

The commercially available AI tool had been trained on datasets of almost 1 million chest and bone X-rays and taught to identify seven key problems including broken bones and abnormalities in the lungs.

10 challenging mock rapid-reporting examinations were evaluated by the AI and by 26 newly-qualified radiologists. The AI was able to pass half the number of mock examinations that the radiologists could pass, when only the images the AI had been trained on were included.

The AI tool also correctly identified 90.5% of all the X-rays that most radiologists also got correct. When the AI was incorrect, it was mostly (64.2%) flagging an abnormality that wasn't there, a false-positive, rather than missing a problem.

There were around 10 X-rays where the AI identified abnormalities that most trained radiologists missed. These were often in images of the extremities, such has the hands and feet, where there are many small bones to assess. This suggests AI could be particularly useful to look at scans which are more time-consuming to evaluate.