Artificial intelligence passes radiologists’ specialist exams, just about!

28 Dec 2022, 9 a.m.

X-ray image of hand

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers.

In the UK they must pass a specialist exam as part of their training. For the first time, researchers have shown that an artificial intelligence (AI) tool could pass this exam too.

Can artificial intelligence pass the exam?

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers. Radiologists in the UK must pass the “Fellowship of the Royal College of Radiologists” (FRCR) exam as part of their training. In one part of this exam, the trainee radiologists are asked to interpret 30 X-rays within 35 minutes and need to get >90% correct to pass.

A publication in the BMJ Christmas issue, led by Dr Susan Shelmerdine at GOSH and Dr Jonathan Weir-McCall at University of Cambridge, shows for the first time, that an artificial intelligence (AI) tool could pass this exam too.

Training artificial intelligence

The commercially available AI tool had been trained on datasets of almost 1 million chest and bone X-rays and taught to identify seven key problems including broken bones and abnormalities in the lungs.

10 challenging mock rapid-reporting examinations were evaluated by the AI and by 26 newly-qualified radiologists. The AI was able to pass half the number of mock examinations that the radiologists could pass, when only the images the AI had been trained on were included.

The AI tool also correctly identified 90.5% of all the X-rays that most radiologists also got correct. When the AI was incorrect, it was mostly (64.2%) flagging an abnormality that wasn't there, a false-positive, rather than missing a problem.

There were around 10 X-rays where the AI identified abnormalities that most trained radiologists missed. These were often in images of the extremities, such has the hands and feet, where there are many small bones to assess. This suggests AI could be particularly useful to look at scans which are more time-consuming to evaluate.

This is the first time an AI tool has gone head-to-head with humans on a radiology exam like this. Although radiologists came out on top, the AI got good grades too, even though it wasn’t built with the intention of passing exams!

AI tools like this are becoming more advanced each year, and we hope this study can help people understand how AI can support clinical work already being carried out by doctors to provide better care for patients. AI won’t replace doctors anytime soon, but there’s real potential for AI support, particularly where specialised radiology services are limited.

Dr Susan Shelmerdine, consultant radiologist and lead author at GOSH, and honorary associate professor at UCL GOS Institute of Child Health

GOSH Surgeon Paolo De Coppi makes first TIME100 Health List

Professor Paolo De Coppi has been added to TIME's inaugural TIME100 Health List of the 100 most influential people in global health this year

Two years of advancing personalised healthcare for children in collaboration with Roche

As we mark the milestone of our second year working together with Roche, we took some time to reflect on the incredible progress we have made in our journey towards advancing personalised healthcare for children.

New treatment for brain tumour approved after over 20 years of research

The first-ever targeted treatment for brain tumours in children has been approved for NHS patients, following decades of research by a Great Ormond Street consultant.

Help pioneer new treatments for millions of people this DNA Day

DNA Day is coming up this Thursday (25 April) and the team behind the DNA, Children + Young People’s Health Resource (D-CYPHR) are encouraging children and young people to contribute to important health research.