Artificial intelligence passes radiologists’ specialist exams, just about!

28 Dec 2022, 9 a.m.

X-ray image of hand

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers.

In the UK they must pass a specialist exam as part of their training. For the first time, researchers have shown that an artificial intelligence (AI) tool could pass this exam too.

Can artificial intelligence pass the exam?

Radiologists are specialist doctors that use scans to diagnose conditions, from broken bones to cancers. Radiologists in the UK must pass the “Fellowship of the Royal College of Radiologists” (FRCR) exam as part of their training. In one part of this exam, the trainee radiologists are asked to interpret 30 X-rays within 35 minutes and need to get >90% correct to pass.

A publication in the BMJ Christmas issue, led by Dr Susan Shelmerdine at GOSH and Dr Jonathan Weir-McCall at University of Cambridge, shows for the first time, that an artificial intelligence (AI) tool could pass this exam too.

Training artificial intelligence

The commercially available AI tool had been trained on datasets of almost 1 million chest and bone X-rays and taught to identify seven key problems including broken bones and abnormalities in the lungs.

10 challenging mock rapid-reporting examinations were evaluated by the AI and by 26 newly-qualified radiologists. The AI was able to pass half the number of mock examinations that the radiologists could pass, when only the images the AI had been trained on were included.

The AI tool also correctly identified 90.5% of all the X-rays that most radiologists also got correct. When the AI was incorrect, it was mostly (64.2%) flagging an abnormality that wasn't there, a false-positive, rather than missing a problem.

There were around 10 X-rays where the AI identified abnormalities that most trained radiologists missed. These were often in images of the extremities, such has the hands and feet, where there are many small bones to assess. This suggests AI could be particularly useful to look at scans which are more time-consuming to evaluate.

This is the first time an AI tool has gone head-to-head with humans on a radiology exam like this. Although radiologists came out on top, the AI got good grades too, even though it wasn’t built with the intention of passing exams!

AI tools like this are becoming more advanced each year, and we hope this study can help people understand how AI can support clinical work already being carried out by doctors to provide better care for patients. AI won’t replace doctors anytime soon, but there’s real potential for AI support, particularly where specialised radiology services are limited.

Dr Susan Shelmerdine, consultant radiologist and lead author at GOSH, and honorary associate professor at UCL GOS Institute of Child Health

Reducing oxygen levels for children in intensive care will save lives

A new landmark study led from GOSH has found that reducing oxygen levels for critically ill children on mechanical ventilators in intensive care could save lives

GOSH to provide support for AI and Machine Learning as part of new paediatric HealthTech Research Centre

GOSH will support the new Child and Young People’s HealthTech Research Centre (HRC), led by Sheffield Children’s Hospital, as leads for AI and Machine Learning.

Clinical trial results give new hope for children with rare brain tumours

Researchers who are searching for better treatments for an incredibly rare type of brain tumour have published successful results from the latest rounds of clinical trials.

GOSH receives new MHRA authorisation to manufacture viral vectors

GOSH has been granted a Manufacturer's Authorisation Licence for the manufacture of viral vectors, which will accelerate Cell and gene therapies (CGT) clinical trials worldwide and expand the novel treatments that we can offer to our patients.