Artificial intelligence fails radiology qualifying examination

iStock.com/Shidlovski

Artificial intelligence cannot yet pass one of the qualifying radiology examinations, according to a new study out today.

Although AI is increasingly being used for some tasks that doctors do, such as interpreting radiographs, it is not yet ready to replace doctors, researchers from Great Ormond Street Hospital, London, report in The BMJ.

The team compared the performance of a commercially available AI tool with 26 radiologists, all of whom had passed the Fellowship of the Royal College of Radiologists (FRCR) examination the previous year.

They developed 10 “mock” rapid reporting exams, based on one of three modules that make up the qualifying FRCR examination.

Each mock exam comprised 30 radiographs at the same or a higher level of difficulty and breadth of knowledge expected for the real FRCR exam and to pass, candidates had to correctly interpret at least 27 (90%) of the 30 images within 35 minutes.

The AI candidate had been trained to assess chest and musculoskeletal radiographs for several conditions including fractures, swollen and dislocated joints, and collapsed lungs, although allowances were made for images relating to body parts the AI candidate had not been trained in.