You are here

  • Home
  • Archive
  • Volume 168, Issue 5
  • 3* Evaluation of an Artificial Intelligence (AI) system to augment clinical risk prediction of Trauma Induced Coagulopathy in the pre-hospital setting: a prospective observational study

Article Text

Article menu
Download PDFPDF
Royal Society of Medicine, Colt Foundation Research Prize Finalists
3* Evaluation of an Artificial Intelligence (AI) system to augment clinical risk prediction of Trauma Induced Coagulopathy in the pre-hospital setting: a prospective observational study
  1. Max Marsden1,2,
  2. Zane Perkins2,3,
  3. William Marsh4,
  4. Michael Christian3,
  5. Richard Lyon5,6,
  6. Ross Davenport2 and
  7. Nigel Tai1,2
  1. 1Academic Department of Military Surgery and Trauma, Royal Centre for Defence Medicine, Birmingham, UK
  2. 2Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, UK
  3. 3London’s Air Ambulance, London, UK
  4. 4School of Electronical Engineering and Computer Sciences, Queen Mary University of London, London, UK
  5. 5Air Ambulance Kent Surrey Sussex, UK
  6. 6School of Health Sciences, University of Surrey, UK

Abstract

Background The potential of AI systems to support pre-hospital clinical decision-making in both military and civilian settings is significant. However, whilst such algorithms are increasingly available, far less attention has been paid to understanding the impact of such systems on clinical performance. This study had two aims; first, to compare the performance of expert clinicians against an AI system in a real-world clinical setting and second, to assess the impact of augmenting expert clinical prediction with an AI system.

Methods Two civilian UK Air Ambulances were selected as surrogate settings relevant to military practice. We performed a prospective study over a six-month period where expert pre-hospital clinicians’ judgement of the risk of Trauma Induced Coagulopathy (TIC) in injured patients was assessed and compared to the performance of an AI system. Two TIC risk predictions were generated for every patient: an AI prediction and a human prediction. Measures of predictive performance included discrimination, calibration, and overall accuracy.

Results Overall, 51 expert clinicians were enrolled in the study providing 184 patient interactions for analysis. The studied patients had a median age of 31 (range 16, 89), median injury severity score of 17 (IQR 9, 34), 75% were male, and 19% developed TIC.

Aim 1: The AI system performed better than clinicians; higher discrimination [AUROC 0.87 (0.79, 0.95) versus 0.83 (0.74, 0.92)] better calibration [0.37 (-0.14, 0.89) versus -1.19 (-1.73, -0.65)] and more accurate [Brier Skill Score 0.34 (0.19, 0.48) versus 0.00 (-0.41, 0.30)].

Aim 2: Risk prediction was better in all performance metrics when clinicians were assisted with the AI system [AUROC 0.88 (0.80, 0.95) versus 0.83 (0.74, 0.92)]

Conclusions AI systems can improve human risk prediction in the pre-hospital setting. In the military environment, where austerity and lack of senior clinical expertise may affect outcomes, the benefit of implementing predictive AI should be substantial.

(*awarded First Place)

https://doi.org/10.1136/bmjmilitary-2022-RSMabstracts.3

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.