The classic website will no longer be available as of June 25, 2024. Please use the modernized ClinicalTrials.gov.
Working…
ClinicalTrials.gov
ClinicalTrials.gov Menu

AI Assisted Reader Evaluation in Acute Computed Tomography (CT) Head Interpretation (AI-REACT)

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been evaluated by the U.S. Federal Government. Read our disclaimer for details.
 
ClinicalTrials.gov Identifier: NCT06018545
Recruitment Status : Active, not recruiting
First Posted : August 30, 2023
Last Update Posted : November 8, 2023
Sponsor:
Information provided by (Responsible Party):
Alex Novak, Oxford University Hospitals NHS Trust

Study Description
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information
Brief Summary:

This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838).

The purpose of the study is to assess the impact of an Artificial Intelligence (AI) tool called qER 2.0 EU on the performance of readers, including general radiologists, emergency medicine clinicians, and radiographers, in interpreting non-contrast CT head scans. The study aims to evaluate the changes in accuracy, review time, and diagnostic confidence when using the AI tool. It also seeks to provide evidence on the diagnostic performance of the AI tool and its potential to improve efficiency and patient care in the context of the National Health Service (NHS). The study will use a dataset of 150 CT head scans, including both control cases and abnormal cases with specific abnormalities. The results of this study will inform larger follow-up studies in real-life Emergency Department (ED) settings.


Condition or disease Intervention/treatment
Intracranial Hemorrhages Acute Ischemic Stroke Hydrocephalus Cerebral Infarction Cerebral Edema Cerebral Injury Other: Ground truthing Other: Reading

Study Design
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information

Layout table for study information
Study Type : Observational
Actual Enrollment : 33 participants
Observational Model: Cohort
Time Perspective: Retrospective
Official Title: AI Assisted Reader Evaluation in Acute CT Head Interpretation
Actual Study Start Date : June 1, 2023
Actual Primary Completion Date : September 1, 2023
Estimated Study Completion Date : September 1, 2024

Resource links provided by the National Library of Medicine


Groups and Cohorts
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information

Group/Cohort Intervention/treatment
Readers
30 readers will be recruited across four NHS trusts including ten general radiologists, fifteen emergency medicine clinicians, and five CT radiographers of varying seniority. Readers will interpret each scan first without, then with, the assistance of the AI tool, with an intervening 4-week washout period. Using a panel of neuroradiologists as ground truth, the stand-alone performance of qER will be assessed, and its impact on the readers' performance will be analysed as change in accuracy, mean review time per scan, and self-reported diagnostic confidence. Subgroup analyses will be performed by reader professional group, reader seniority, pathological finding, and neuroradiologist-rated difficulty.
 Other: Reading
All 30 readers will review all 150 cases, in each of two study phases. The readers will provide their opinion on the presence or absence of some acute abnormalities, including intracranial haemorrhage, infarct, midline shift and fracture. They will provide a confidence in their diagnosis (10-point visual analogue scale), and a single click point to mark the location of each abnormality that they consider as being present. The time taken for each scan will be automatically recorded.
Ground truthers
Two Consultant neuroradiologists will independently review the images to establish the 'ground truth' findings on the CT scans which will be used as the reference standard. In the case of disagreement, a third senior neuroradiologist's opinion will be sought for arbitration. A difficulty score will be assigned to each scan by the ground truthers using a 5-point Likert scale.
 Other: Ground truthing
Two Consultant neuroradiologists will independently review the images to establish the 'ground truth' findings on the CT scans which will be used as the reference standard. In the case of disagreement, a third senior neuroradiologist's opinion will be sought for arbitration.


Outcome Measures
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information

Primary Outcome Measures :
  1. Reader performance: Sensitivity, specificity, comparative between with and without AI assistance. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    Reader performance will be evaluated as sensitivity, specificity, with and without AI assistance.

  2. Reader performance: Positive and negative predictive value, comparative between with and without AI assistance. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    Reader performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV), with and without AI assistance.

  3. Reader performance: Area Under Receiver Operating Characteristic Curve (AUROC), comparative between with and without AI assistance. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    Reader performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC), with and without AI assistance.

  4. Reader speed: Mean time taken to review a scan, with versus without AI assistance. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    Reader speed will be evaluated as the man time taken to review a scan, using time unite of seconds.

  5. Reader confidence: Self-reported diagnostic confidence on a 10 point visual analogue scale, with vs without AI assistance. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    On the reading platform (RAIQC), one of the questions asks the level of confidence that the participant has in their diagnostic opinion. The question offers a scale of 1 to 10, where 1 is not confident, and 10 is highly confident.

  6. qER (AI algorithm) performance: Sensitivity and specificity [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    qER performance will be evaluated as sensitivity, specificity.

  7. qER (AI algorithm) performance: Positive and negative predictive value. [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    qER performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV).

  8. qER (AI algorithm) performance: Area Under Receiver Operating Characteristic Curve (AUROC). [ Time Frame: During 6 weeks, which is the period for reading or reviewing the cases/scans. ]
    qER performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC)


Eligibility Criteria
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information

Information from the National Library of Medicine

Choosing to participate in a study is an important personal decision. Talk with your doctor and family members or friends about deciding to join a study. To learn more about this study, you or your doctor may contact the study research staff using the contacts provided below. For general information, Learn About Clinical Studies.


Layout table for eligibility information
Ages Eligible for Study:   Child, Adult, Older Adult
Sexes Eligible for Study:   All
Accepts Healthy Volunteers:   Yes
Sampling Method:   Non-Probability Sample
Study Population

Setting:

Readers will be recruited from the following four hospital Trusts (secondary and tertiary level:

  • Guy's & St Thomas NHS Foundation Trust
  • Northumbria Healthcare NHS Foundation Trust
  • NHS Greater Glasgow and Clyde
  • Oxford University Hospitals NHS Foundation Trust

Participants:

30 volunteer participant readers will be selected from the following groups:

  • Emergency Medicine Consultants and Registrars (5 Consultant, 5 Registrar (ST3-6), 5 junior (F1-ST2)
  • General Radiologist Consultants and Registrars (5 Consultant, 5 Registrar)
  • 5 CT Radiographers
Criteria

Inclusion Criteria:

  • Radiologists/Radiographers/ED clinicians who review CT head scans as part of their clinical practice

Exclusion Criteria:

  • Neuroradiologists.
  • Non-radiologist groups: Clinicians with previous formal postgraduate CT reporting training
  • Emergency Medicine group: Clinicians with previous career in radiology/neurosurgery to registrar level
Contacts and Locations
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information

Information from the National Library of Medicine

To learn more about this study, you or your doctor may contact the study research staff using the contact information provided by the sponsor.

Please refer to this study by its ClinicalTrials.gov identifier (NCT number): NCT06018545


Locations
Layout table for location information
United Kingdom
Oxford University Hospitals NHS Foundation Trust
Oxford, Oxfordshire, United Kingdom, OX3 9DU
NHS Greater Glasgow and Clyde
Glasgow, United Kingdom, G12 0XH
Guy's & St Thomas NHS Foundation Trust
London, United Kingdom, SE1 7EH
Northumbria Healthcare NHS Foundation Trust
Newcastle Upon Tyne, United Kingdom, NE27 0QJ
Sponsors and Collaborators
Oxford University Hospitals NHS Trust
Investigators
Layout table for investigator information
Principal Investigator: Alex Novak, MSc National Health Services in the United Kingdom (NHS UK)
Principal Investigator: Sarim Ather, PhD National Health Services in the United Kingdom (NHS UK)
More Information
Go to 
Top of Page Study Description Study Design Groups and Cohorts Outcome Measures Eligibility Criteria Contacts and Locations More Information
Additional Information:

Publications:

Layout table for additonal information
Responsible Party: Alex Novak, Primary Investigator, Oxford University Hospitals NHS Trust
ClinicalTrials.gov Identifier: NCT06018545    
Other Study ID Numbers: 310995 - A
First Posted: August 30, 2023    Key Record Dates
Last Update Posted: November 8, 2023
Last Verified: August 2023

Layout table for additional information
Studies a U.S. FDA-regulated Drug Product: No
Studies a U.S. FDA-regulated Device Product: No
Keywords provided by Alex Novak, Oxford University Hospitals NHS Trust:
Radiology
Head tomography
Emergency medicine
Radiographer
Artificial intelligence
Additional relevant MeSH terms:
Layout table for MeSH terms
Ischemic Stroke
Hydrocephalus
Cerebral Infarction
Intracranial Hemorrhages
Brain Edema
Brain Injuries
Infarction
Hemorrhage
Ischemia
Pathologic Processes
Necrosis
Stroke
 Cerebrovascular Disorders
Brain Diseases
Central Nervous System Diseases
Nervous System Diseases
Vascular Diseases
Cardiovascular Diseases
Brain Infarction
Brain Ischemia
Craniocerebral Trauma
Trauma, Nervous System
Wounds and Injuries