Use of Artificial Intelligence-Based Software as Medical Devices for Chest Radiography: A Position Paper from the Korean Society of Thoracic Radiology

Utilization of AI-SaMDs for CR

1) AI-SaMDs currently available for CR cannot replace radiologists' interpretation.

2) AI-SaMDs currently available for CR can be used as an assistance tool in the interpretation by radiologists.

3) AI-SaMDs currently available for CR can be used as a decision support tool for clinicians in a situation where radiologists' interpretation is unavailable.

Performance of AI-SaMDs for CR and Considerations for Its Clinical Application

4) Adjustment or recalibration of AI-SaMDs for CR for the target institution or population is recommended.

Policy and Education on AI-SaMD for CR

5) A separate fee for using AI-SaMDs for CR may be required for its clinical implementation.

6) Education programs for medical students and radiologists assuming the use of AI-SaMDs for CR in daily clinical practice need to be established.

Approved AI-SaMDs for CR in Korea

As of the end of May 2021, the Korean Ministry of Food and Drug Safety (KMFDS) has approved seven AI-SaMDs for CR for clinical use (Table 1) [4]. Every AI-SaMD was approved as an assistant tool for physicians' interpretation, and not as a stand-alone interpretation tool. All approved devices can detect specific abnormalities in a single CR. Three of the seven approved AI-SaMDs can detect only lung nodules, while the others can detect various types of abnormalities. However, AI-SaMDs cannot cover all types of abnormalities that need to be evaluated in a CR. Furthermore, none of the approved devices provide differentiation of detected abnormalities or diagnosis of a specific disease.

Table 1 List of Approved Artificial Intelligence-Based Software as Medical Devices for Chest Radiographs in Korea Click for larger imageClick for full tableDownload as Excel file

Performance of AI-SaMDs for CR and Considerations for Clinical Application

AI-SaMDs for CR have shown excellent performance in early investigations, achieving radiologist-level or higher performances for a single or limited number of pre-specified tasks [5, 6, 7, 8, 9]. However, the reproducibility of those performances in the actual practice remains unclear, since retrospectively collected data may not fully reflect the prevalence and diversity of abnormalities in the actual clinical situation [10, 11]. Several recent investigations reported excellent performance of AI-SaMDs for the identification of specific abnormalities or diseases such as pulmonary nodules [12], tuberculosis [13, 14, 15], and coronavirus disease pneumonia [16, 17], in consecutive cohorts reflecting actual clinical situations. Nevertheless, further investigations validating the performance of AI-SaMDs during their utilization in the real clinical practice should be conducted to confirm the reproducibility of such in the daily practice.

Discussions regarding the application of AI-SaMDs in the daily practice are also necessary. The currently approved method, utilization as an assistant tool for physicians' interpretation is the most traditional and conservative method [18]. Based on studies reporting improved detection performance of radiologists with the assistance of AI-SaMD [5, 6, 7, 8, 19], it would be feasible to use currently available AI-SaMDs as an assistance tool for radiologists' interpretation. In several studies, non-radiologist clinicians also exhibited improvement of detection performance with the assistance of AI-SaMD, and the magnitude of improvement in the clinicians was greater than in the radiologists [7, 8, 20, 21]. Therefore, the use of AI-SaMDs as a decision support tool for clinicians would be acceptable in situations where radiologists' interpretation is unavailable.

Using AI-SaMDs in screening for images with findings of emergency disease requiring timely interpretation, automated assignment of interpreting radiologists according to the presence of abnormality or difficulty of interpretation, and automated notification of suspected interpretive errors have also been proposed [18, 22]. However, their usefulness in clinical practice has yet to be validated.

Finally, the performance of AI-SaMD may differ depending on the characteristics of the target population or institutions, including disease prevalence, diversity of image findings, and equipment or techniques for radiographic acquisition. Therefore, it may be necessary to adjust the threshold of detection or recalibrate the numerical scores of the AI-SaMDs, depending on the characteristics of the target population or institution [23, 24].

Policy on AI-SaMD for CR

Apart from approval for clinical use by the KMFDS, the use of AI-SaMD for CR interpretation currently does not grant additional reimbursement in Korea. According to the guidelines of the Health Insurance Review and Assessment Service in 2019 [25], AI-SaMDs for detection or diagnosis on medical imaging can undergo Health Technology Assessment only when they show significant improvement in accuracy or reduction in errors compared to humans. Based on the result of this assessment, a separate reimbursement can be considered when the AI-SaMD shows a significant improvement in diagnostic performance compared to existing practices, provides new diagnostic information that cannot be obtained using existing practices, or proves therapeutic effectiveness. Currently, medical institutions need to cover the cost of using AI-SaMDs by themselves. In a situation where timely reading of CRs is difficult and considering the risk of interpretive errors, medical institutions may voluntarily implement AI-SaMD at their own expense for patient safety and practice efficiency. However, in order for AI-SaMDs for CR to be implemented in clinical practice in the long term, consideration should be given to charging of a separate fee, as the devices demonstrate further improvement in performance and validation of clinical usefulness.

Education on AI-SaMD for CR

Since interpreting CR using AI-SaMD may gradually expand in daily clinical practice, reorganization of education system for radiologists includint trainess and medical students seems necessary [26, 27]. Education on AI needs to be strengthened, covering a basic understanding of the technology as well as the function and working principle of AI-SaMDs in the interpretation of CR. It would also be important to maintain the traditional education on the technique and knowledge for interpreting CR to accurately interpret and judge the results of AI-SaMD.

Liability Related to the Utilization of AI-SaMD

As AI-SaMDs for CR begin to be applied in clinical practice, there is a growing concern regarding the legal liability for any patient harm related to the utilization of AI-SaMDs. It is difficult to present an evidence-based opinion regarding liability, as AI-SaMD itself or its clinical introduction remains in its infancy. However, considering the current legal system in Korea, an AI-SaMD that assists physicians' interpretation cannot be a legal subject. Thus in case of patient harm, the physician would be held liable.

In case of patient harm, the key factor in determining a physician's liability would be whether the physician followed the existing standard of care [28, 29]. Currently, the standard of care for the interpretation of CR is interpretation by radiologists, and interpreting radiologists' decision to reject the result of AI-SaMD would be within the range of standard of care. Therefore, the discrepancy in interpretation between AI-SaMDs and radiologists cannot be the basis for judging liability. However, if AI-SaMDs are used for unapproved purposes (e.g., using AI-SaMD results without physicians' confirmation), this could be deemed as a deviation from the standard of care, and the physician may incur liability in case of patient harm.