TY - JOUR
T1 - Commercial artificial intelligence lateral cephalometric analysis
T2 - part 2—effects of human examiners on artificial intelligence performance, a pilot study
AU - Lee, Jaesik
AU - Bae, Seong Ryeol
AU - Noh, Hyung Kyu
N1 - Publisher Copyright:
© 2023 The Author(s). Published by MRE Press.
PY - 2023/11
Y1 - 2023/11
N2 - At the current technology level, a human examiner’s review must be accompanied to compensate for the insufficient commercial artificial intelligence (AI) performance. This study aimed to investigate the effects of the human examiner’s expertise on the efficacy of AI analysis, including time-saving and error reduction. Eighty-four pretreatment cephalograms were randomly selected for this study. First, human examiners (one beginner and two regular examiners) manually detected 15 cephalometric landmarks and measured the required time. Subsequently, commercial AI services automatically identified these landmarks. Finally, the human examiners reviewed the AI landmark determination and adjusted them as needed while measuring the time required for the review process. Then, the elapsed time was compared statistically. Systematic and random errors among examiners (human examiners, AI and their combinations) were assessed using the Bland-Altman analysis. Intraclass correlation coefficients were used to estimate the inter-examiner reliability. No clinically significant time difference was observed regardless of AI use. AI measurement error decreased substantially after the review of the human examiner. From the standpoint of the human examiner, beginners could obtain better results than manual landmarking. However, the AI review outcomes of the regular examiner were not as good as those of manual analysis, possibly due to AI-dependent landmark decisions. The reliability of AI analysis could also be improved by employing the human examiner’s review. Although the time-saving effect was not evident, commercial AI cephalometric services are currently recommendable for beginners.
AB - At the current technology level, a human examiner’s review must be accompanied to compensate for the insufficient commercial artificial intelligence (AI) performance. This study aimed to investigate the effects of the human examiner’s expertise on the efficacy of AI analysis, including time-saving and error reduction. Eighty-four pretreatment cephalograms were randomly selected for this study. First, human examiners (one beginner and two regular examiners) manually detected 15 cephalometric landmarks and measured the required time. Subsequently, commercial AI services automatically identified these landmarks. Finally, the human examiners reviewed the AI landmark determination and adjusted them as needed while measuring the time required for the review process. Then, the elapsed time was compared statistically. Systematic and random errors among examiners (human examiners, AI and their combinations) were assessed using the Bland-Altman analysis. Intraclass correlation coefficients were used to estimate the inter-examiner reliability. No clinically significant time difference was observed regardless of AI use. AI measurement error decreased substantially after the review of the human examiner. From the standpoint of the human examiner, beginners could obtain better results than manual landmarking. However, the AI review outcomes of the regular examiner were not as good as those of manual analysis, possibly due to AI-dependent landmark decisions. The reliability of AI analysis could also be improved by employing the human examiner’s review. Although the time-saving effect was not evident, commercial AI cephalometric services are currently recommendable for beginners.
KW - Accuracy
KW - Artificial intelligence
KW - Cephalometric
KW - Efficacy
KW - Precision
KW - Reliability
UR - http://www.scopus.com/inward/record.url?scp=85176752479&partnerID=8YFLogxK
U2 - 10.22514/jocpd.2023.087
DO - 10.22514/jocpd.2023.087
M3 - Article
C2 - 37997244
AN - SCOPUS:85176752479
SN - 1053-4628
VL - 47
SP - 130
EP - 141
JO - Journal of Clinical Pediatric Dentistry
JF - Journal of Clinical Pediatric Dentistry
IS - 6
ER -