Evaluation of the accuracy of an artificial intelligence in identifying contraindications to exercise therapy : Comparison with and interrater reliability of physical therapists judgments
- Purpose The study validates a rule-based system for identifying contraindications to exercise therapy in a medical context. It evaluates accuracy and performance by comparing it with physical therapists’ assessments and patients' characteristics. Method The dataset included 80 patient cases with clinical characteristics assessed by 20 physical therapists for contraindications to exercise therapy. Fleiss kappa and pooled kappa values measured agreement between physical therapists and AI. AI performance was assessed by sensitivity, specificity, accuracy and F1 score. Clinical characteristics were compared between therapists' votes using ANOVA and Bonferroni post-hoc test. Results The physical therapists had a mean age of 40.85 (8.23) years and a mean experience of 14.53 (8.20) years. Out of 64 patient cases, there was consensus on 35 cases with no contraindication and 29 cases with a consensus on “contraindication exists” for exercise therapy. In 16 cases there was no consensus between therapists. Overall, therapists had 87.5% agreement with Fleiss Kappa κπ = .43. The pooled kappa value between therapists and AI was κpooled = .63. AI achieved perfect values (1) for sensitivity, specificity, accuracy and F1 score. Statistically, consensus-based comparisons by therapists revealed significant differences in pain intensity, duration, timing, and quality. Conclusion The study shows significant agreement between physical therapists and the AI, consistent with similar musculoskeletal studies. Various clinical characteristics highlight the importance of clinical reasoning and contraindication detection. In conclusion, advanced technologies such as decision support and expert systems could have a profound impact on clinical practice, improving accuracy, personalized exercises and telemedicine referrals for efficient care and improved patient decisions.
Author: | Annika GriefahnORCiD, Christoff ZalpourORCiD, Kerstin LuedtkeORCiD |
---|---|
Title (English): | Evaluation of the accuracy of an artificial intelligence in identifying contraindications to exercise therapy : Comparison with and interrater reliability of physical therapists judgments |
DOI: | https://doi.org/10.1007/s12553-024-00827-w |
ISSN: | 2190-7196 |
ISSN: | 2190-7188 |
Parent Title (English): | Health and Technology |
Document Type: | Article |
Language: | English |
Year of Completion: | 2024 |
electronic ID: | Zur Anzeige in scinos |
Release Date: | 2024/07/05 |
Issue: | 14 |
First Page: | 513 |
Last Page: | 522 |
Note: | Zugriff im Hochschulnetz |
Faculties: | Fakultät WiSo |
DDC classes: | 600 Technik, Medizin, angewandte Wissenschaften / 610 Medizin, Gesundheit |
Review Status: | Veröffentlichte Fassung/Verlagsversion |