A guideline of selecting and reporting intraclass correlation coefficients for reliability research

wbldb

Koo, Terry K.¹; Li, Mae Y.²

A guideline of selecting and reporting intraclass correlation coefficients for reliability research

J Chiropr Med. June 2016;15(2):155-163

©2015 National University of Health Sciences.

Affiliations

¹Director & Associate Professor, Foot Levelers Biomechanics Research Laboratory, New York Chiropractic College, Seneca Falls, NY

²DC Candidate, Foot Levelers Biomechanics Research Laboratory, New York Chiropractic College, Seneca Falls, NY
Abstract

Objective: Intraclass correlation coefficient (ICC) is a widely used reliability index in test-retest, intrarater, and interrater reliability analyses. This article introduces the basic concept of ICC in the content of reliability analysis.

Discussion for Researchers: There are 10 forms of ICCs. Because each form involves distinct assumptions in their calculation and will lead to different interpretations, researchers should explicitly specify the ICC form they used in their calculation. A thorough review of the research design is needed in selecting the appropriate form of ICC to evaluate reliability. The best practice of reporting ICC should include software information, “model,” “type,” and “definition” selections.

Discussion for Readers: When coming across an article that includes ICC, readers should first check whether information about the ICC form has been reported and if an appropriate ICC form was used. Based on the 95% confident interval of the ICC estimate, values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.

Conclusion: This article provides a practical guideline for clinical researchers to choose the correct form of ICC and suggests the best practice of reporting ICC parameters in scientific publications. This article also gives readers an appreciation for what to look for when coming across ICC while reading an article.
Links

DOI: 10.1016/j.jcm.2016.02.012

PubMed: 27330520

PubMedCentral: PMC4913118
History

Received: 2015-07-30

Revised: 2015-11-3

Accepted: 2015-11-9

Online: 2016-03-31

Cited Works (2)

Year	Entry
1979	Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420-428.
1986	Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. February 8, 1986;327(8476):307-310.

Cited By (57)

Year	Entry
2022	Goggins KA, Tetzlaff EJ, Young WW, Godwin AA. SARS-CoV-2 (Covid-19) workplace temperature screening: seasonal concerns for thermal detection in northern regions. Appl Ergon. January 2022;98:103576.
2021	Whitney DG, Alford AI, Devlin MJ, Li Y, Caird MS. Intersite reliability of vertebral bone marrow lipidomics-derived lipid composition among children with varying degrees of bone fragility undergoing routine orthopedic surgery. Bone. February 2021;143:115633.
2023	Durongbhan P, Silva MO, Li Z, Ansari N, Kour RYN, Davey CE, Stok KS. A microCT imaging protocol for reproducible and efficient quantitative morphometric analysis (QMA) of joint structures of the in situ mouse tibio-femoral joint. Bone. January 2023;166:116606.
2023	Vu B-TD, Jones BC, Lee H, Kamona N, Deshpande RS, Wehrli FW, Rajapakse CS. Six-minute, in vivo MRI quantification of proximal femur trabecular bone 3D microstructure. Bone. December 2023;177:116900.
2023	Onuma K, Yanai T. A method for measuring muscle strength in restraining valgus joint angulation: elbow varus muscle strength against valgus loading. J Biomech. January 2023;147:111427.
2023	Di Bacco VE, Gage WH. Evaluation of a smartphone accelerometer system for measuring nonlinear dynamics during treadmill walking: concurrent validity and test-retest reliability. J Biomech. April 2023;151:111527.
2023	Wang Z, Destro A, Petersson S, Cenni F, Wang R. In vivo 3D muscle architecture quantification based on 3D freehand ultrasound and magnetic resonance imaging. J Biomech. May 2023;152:111567.
2023	Andersen JT, McCarthy AM, Wills JA, Fuller JT, Lenton GK, Doyle TLA. A markerless motion capture system can reliably determine peak trunk flexion while squatting with and without a weighted vest. J Biomech. May 2023;152:111587.
2023	Song Y, Cen X, Chen H, Sun D, Munivrana G, Bálint K, Bíró I, Gu Y. The influence of running shoe with different carbon-fiber plate designs on internal foot mechanics: a pilot computational analysis. J Biomech. May 2023;153:111597.
2023	Ripic Z, Signorile JF, Best TM, Jacobs KA, Nienhuis M, Whitelaw C, Moenning C, Eltoukhy M. Validity of artificial intelligence-based markerless motion capture system for clinical gait analysis: spatiotemporal results in healthy adults and adults with Parkinson’s disease. J Biomech. June 2023;155:111645.
2023	Mattiussi AM, Shaw JW, Price P, Brown DD, Cohen DD, Lineham J, Pedlar CR, Tallent J, Atack AC. Reliability of ankle mechanics during jump landings in turned-out and parallel foot positions in professional ballet dancers. J Biomech. July 2023;156:111662.
2023	Leporace G, Metsavaht L, Gonzalez FF, Arcanjo de Jesus F, Machado M, Celina Guadagnin E, Gomes-Neto M. Validity and reliability of two-dimensional video-based assessment to measure joint angles during running: a systematic review and meta-analysis. J Biomech. August 2023;157:111747.
2023	Ripic Z, Theodorakos I, Andersen MS, Signorile JF, Best TM, Jacobs KA, Eltoukhy M. Prediction of gait kinetics using Markerless-driven musculoskeletal modeling. J Biomech. August 2023;157:111712.
2023	Cormier P, Tsai M-C, Meylan C, Klimstra M. Comparison of acceleration-speed profiles from training and competition to individual maximal sprint efforts. J Biomech. August 2023;157:111724.
2023	Ripic Z, Nienhuis M, Signorile JF, Best TM, Jacobs KA, Eltoukhy M. A comparison of three-dimensional kinematics between markerless and marker-based motion capture in overground gait. J Biomech. October 2023;159:111793.
2023	Hashimoto S, Fujimori T, Ohyama-Byun K, Okamoto Y, Nakajima T, Sado N. Distal muscle cross-sectional area is correlated with shot put performance. J Biomech. November 2023;160:111819.
2023	Straatman L, Norman E, Knowles N, Walton D, Suh N, Lalone E. Use it or lose it: the relationship between two image-based biomarkers in better understanding osteoarthritis progression in the wrist. J Biomech. December 2023;161:111849.
2024	Kvist A, Tinmark F, Bezuidenhout L, Reimeringer M, Conradsson DM, Franzén E. Validation of algorithms for calculating spatiotemporal gait parameters during continuous turning using lumbar and foot mounted inertial measurement units. J Biomech. January 2024;162:111907.
2022	Iconaru L, Charles A, Baleanu F, Surquin M, Benoit F, Mugisha A, Moreau M, Paesmans M, Karmali R, Rubinstein M, Rozenberg S, Body J-J, Bergmann P. Prediction of an imminent fracture after an index fracture: models derived from the Frisbee cohort. J Bone Miner Res. January 2022;37(1):59-67.
2022	Florkow MC, Willemsen K, Zijlstra F, Foppen W, van der Wal BCH, van der Voort van Zyp JRN, Viergever MA, Castelein RM, Weinans H, van Stralen M, Sakkers RJB, Seevinck PR. MRI-based synthetic CT shows equivalence to conventional CT for the morphological assessment of the hip joint. J Orthop Res. April 2022;40(4):954-964.
2022	Requist MR, Sripanich Y, Rolvien T, Lenz AL, Barg A. Micro-CT analysis of the Lisfranc complex reveals higher bone mineral density in dorsal compared to plantar regions. J Orthop Res. June 2022;40(6):1457-1469.
2022	Cone SG, Barnes RH, Howe D, Fordham LA, Fisher MB, Spang JT. Age- and sex-specific differences in ACL and ACL bundle size during adolescent growth. J Orthop Res. July 2022;40(7):1613-1620.
2022	Yang L, Martin JA, Brouillette MJ, Buckwalter JA, Goetz JE. Objective evaluation of chondrocyte density & cloning after joint injury using convolutional neural networks. J Orthop Res. November 2022;40(11):2609-2619.
2022	Kuiper RJA, Sakkers RJB, van Stralen M, Arbabi V, Viergever MA, Weinans H, Seevinck PR. Efficient cascaded V-net optimization for lower extremity CT segmentation validated using bone morphology assessment. J Orthop Res. December 2022;40(12):2894-2907.
2023	Yang J-M, Wang C-K, Lin W-C, Li C-Y, Su F-C, Jou I-M, Wu P-T. Comparison of ultrasound and dorsal tangential view for dorsal cortex screw penetration in volar plating of the distal radius. J Orthop Res. January 2023;41(1):235-240.
2023	Dudli S, Heggli I, Laux CJ, Spirig JM, Wanivenhaus F, Betz M, Germann C, Farshad-Amacker NA, Herger N, Mengis T, Brunner F, Farshad M, Distler O. Role of C-reactive protein in the bone marrow of Modic type 1 changes. J Orthop Res. May 2023;41(5):1115-1122.
2023	Yagi M, Taniguchi M, Tateuchi H, Yamagata M, Hirono T, Asayama A, Umehara J, Nojiri S, Kobayashi M, Ichihashi N. Properties of the iliotibial band and their relationships with gait parameters among patients with knee osteoarthritis. J Orthop Res. June 2023;41(6):1177-1185.
2023	Nahal C, Slullitel PA, Kamenaga T, Payne ER, Nepple JJ, Clohisy JC, Pascual-Garrido C. Acetabular coverage analysis of the proximal femoral head accurately characterizes dysplastic acetabular morphology. J Orthop Res. June 2023;41(6):1273-1282.
2023	Winkler PW, Chan CK, Lucidi GA, Polamalu SK, Wagala NN, Hughes JD, Debski RE, Musahl V. Increasing the posterior tibial slope lowers in situ forces in the native ACL primarily at deep flexion angles. J Orthop Res. July 2023;41(7):1430-1438.
2023	Chatterjee M, Evans MK, Bell R, Nguyen PK, Kamalitdinov TB, Korntner S, Kuo CK, Dyment NA, Andarawis-Puri N. Histological and immunohistochemical guide to tendon tissue. J Orthop Res. October 2023;41(10):2114-2132.
2023	Ye K, Xing Y, Zou D, Zhou F, Zhang Z, Du G, Tian Y. Positive correlation between the proximal femur Hounsfield units from routine CT and DXA results. J Orthop Res. December 2023;41(12):2648-2656.
2021	Vance B, Goggins KA, Godwin A, Oddson BE, Eger TR. Evaluation of the inter-rater and intra-rater reliability of a protocol for the measurement of foot-transmitted vibration. Theor Issues Egron Sci. 2021;22(2):218-235.
2021	Sorby-Adams AJ. Targeting Cerebral Oedema and Elevated Intracranial Pressure in a Preclinical Model of Ischaemic Stroke [PhD thesis]. University of Adelaide; May 2021.
2018	Vance B. Establishing a Protocol for the Measurement of Human Exposure to Foot-Transmitted Vibration [Master's thesis]. Sudbury, ON: Laurentian University; 2018.
2021	Jacobson N. Design and Validation of a Non-Invasive Intra-Abdominal Pressure Measurement Device [PhD thesis]. McGill University; August 2021.
2021	Eveleigh KJ. Balance Evaluation Using a Markerless Motion Capture System [Master's thesis]. Queen's University; August 2021.
2023	Riek P. Validation and Use of Inertial Sensors to Assess Changes in Gait Stability Introduced by a Robot Companion [Master's thesis]. Queen's University; March 2023.
2021	O'Day JJ. Open-Source Tools for Measuring, Understanding, and Improving Walking [PhD thesis]. Stanford University; August 2021.
2020	Pai S A. Investigation of Thoracic Spine and Muscle Morphology With Open Upright MRI [Master's thesis]. Vancouver, BC: University of British Columbia; December 2020.
2018	Lorenzen KN. A Comparison of Biomechanical Outcomes in Single Leg Squat and Vertical Drop Jump in Youth and Young Adults With and Without a Previous Youth Sport-Related Knee Joint Injury [Master's thesis]. Calgary, AB: University of Calgary; April 2018.
2018	Mohr M. Lower Extremity Muscle Activation Following a Previous Knee Injury: Implications for Post-Traumatic Knee Osteoarthritis [PhD thesis]. Calgary, AB: University of Calgary; December 2018.
2020	Spanswick P. Using HR-PQCT and Finite Element Analysis to Inform Clinical Assessment of Distal Radius Fractures [Master's thesis]. Calgary, AB: University of Calgary; August 2020.
2021	Firminger CR. Experimental Measurement and Applied Modelling of Patellar Tendon Strain [PhD thesis]. Calgary, AB: University of Calgary; May 2021.
2021	Yang L. Developing Image Analysis Methods to Evaluate Cartilage Degeneration in Animal Models of Osteoarthritis [PhD thesis]. University of Iowa; May 2021.
2020	Van Wyngaarden JJ. Psychosocial Predictors of Chronic Pain and Pain-Related Disability 12 Months After Lower Extremity Fracture [PhD thesis]. University of Kentucky; 2020.
2022	St. Amant J. Interplay Between Integrin Alpha 1 Beta 1 and Transforming Growth Factor Beta Receptor II in the Development of Spontaneous Osteoarthritis in the Mouse Knee [Master's thesis]. University of Guelph; September 2022.
2022	Young C. 3D Printed Wearable Devices for Out-of-Lab Joint Performance Assessment: Design, Validation and Applications [PhD thesis]. University of Guelph; August 2022.
2019	Hunter AH. Predicting Soccer Penalty Success: An Optimality Model [PhD thesis]. Brisbane, Australia: University of Queensland; 2019.
2020	Burns DM. Computer Vision and Machine Learning in Orthopaedic Shoulder Surgery [PhD thesis]. University of Toronto; November 2020.
2017	Levine IC. The Effects of Body Composition and Body Configuration on Impact Dynamics During Lateral Falls: Insights from in-Vivo, in-Vitro, and in-Silico Approaches [PhD thesis]. University of Waterloo; 2017.
2021	Roumengous T. Evaluating Neuromuscular Function of the Biceps Brachii After Spinal Cord Injury: Assessment of Voluntary Activation and Motor Evoked Potential Input-Output Curves Using Transcranial Magnetic Stimulation [PhD thesis]. Richmond, VA: Virginia Commonwealth University; October 2021.
2020	Dickerson LC. Exploring the Effect of Ankle Braces on Foot Posture [Master's thesis]. Virginia Polytechnic Institute and State University; April 14, 2020.
2020	Peebles AT. Development and Validation of Clinically Feasible Methods to Assess Landing Mechanics in Patients Following Anterior Cruciate Ligament Reconstruction [PhD thesis]. Virginia Polytechnic Institute and State University; May 1, 2020.
2018	Banayan SM. The Kinematic and Biomechanical Effects of Bracing on the Rehabilitation of the LCL Injured Elbow [Master's thesis]. University of Western Ontario; 2018.
2022	du Toit C. Three-Dimensional Ultrasound Imaging for Characterization of Synovitis in First Carpometacarpal Osteoarthritis [Master's thesis]. University of Western Ontario; 2022.
2018	Vignos MF. Longitudinal Investigation of in Vivo Tibiofemoral Kinematics, Cartilage Contact, and Cartilage Composition Prior to and Following Anterior Cruciate Ligament Reconstruction [PhD thesis]. University of Wisconsin – Madison; 2018.
2021	Moon GR. The Comparison of Treadmill and Overground Running With the Use of Inertial Measurement Units (IMUs) [Master's thesis]. West Virginia University; 2021.