Knee osteoarthritis (OA) is a debilitating disease that involves inflammation and degradation of the structures of the knee. It affects over 250 million people globally and has no cure. Research to improve our understanding of this condition and to develop better treatments requires objective measures of disease severity. Radiography and magnetic resonance imaging (MRI) enable visualization of the knee’s structures and play an important role in measuring joint health and OA status. However, evaluation of these images by physicians and researchers is prone to subjectivity and requires significant time and expertise, creating a bottleneck that hinders research progress. Given the importance of medical image assessments for OA research, automated tools that mitigate human bias and costs are needed. Deep learning algorithms are capable of training neural network models to automate many medical image analysis tasks. This dissertation describes the development, validation, and deployment of neural networks to automate the assessment of OA in X-rays and MRIs.

One of the most commonly used evaluations of OA severity is the Kellgren-Lawrence (KL) score, a semi-quantitative 0-4 score of OA progression based on X-ray findings. We developed a model to automate the staging of OA severity from X-rays using the KL scoring system and compared its performance to that of musculoskeletal radiologists. The model was evaluated on 4,090 images staged by a radiologist committee. Saliency maps were generated to reveal features used by the model to determine KL grades. The model agreed with the consensus of a musculoskeletal radiologist committee as closely as individual musculoskeletal radiologists agreed with the committee. It takes full radiographs as input and predicts KL scores with state-of-the-art accuracy and does not require manual preprocessing. Saliency maps suggested the model’s predictions were based on clinically relevant information.

Compositional MRI sequences are designed to measure properties of the cartilage microstructure that undergo change with OA. Spin-spin T2 mapping is one of the most widely used compositional MRI techniques for this purpose. We developed a fully-automated femoral cartilage segmentation model for measuring T2 relaxation values and longitudinal changes using multi-echo spin echo (MESE) MRI. Subregional T2 values and four-year changes were calculated using a musculoskeletal images. A subset of 14 images were also evaluated by a second expert for comparison. Assessments of cartilage health using the model’s segmentations agreed with those of an expert as closely as experts agreed with one another.

We have made both our KL model and T2 mapping model publicly available. The KL model is available at https://github.com/stanfordnmbl/kneenet-docker and the T2 mapping model is available at https://github.com/kathoma/AutomaticKneeMRISegmentation. We have also deployed them as user-friendly web applications at http://kl.stanford.edu/ to enable their use by the OA research community. This has the potential to accelerate osteoarthritis research.