The role of the infrapatellar fat pad (IPFP) in knee osteoarthritis is not understood. This study aimed to identify relationships between MRI-based signal abnormalities in the IPFP and measures of structural pathology and symptom severity in PFJOA, as well as investigate the influence of obesity and sex on these relationships. Seventy participants (ages 28–80) with isolated PFJOA underwent bilateral knee MRI scan acquisitions and completed the Knee Injury and Osteoarthritis Outcome Score (KOOS). MR images were scored for abnormal IPFP area and signal intensity, joint effusion, synovial proliferation, and patellar and trochlear cartilage damage. Repeated measures correlations were performed to assess associations between abnormal area and signal of IPFP and PFJOA pathology and KOOS, respectively. Associations were interrogated across weight-based groups based on BMI and sex-based groups. Between abnormal IPFP and PFJOA pathology, we observed no significant associations. Between abnormal IPFP and patient-reported outcomes, we observed weak to moderate significant negative associations between the size of the abnormal IPFP area and all KOOS subscales. In a sex-based analysis of IPFP and KOOS associations, we observed significant moderate negative correlations between IPFP and KOOS scores across all subcategories in female participants. In male participants, abnormal IPFP was not associated with KOOS scores. The IPFP is significantly related to PFJOA patient-reported pain and function, and this correlation is stronger in high-risk OA groups.