The accumulation of advanced glycation end-products (AGEs) in the organic matrix of bone with aging and chronic disease such as diabetes is thought to increase fracture risk independently of bone mass. However, to date, there has not been a clinical trial to determine whether inhibiting the accumulation of AGEs is effective in preventing low-energy, fragility fractures. Moreover, unlike with cardiovascular or kidney disease, there are also no pre-clinical studies demonstrating that AGE inhibitors or breakers can prevent the age- or diabetes-related decrease in the ability of bone to resist fracture. In this review, we critically examine the case for a long-standing hypothesis that AGE accumulation in bone tissue degrades the toughening mechanisms by which bone resists fracture. Prior research into the role of AGEs in bone has primarily measured pentosidine, an AGE crosslink, or bulk fluorescence of hydrolysates of bone. While significant correlations exist between these measurements and mechanical properties of bone, multiple AGEs are both non-fluorescent and non-crosslinking. Since clinical studies are equivocal on whether circulating pentosidine is an indicator of elevated fracture risk, there needs to be a more complete understanding of the different types of AGEs including non-crosslinking adducts and multiple non-enzymatic crosslinks in bone extracellular matrix and their specific contributions to hindering fracture resistance (biophysical and biological). By doing so, effective strategies to target AGE accumulation in bone with minimal side effects could be investigated in pre-clinical and clinical studies that aim to prevent fragility fractures in conditions that bone mass is not the underlying culprit.