Bone fragility and obesity are both diseases that are multifactorial in etiology and pathology. The contributing role of high fat diet (HFD) versus energy overconsumption on bone health is controversial. Exercise is often prescribed for improving bone health, but it is unclear whether HFD or overconsumption influences skeletal adaptations to exercise. Female and male Wistar rats were fed HFD or low fat diet (LFD) for 10 weeks, starting at 8 weeks of age. Within HFD, rats were labeled Obesity-Resistant (OR) or Obesity-Prone (OP) based on weight and fat gain. Within each diet and phenotype group, rats were randomized to treadmill exercise or sedentary control (SED) for the final 4 weeks. Femurs were assessed for fracture toughness. Cortical lamellar and nonlamellar bone microscale material behavior and chemistry were assessed using nanoindentation and Raman spectroscopy. Female bones had higher fracture toughness and mineral:​ matrix ratio than male bones. Diet and energy overconsumption affected bone characteristics in a sex-dependent manner, where the divergence between OP and OR in response to HFD occurred more rapidly in males. Diet composition, in general, had a stronger effect on bone quality than overconsumption. HFD dramatically decreased bone size and lamellar mineral:​matrix compared to LFD. Effects of short-term exercise training on microscale tissue properties were generally more robust with LFD. Exercise enhanced the contrast between lamellar and nonlamellar bone for nanoindentation modulus but decreased this contrast for plastic work. Our data demonstrate the complexities in the relationship between diet and obesity and highlight the importance of addressing both aspects when characterizing bone quality and fracture resistance.