Bisphosphonates (BPs) are the most widely used drugs for the treatment of osteoporosis but prolonged use of BPs might increase the risk of atypical femur fracture (AFF). There are only a few studies that address the bone material quality in patients on long-term BP treatment with or without AFFs. We analyzed 52 trans-iliac bone biopsies from patients on long-term BP therapy with (n = 26) and without (n = 26) AFF. At the microscopic level, the degree of mineralization of bone (DMB) was assessed on whole bone by X-ray digitized microradiography while microhardness by Vickers microindentation, and bone matrix characteristics by Fourier transform infrared microspectroscopy (FTIRM) (mineral/organic ratio, mineral maturity and crystallinity, and collagen maturity) were measured at random focal areas. The AFF patients were treated longer than non-AFF patients (9.7 ± 3.3 years versus 7.9 ± 2.7 years). As expected, bone remodeling was low in both groups, without difference between them. The AFF group had significantly higher DMB in cortical bone (+2.9%, p = .001), which remained so after adjusting for treatment duration (p = .007), and showed a trend in cancellous bone (+1.6%, p = .05). Consistent with higher DMB, heterogeneity index (HI) was lower in the AFF than in the non-AFF group, illustrating lower heterogeneity of mineralization in the AFF group. A significant positive correlation between the duration of treatment and DMB in cortical bone was found in AFF, and not in the non-AFF group. Microhardness and bone matrix characteristics were similar between groups. We conclude that the AFF group had a duration-dependent increase in DMB leading to a significantly higher DMB than the non-AFF. Because BPs have high affinity to bone mineral and lining the walls of the osteocyte lacunae, the accumulation of matrix-bound BPs in AFF could lead to inhibition of the osteocyte cytoskeleton blunting their response to mechanical strains, a hypothesis to be further investigated.