The temporal and spatial fluctuations in the dynamics of secondary osteonal remodeling impart heterogeneity to the compositional quality of bone. Bone mineral density (BMD) fails to reflect this heterogeneity as being a single score, and thus it cannot resolve the overlap between healthy individuals and those who experience fractures. Such information on tissue heterogeneity is lacking in the literature. In the current study, specimens were prepared from mid-diaphyseal portions of human femora (N = 16, age range 52–85 years old) and grouped based on the anatomical location (anterior, lateral, medial and posterior quadrants). Raman microscopy was used to obtain multiple measurements from each specimen which allowed the construction of histograms of mineralization, crystallinity and carbonation. The coefficient of variation (COV) and skewness were extracted from histograms as measures of heterogeneity. Results demonstrated that average mineralization of the medial quadrant and the data pooled over quadrants significantly increased with age. The mean carbonation increased within the observed age range for the pooled data. The variations of values about the mean became tighter for mineralization, crystallinity and type-B carbonation with age, indicating an overall reduction in compositional heterogeneity of aging femoral cortex. Skewness values indicated that the distributions of histograms were not Gaussian. We conclude that age-related changes in mean tissue composition are confounded with changes in the variation of tissue make-up about the mean. Future studies will establish as to whether compositional heterogeneity correlates with the mechanical strength of bone.