Changes in the distribution of bone mineralization occurring with aging, disease, or treatment have prompted concerns that alterations in mineralization heterogeneity may affect the fracture resistance of bone. Yet, so far, studies assessing bone from hip fracture cases and fracture-free women have not reached a consensus on how heterogeneity in tissue mineralization relates to skeletal fragility. Owing to the multifactorial nature of toughening mechanisms occurring in bone, we assessed the relative contribution of heterogeneity in mineralization to fracture resistance with respect to age, porosity, and area fraction of osteonal tissue. The latter parameters were extracted from quantitative backscattered electron imaging of human cortical bone sections following R-curve tests of single-edge notched beam specimens to determine fracture toughness properties. Microstructural heterogeneity was determined as the width of the mineral distribution (bulk) and as the sill of the variogram (local). In univariate analyses of measures from 62 human donors (21 to 101 years), local but not bulk heterogeneity as well as pore clustering negatively correlated with fracture toughness properties. With age as covariate, heterogeneity was a significant predictor of crack initiation, though local had a stronger negative contribution than bulk. When considering all potential covariates, age, cortical porosity and area fraction of osteons explained up to 50% of the variance in bone׳s crack initiation toughness. However, including heterogeneity in mineralization did not improve upon this prediction. The findings of the present work stress the necessity to account for porosity and microstructure when evaluating the potential of matrix-related features to affect skeletal fragility.