High-resolution architecture-based finite element models are commonly used for characterizing the mechanical behavior of cancellous bone. The vast majority of studies use homogeneous material properties to model trabecular tissue. The objectives of this study were to demonstrate that inhomogeneous finite element models that account for microcomputed tomography-measured tissue modulus variability more accurately predict the apparent stiffness of cancellous bone than homogeneous models, and to examine the sensitivity of an inhomogeneous model to the degree of tissue property variability. We tested five different material cases in finite element models of ten cancellous cubes in simulated uniaxial compression. Three of these cases were inhomogeneous and two were homogeneous. Four of these cases were unique to each specimen, and the remaining case had the same tissue modulus for all specimens. Results from all simulations were compared with measured elastic moduli from previous experiments. Tissue modulus variability for the most accurate of the three inhomogeneous models was then artificially increased to simulate the effects of non-linear CT-attenuation-modulus relationships. Uniqueness of individual models was more critical for model accuracy than level of inhomogeneity. Both homogeneous and inhomogeneous models that were unique to each specimen had at least 8% greater explanatory power for apparent modulus than models that applied the same material properties to all specimens. The explanatory power for apparent modulus of models with a tissue modulus coefficient of variation (COV) range of 21–31% was 13% greater than homogeneous models (COV=0). The results of this study indicate that inhomogenous finite element models that have tissue moduli unique to each specimen more accurately predict the elastic behavior of cancellous cubic specimens than models that have common tissue moduli between all specimens.