Recently published compression tests on PMMA/bone specimens extracted after vertebral bone augmentation indicated that PMMA/bone composites were not reinforced by the trabecular bone at all. In this study, the reasons for this unexpected behavior should be investigated by using non-linear micro-FE models. Six human vertebral bodies were augmented with either standard or low-modulus PMMA cement and scanned with a HR-pQCT system before and after augmentation. Six cylindrical PMMA/bone specimens were extracted from the augmented region, scanned with a micro-CT system and tested in compression. Four different micro-FE models were generated from these images which showed different bone tissue material behavior (with/without damage), interface behavior (perfect bonding, frictionless contact) and PMMA shrinkage due to polymerization. The non-linear stress–strain curves were compared between the different micro-FE models as well as to the compression tests of the PMMA/bone specimens. Micro-FE models with contact between bone and cement were 20% more compliant compared to those with perfect bonding. PMMA shrinkage damaged the trabecular bone already before mechanical loading, which further reduced the initial stiffness by 24%. Progressing bone damage during compression dominated the non-linear part of the stress–strain curves. The micro-FE models including bone damage and PMMA shrinkage were in good agreement with the compression tests. The results were similar with both cements. In conclusion, the PMMA/bone interface properties as well as the initial bone damage due to PMMA polymerization shrinkage clearly affected the stress–strain behavior of the composite and explained why trabecular bone did not contribute to the stiffness and strength of augmented bone.