Bone is a hierarchically structured material with remarkable mechanical performance which may serve as a model for the development of biomimetic materials. Understanding its properties is essential for the assessment of diseases such as osteoporosis. This will lead to a critical evaluation of current therapies and aid in their more targeted development. While the full hierarchical structure of bone is extremely complex and variable, its basic building block, the mineralized collagen fibril, is rather universal. Due to the progress in experimental methods to characterize materials at the nanoscale, new insights have been gained into the structure/mechanical function relation in this nanocomposite. The amount of mineral is usually thought to determine the stiffness of the material, but recent results suggest that the properties of the organic matrix as well as the geometrical arrangement of the two components might have a much larger influence on the properties than traditionally assumed. Some recent results from experiment and numerical modeling leading to these ideas are reviewed.