The term bone refers to a family of materials, all of which are built up of mineralized collagen fibrils. They have highly complex structures, described in terms of up to 7 hierarchical levels of organization. These materials have evolved to fulfill a variety of mechanical functions, for which the structures are presumably fine-tuned. Matching structure to function is a challenge. Here we review the structure-mechanical relations at each of the hierarchical levels of organization, highlighting wherever possible both underlying strategies and gaps in our knowledge. The insights gained from the study of these fascinating materials are not only important biologically, but may well provide novel ideas that can be applied to the design of synthetic materials.