Bone and nacre are the most-known biological hard tissues to materials researchers. Although highly mineralized, both bone and nacre are amazingly tough and exhibit remarkable inelasticity, properties that are still beyond the reach of many modern ceramic materials. Very interestingly, the two hard tissues seem to have adopted totally different structural strategies for achieving mechanical robustness. Starting from a true nanocomposite of the mineralized collagen fibril and following up to seven levels of hierarchical organization, bone is built on a structure with extreme complexity. In contrast, nacre possesses a structure of surprising simplicity. Polygonal mineral tablets of micrometer size are carefully cemented together into a macroscopic wonder. A comparative analysis of the structure-property relations in bone and nacre helps us to unveil the underlying mechanisms of this puzzling phenomenon. In this review, we critically compare the various levels of structures and their mechanical contributions between bone and nacre, with a focus on inelasticity and the toughening process. We demonstrate that, although nacre and bone differ from each other in many aspects, they have adopted very similar deformation and toughening mechanisms.