Osteoarthritis (OA) is the most common joint disease, where articular cartilage degradation is often accompanied with sclerosis of the subchondral bone. However, the association between OA and tissue mineralization at the nanostructural level is currently not understood. In particular, it is technically challenging to study calcified cartilage, where relevant but poorly understood pathological processes such as tidemark multiplication and advancement occur. Here, we used state-of-the-art microfocus small-angle X-ray scattering with a 5-μm spatial resolution to determine the size and organization of the mineral crystals at the nanostructural level in human subchondral bone and calcified cartilage. Specimens with a wide spectrum of OA severities were acquired from both medial and lateral compartments of medial compartment knee OA patients (n = 15) and cadaver knees (n = 10). Opposing the common notion, we found that calcified cartilage has thicker and more mutually aligned mineral crystals than adjoining bone. In addition, we, for the first time, identified a well-defined layer of calcified cartilage associated with pathological tidemark multiplication, containing 0.32 nm thicker crystals compared to the rest of calcified cartilage. Finally, we found 0.2 nm thicker mineral crystals in both tissues of the lateral compartment in OA compared with healthy knees, indicating a loading-related disease process because the lateral compartment is typically less loaded in medial compartment knee OA. In summary, we report novel changes in mineral crystal thickness during OA. Our data suggest that unloading in the knee might be involved with the growth of mineral crystals, which is especially evident in the calcified cartilage.