Although increased bone fragility is a well-recognized consequence in patients with rheumatoid arthritis (RA), the essential cause of degenerate bone strength remains unknown. This study aimed to determine factors contributing to bone dysfunction in RA by focusing on the bone matrix micro-arrangement, based on the preferential orientation of collagen and the related apatite c-axis as a bone quality index. The classical understanding of RA is limited to its severe pathological conditions associated with inflammation-induced bone loss. This study examined periarticular proximal tibiae from RA patients as compared with osteoarthritis (OA) patients as controls. Bone tissue material strength was disrupted in the RA group compared with the control. Collagen/apatite micro-arrangement and vBMD were significantly lower in the RA group, and the rate of decrease in apatite c-axis orientation (−45%) was larger than that in vBMD (−22%). Multiple regression analysis showed that the degree of apatite c-axis orientation (β = 0.52, p = 1.9 × 10⁻²) significantly contributed to RA-induced bone material impairment as well as vBMD (β = 0.46, p = 3.8 × 10⁻²). To the best of our knowledge, this is the first report to demonstrate that RA reduces bone material strength by deteriorating the micro-arrangement of collagen/apatite bone matrix, leading to decreased fracture resistance. Our findings represent the significance of bone quality-based analysis for precise evaluation and subsequent therapy of the integrity and soundness of the bone in patients with RA.