Gender-affirming hormone therapy aligns physical characteristics with an individual's gender identity, but sex hormones regulate bone remodeling and influence bone morphology. We hypothesized that trans men receiving testosterone have compromised bone morphology because of suppression of ovarian estradiol production, whereas trans women receiving estradiol, with or without anti-androgen therapy, have preserved bone microarchitecture. We compared distal radial and tibial microarchitecture using high-resolution peripheral quantitative computed tomography images in a cross-sectional study of 41 trans men with 71 cis female controls, and 40 trans women with 51 cis male controls. Between-group differences were expressed as standardized deviations (SD) from the mean in age-matched cisgender controls with 98% confidence intervals adjusted for cross-sectional area (CSA) and multiple comparisons. Relative to cis women, trans men had 0.63 SD higher total volumetric bone mineral density (vBMD;​ both p = 0.01). Cortical vBMD and cortical porosity did not differ, but cortices were 1.11 SD thicker (p < 0.01). Trabeculae were 0.38 SD thicker (p = 0.05) but otherwise no different. Compared with cis men, trans women had 0.68 SD lower total vBMD (p = 0.01). Cortical vBMD was 0.70 SD lower (p < 0.01), cortical thickness was 0.51 SD lower (p = 0.04), and cortical porosity was 0.70 SD higher (p < 0.01). Trabecular bone volume (BV/TV) was 0.77 SD lower (p < 0.01), with 0.57 SD fewer (p < 0.01) and 0.30 SD thicker trabeculae (p = 0.02). There was 0.56 SD greater trabecular separation (p = 0.01). Findings at the distal radius were similar. Contrary to each hypothesis, bone microarchitecture was not compromised in trans men, perhaps because aromatization of administered testosterone prevented bone loss. Trans women had deteriorated bone microarchitecture either because of deficits in microstructure before treatment or because the estradiol dosage was insufficient to offset reduced aromatizable testosterone. Prospective studies are needed to confirm these findings.