A significant increase in the risk of hip fracture was observed in middle-aged men living with human immunodeficiency virus (MLWH), almost a decade earlier than those without infection. Data regarding cortical and trabecular bone deficit of hip, an important determinant of bone strength, in MLWH are limited. Quantitative CT was performed in consecutive MLWH aged ≥30 years between November 2017 and October 2018 at Severance Hospital, Seoul, Korea. Volumetric bone mineral density (vBMD) and cortical bone mapping parameters of hip (cortical thickness [CTh], cortical bone vBMD [CBMD], cortical mass surface density [CMSD], endocortical trabecular density [ECTD]) were compared to age-matched and body mass index (BMI)-matched controls (1:​2) using a community-based healthy adults cohort. Among 83 MLWH and 166 controls (mean age:​ 47.2 years;​ BMI:​ 23.6 kg/m2), MLWH had lower total hip vBMD (280 ± 41 versus 296 ± 41 mg/cm³), CMSD (155 versus 160 mg/cm2), and ECTD (158 versus 175 mg/cm³) than controls that remained robust after adjustment for covariates (adjusted β:​ total hip vBMD, −18.8;​ CMSD, −7.3;​ ECTD, −18.0;​ p < 0.05 for all). Cortical bone mapping revealed localized deficit of CTh, CBMD, and CMSD in the anterolateral trochanteric region and femoral neck in MLWH compared to controls, with a more extensive ECTD deficit. In MLWH, lower CD4 T-cell count (/100 cells/mm³ decrement) and protease inhibitor (PI)-based regimen (versus non-PI regimen) at the time of antiretroviral treatment initiation were associated with lower total hip vBMD (adjusted β −7.5 for lower CD4 count;​ −28.3 for PI-based regimen) and CMSD (adjusted β −2.6 for lower CD4 count;​ −12.7 for PI-based regimen;​ p < 0.05 for all) after adjustment for covariates including age, BMI, smoking, alcohol use, hepatitis C virus co-infection, tenofovir exposure, and CT scanner types. MLWH had lower hip bone density with cortical and trabecular bone deficit compared to community-dwelling controls.