The association of insulin resistance (IR), as indicated by the homeostasis model assessment of insulin resistance, with bone metabolism is yet to be clarified. We aimed to investigate the relationship of IR with bone mass by using the triglyceride-glucose (TyG) index as an alternative marker of IR. Data of 4810 non-diabetes individuals (2552 men aged ≥ 50 years and 2258 postmenopausal women) from the Korean National Health and Nutritional Examination Survey IV and V were analyzed. Bone mineral density (BMD) at the lumbar spine, femoral neck, total hip, and whole body were measured using dual-energy X-ray absorptiometry. After adjusting for confounding factors, there were inverse relationships of TyG index with femoral neck, total hip, and whole body BMD in men (β = − 0.085, P < 0.001 at femoral neck;​ β = − 0.046, P = 0.037 at total hip;​ β = − 0.098, P < 0.001 at whole body). However, in women, femoral neck and whole body BMD were negatively associated with the TyG index (β = − 0.071, P = 0.008 at femoral neck and β = − 0.065, P = 0.005 at whole body). The highest TyG index tertile exhibited reduced femoral neck BMD in both sexes (P = 0.003 in men and P = 0.013 in women) and reduced whole body BMD in men (P < 0.001) after adjusting for confounders. When the study subjects were divided into BMI (body mass index) < 23 kg/m² and ≥ 23 kg/m² groups, the TyG index was significantly associated with femoral neck BMD only in the women with BMI < 23 kg/m² (P = 0.009). We observed a significant association between the highest TyG index tertile and low bone mass at the femoral neck in women with BMI < 23 kg/m² (P = 0.003) that was not observed in women with BMI ≥ 23 kg/m² and men. In conclusion, IR evaluated using the TyG index was inversely associated with femoral neck BMD in non-diabetic men aged ≥ 50 years and postmenopausal women. The negative influence of IR on femoral neck BMD was robust in the women with BMI < 23 kg/m². This indicates a differential effect of IR on BMD according to skeletal site, sex, and BMI.