This study aimed to compare the bone mineral density (BMD) of older women living in rural and urban areas, and evaluate the potential factors affecting the risk of osteoporosis. We recruited 574 women aged 65 years or older from rural areas and 496 from urban areas in Shanghai, China. The BMD values of the lumbar vertebrae and total left hip were measured by a dual energy X-ray absorptiometry densitometer. We also recorded information about education level, family income, medications, reproductive and menstrual history, diet, smoking, and alcohol consumption. Women in urban areas had significantly higher BMD in their lumbar spine, and there was a dramatic increase in the proportion of women with osteoporosis in rural areas. The age at menarche was significantly higher among women living in rural areas, and there were more years from menarche to menopause among urban women. Rural women had significantly higher numbers of both pregnancies and parity, and a significantly lower age at first parity. In multiple linear regression analyses, years from menarche to menopause was independently related to high lumbar spine BMD, while age at menarche and parity was independently related to low lumbar spine BMD. More older women in rural areas had osteoporosis. Later menarche, less years from menarche to menopause and higher parity might partially contribute to decreased BMD among women in rural areas. More attention should be paid to women in rural areas to prevent bone loss and further bone and health impairment.