Maintenance of vigorous exercise habits from young to old age is considered protective against hip fractures, but data on fracture risk in lifelong vigorous exercisers are lacking. This longitudinal cohort study examined the hazard of hip fractures in 1844 male former athletes and 1216 population controls and in relation to exercise volume and intensity in later years. Incident hip fractures after age 50 years were identified from hospital discharge register from 1972 to 2015. Exercise and covariate information was obtained from questionnaires administered in 1985, 1995, 2001, and 2008. Analyses were conducted using extended proportional hazards regression model for time-dependent exposures and effects. During the mean ± SD follow-up of 21.6 ± 10.3 years, 62 (3.4%) athletes and 38 (3.1%) controls sustained a hip fracture. Adjusted hazard ratio (HR) indicated no statistically significant difference between athletes and controls (0.84;​ 95% confidence interval [CI], 0.55–1.29). In subgroup analyses, adjusted HRs for athletes with recent high (≥15 metabolic equivalent hours [MET-h]/week) and low (<15 MET-h/week) exercise volume were 0.83 (95% CI, 0.46–1.48) and 1.04 (95% CI, 0.57–1.87), respectively, compared with controls. The adjusted HR was not statistically significant between athletes with low-intensity exercise (<6 METs) and controls (1.08;​ 95% CI, 0.62–1.85). Athletes engaging in vigorous-intensity exercise (≥6 METs at least 75 minutes/week) had initially 77% lower hazard rate (adjusted HR 0.23;​ 95% CI, 0.06–0.86) than controls. However, the HR was time-dependent (adjusted HR 1.04;​ 95% CI, 1.01–1.07);​ by age 75 years the HRs for the athletes with vigorous-intensity exercise reached the level of the controls, but after 85 years the HRs for these athletes increased approximately 1.3-fold annually relative to the controls. In conclusion, these data suggest that continuation of vigorous-intensity exercise is associated with lower HR of hip fracture up to old age.