Cigarette smoking is a risk factor for osteoporosis and bone fracture. Moreover, smoking causes exposure to cadmium, which is a known risk factor for osteoporosis. It is hypothesized that part of smoking‐induced osteoporosis may be mediated via cadmium from tobacco smoke. We investigated this hypothesis using mediation analysis in a Swedish cohort of elderly men. This study was performed in 886 elderly men from the Swedish cohort of the Osteoporotic Fractures in Men (MrOS) study. Urinary samples, bone mineral density (BMD), smoking data, and other background information were obtained at baseline in 2002–2004. Urinary cadmium was analyzed in baseline samples and adjusted for creatinine. The cohort was followed until August 2018 for fracture incidence, based on the X‐ray register. Mediation analysis was conducted to evaluate the indirect effect (via cadmium) of smoking on both BMD and fractures. Time to first fracture was analyzed using the accelerated failure time (AFT) model and Aalen's additive hazard model. The mean level of urinary cadmium was 0.25 μg/g creatinine. There were significant inverse associations between smoking and total body, total hip, and trochanter BMD. The indirect effects via cadmium were estimated to be 43% of the total effects of smoking for whole‐body BMD, and even more for total hip and trochanter BMD. Smoking was also associated with higher risk of all fractures and major osteoporosis fractures. The indirect effects via cadmium were largest in nonvertebral osteoporosis fractures and hip fractures, constituting at least one‐half of the total effects, in both the AFT and Aalen's model. The findings in this study provide evidence that cadmium exposure from tobacco smoke plays an important role in smoking‐induced osteoporosis