Extreme presentations of common disease in children are often presumed to be of Mendelian etiology, but their polygenic basis has not been fully explored. We tested whether children with significant fracture history and no osteogenesis imperfecta (OI) are at increased polygenic risk for fracture. A childhood significant fracture history was defined as the presence of low‐trauma vertebral fractures or multiple long bone fractures. We generated a polygenic score of heel ultrasound‐derived speed of sound, termed “gSOS,” which predicts risk of osteoporotic fracture. We tested if individuals from three cohorts with significant childhood fracture history had lower gSOS. A Canadian cohort included 94 children with suspected Mendelian osteoporosis, of which 68 had negative OI gene panel. Two Finnish cohorts included 59 children with significant fracture history and 22 with suspected Mendelian osteoporosis, among which 18 had no OI. After excluding individuals with OI and ancestral outliers, we generated gSOS estimates and compared their mean to that of a UK Biobank subset, representing the general population. The average gSOS across all three cohorts (n = 131) was −0.47 SD lower than that in UK Biobank (n = 80,027, p = 1.1 × 10⁻⁵). The gSOS of 78 individuals with suspected Mendelian osteoporosis was even lower (−0.76 SD, p = 5.3 × 10⁻¹⁰). Among the 131 individuals with a significant fracture history, we observed 8 individuals with gSOS below minus 2 SD from the mean;​ their mean lumbar spine DXA‐derived bone mineral density Z‐score was −1.7 (SD 0.8). In summary, children with significant fracture history but no OI have an increased burden of common risk alleles. This suggests that a polygenic contribution to disease should be considered in children with extreme presentations of fracture.