Glycosaminoglycans (GAGs) are responsible for preserving bone tissue toughness as well as regulating collagen formation and mineralization in the extracellular matrix. However, current methods for characterization of GAGs in bone are destructive, thus unable to capture in situ changes or differences in GAGs between experimental groups. As an alternative, Raman spectroscopy is a non-destructive method and can detect concurrent changes in GAGs and other bone constituents. In this study, we hypothesized that the two most prominent Raman peaks of sulfated GAGs (at ~1066 cm⁻¹ and at ~1378 cm⁻¹) could be used to detect differences in GAGs content of bone. To test this hypothesis, three experimental models were utilized:​ an in vitro model (enzymatic removal of GAGs from human cadaver bone), an ex vivo mouse model (biglycan KO vs. WT), and an ex vivo aging model (comparing cadaveric bone samples from young and old donors). All Raman measurements were compared to Alcian blue measurements to confirm the validity of Raman spectroscopy in detecting GAGs changes in bone. Irrespective of different models, it was found that the ~1378 cm⁻¹ peak in Raman spectra of bone was uniquely sensitive to changes of GAGs content in bone when normalized with respect to the phosphate phase (~960 cm⁻¹);​ i.e., 1378 cm⁻¹/960 cm⁻¹ (peak intensity ratio) or 1370–1385 cm⁻¹/930–980 cm⁻¹ (integrated peak area ratio). In contrast, the 1070 cm⁻¹ peak, which includes another major peak of GAGs (1066 cm⁻¹), seemed to be compromised to detect changes of GAGs in bone due to concurrent changes of carbonate (CO3) in the similar peak range. This study validates the ability of Raman spectroscopy to detect in situ treatment-, genotype-, and age-related changes in GAG levels of bone matrix.