Fascia sliding mobility and deformation magnitude are potential biomarkers for musculoskeletal disorders, particularly in the thoracolumbar fascia over the erector spinae muscles, which are associated with low back pain. The use of speckle tracking analysis of ultrasound images through open-source software has been proposed for assessing fascia sliding mobility and deformation of the fascia. However, little is known about the validity and reliability of speckle tracking analysis. Since open-source projects for speckle tracking analysis have made great progress, an assessment of validity and reliability is required. Therefore, this study aimed to test the metric quality of speckle tracking analysis using an open-source software program. A custom-made tissue sliding device was developed to slide two gel pad phantoms over each other at a constant speed. The shear displacement was documented in real-time as the ground truth, while ultrasound videos were recorded. The ground truth data were then compared with the speckle tracking analysis data extracted from the ultrasound videos. Speckle tracking analysis for assessing tissue displacement using free and open-source software achieved excellent test–retest reliability and showed very high validity and reliability with low measurement errors. The presented open-source ultrasound-based speckle tracking analysis method can be recommended for research and clinical use in various environments.