This thesis presents a real-time coarse-to-fine human gait tracking system based on a cascaded particle filter using a single RGB-D sensor. The tracking system is a combination of two different layers which explores how the information between the two sensing modalities can be chained to distribute and share the implicit knowledge associated with the tracking environment. In the first layer, the RGB information is exploited for tracking the coarse body shape, when the prior estimate of the state of the object is distributed based on the hierarchical sampling. For the second layer, the segmented output is used for tracking marked feature points of interest in the depth image. Two approaches, spin image, and geodesic distance, for associating a measure of the estimates are used in this phase. The thesis exhibits the overall implementation of the proposed method combined with a series of experimental analysis.