Approximate computing is emerging as a viable way to achieve significant performance enhancement in terms of power, speed, and area for system on chip (SoC) designs. Utilizing approximate computing in the design of deep neural networks (DNNs) can significantly reduce the system’s power, delay, and area at a cost of a tolerable drop in accuracy. This thesis demonstrates how approximate computing methods such as approximate multiplication, low quantization, and shared neural networks can achieve these performance enhancements in DNN designs. In terms of approximate multipliers which are the primary focus of the thesis, a study on the impact of approximate multipliers on the inference accuracy of convolutional neural networks (CNNs) is presented. Additionally, an efficient hybrid training approach using both exact and approximate multipliers is proposed. Most importantly, the thesis introduces the new concept of boosting CNN multiplication performance using a precision prediction preprocessor that controls approximate multipliers with various precisions. Another important research contribution of this thesis is studying practical considerations for accuracy evaluation of sensor-based machine learning and deep learning designs. Certain aspects can negatively impact the system’s accuracy in production. These aspects are not usually considered when evaluating and comparing models’ accuracy during development and prototyping. Examples include accuracy loss due to the component’s variable thermal noise, component failure or partial failure, and analog-to-digital converter (ADC) quantization error. Finally, the thesis presents the new concept of utilizing machine learning for person identification through physical activity. This research finding demonstrates that machine learning can be applied not only for the identification of physical activities but also for the identification of the activity performer as well. Based on this finding, a novel multi-label shared deep neural network (DNN) to identify both the physical activity and the activity performer simultaneously is proposed.