The urban search and rescue (USAR) task is a very time sensitive operation which occurs in environments with high uncertainty. The task of a robot in USAR is to explore unknown and cluttered environments and identify any potential victims within the environments. In this thesis, an autonomous controller is presented which utilizes a hierarchical partially observable Markov Decision Process approach to model the environment uncertainty inside a robot’s decision-making scheme. Additionally, an intelligent exploration technique for USAR which utilizes deep reinforcement learning is proposed. The aim of these two techniques is to allow a robot to perform the full USAR task autonomously without the assistance of a human operator and to also follow an efficient exploration strategy which assists in the faster discovery of potential victims. Several experiments were conducted in USAR-like environments to validate the performance and robustness of each technique.