Fundamental aspects of decentralized systems are considered from a control perspective. The stochastic framework afforded by Markov systems is presented as a formal setting in which to study decentralized systems. A stochastic algebra is introduced which allows Markov systems to be considered in matrix format but also strikes an important connection to the classic linear system originally studied by Kalman [1960]. The process of decentralization is shown to impose constraints on observability and controllability of a system. However, it is argued that communicating decentralized controllers can implement any control law possible with a centralized controller. Communication is shown to serve a dual role, both enabling sensor data to be shared and actions to be coordinated. The viabilities of these two types of communication are tested on a real network of mobile robots where they are found to be successful at a variety of tasks. Action coordination is reframed as a decentralized decision making process whereupon stochastic cellular automata (SCA) are introduced as a model. Through studies of SCA it is found that coordination in a group of arbitrarily and sparsely connected agents is possible using simple rules. The resulting stochastic mechanism may be immediately used as a practical decentralized decision making tool (it is tested on a group of mobile robots) but, it furthermore provides insight into the general features of self-organizing systems.