Vision systems greatly enhance the capabilities of robots and allow them to be applied to complex tasks within dynamic environments. In this thesis, we explore the problem of controlling a robotic arm using image-based servoing in a monocular eye-in-hand configuration. Specifically, we develop a visual servoing system capable of tracking non-planar objects in the presence of uncertainties in both the robotic arm and the visual system.

To track selected features of a target object, we propose a feature extraction algorithm that behaves as an immune systems. We evaluate the performance of our artificial immune system for three object representations:​ template, histogram and contour, and we show that the AIS can track multiple features under affine transformations and nonlinear distortions.

We then develop an image-based visual servoing control that is robust to parametric uncertainties in the robot model and camera calibrations. We use the LaSalle's invariance principle to prove the stability of the system and that the tracking error approaches zero if the uncertainty is bounded. Simulations verify the robustness of the system.

To implement the visual servoing system on an experimental robot, we design an open-architecture controller to replace the industrial controller of a PUMA robot. We then compare the performance and robustness of the proposed control versus that of a proportional control and a quasi-newton adaptive control under a variety of test conditions. We conclude that the proposed control has the best performance of the three controls tested.