In this thesis, a constraint-aware visual servoing control law is proposed. The control law is designed for a robot manipulator with an uncalibrated camera mounted on its end-effector. This control law allows the robot to execute large, collision-free motions with closed-loop positional accuracy. A reference image visually describes the desired end-effector position with respect to a target object whose location is initially unknown. The control law uses this reference image with online feedback from the camera to direct the trajectory of robot towards the completion of the positioning task. The control law generates feasible and realistic robot trajectories that respect the robot’s joint position and velocity limits, even in the presence of large control gains. The control law also explicitly keeps the target object within the camera’s field of view to provide uninterrupted visual feedback. The control law avoids potential whole-arm collisions with workspace obstacles via planning and control strategies.

The visual servoing control law is implemented in a nonlinear model predictive control framework, using an estimated model of the eye-in-hand configuration and an estimated location of the target object. Two methods of approximating the object’s location for joint-space path planning are demonstrated in simulations and experiments. The first uses homography estimation and decomposition on an un-modelled object. The second uses an extended Kalman filter with a prior object model to improve robustness against image noise and disturbances.

Two planning and control strategies are presented. The first strategy uses an offline plan-then-servo approach that integrates probabilistic roadmaps with visual servoing. A method to construct paths between two robot configurations that keep the target object within the camera’s field of view is demonstrated, allowing feasible transitions from planned motion to visual servoing. A method to address pose uncertainty to ensure collision-free, closed-loop motion is statistically tested with multiple positioning tasks. The second strategy uses an online iterative plan-and-servo approach that dynamically updates its estimate of the collision-free space while visual servoing. Experiments using an uncalibrated eye-in-hand platii form demonstrate the ability of the visual servoing control law to achieve closed-loop positioning via collision-free trajectories, even when the object location is highly uncertain.