Laparoscopic surgery is performed by entering a patient's abdominal area with two surgical tools and a laparoscope through small incisions. An assistant is required to hold and move the laparoscope to keep the operation site in view while the surgeon manipulates the two surgical tools. Timely. accurate, and stable adjustments of the laparoscope cannot be guaranteed in lengthy operations due to human fatigue. We propose a novel system with a robot that replaces the human assistant and automatically tracks the tip of a surgical tool. This robot can also be used as a part of a hturc tclesurgical sctup. Thc surgcon may want to havc automated access to various medical data and images pertaining to the patient. A novel surgcon-computer interface that features real-time laparoscopic image processing is proposed. The proposed system allows the surgeon to retrieve medical information by recognizing surgical tool gestures and motions.