Abstract In this paper, an optimal control scheme based on the application of genetic algorithm (GA) is applied to a tracker robot with machine vision capability. First, dynamic equations governing the robot system are extracted and then, kinematic relationships are acquired according to camera specifications. Then, the obtained open-loop model of the robot is simulated and its validity is evaluated through an experimental work. The computed torque control algorithm with conventional tuning technique is initially used to control the robotic system. However, it is found that tuning the mentioned robot controller via the conventional technique is not so effective due to the unseen hard non-linearity presented in the robot controller such as saturation of the actuators. Consequently, a more advanced GA-based optimal control scheme is proposed in this investigation. Thus, GA is used to obtain the optimum values of control constants based on a suitable cost function. Accordingly, a simulation study is conducted to highlight the superiority of the GA-based controller compared to the former algorithm. Besides, the linking approach of the vision subsystem to the robot controller is further described from both mathematical and experimental standpoints. Finally, the proposed visual tracker robot is developed and its performance to track a moving object on a circular as well as a butterfly-shape trajectory in the camera coordinates is investigated. The experimental results clearly reveal the effectiveness of the visual robot incorporating the optimal controller to follow the target trajectories and confirm the stability of the control system.