Object Tracking is a thirsty region in the field of computer vision. Object tracking algorithms have greater priority because of the availability of highly facilitated computers, good quality and low cost cameras. Various researches are still carried out in this field, but it is nevertheless difficult to overcome a few drawbacks of object tracking. The challenges in object tracking involve occlusion, change of pattern appearance in both the scene and the object, poor images, changes in scene illumination, complex object movements, shape of the object etc. The current tracking algorithms also involve more mathematical complexity too. This research focuses on tracking of volley ball players playing in a test session. The videos are captured using high resolution camera. The video are captured when the players are under test session. The players are at first detected from the video frames using Cuckoo Search algorithm. The shadow that is present in the video frames is removed by segregating the pixel of the object and the pixel of shadow and normalized the RGB values and multiplied with matrix. Later, the value of the threshold is compared with the output and the shadow is segregated with reference to this threshold value. Then, the players are tracked by using three different metaheuristic algorithms such as Firefly, Cuckoo Search and Bat algorithms. The performance of the algorithms was compared against four measuring parameters such as Correct Detected Track, Latency in Track, Track Matching Error and Track Completeness. TMET and TCM are very important parameters among this. The result shows that TMET is less than 10.51 and TCM is maximum 0.85 for Bat algorithm. Thus, Bat algorithm found to be outperforming well in tracking the players from the video frames. [ABSTRACT FROM AUTHOR]