The detection of moving object is important in many tasks, such as video surveillance and moving object tracking. In this paper, a review has been made on a video surveillance scenario with real-time moving object detection and tracking. The design of a video surveillance system is directed on automatic identification of events of interest, especially on tracking and classification of moving objects. The object tracking and detection is used to establish a correspondence between objects or object parts in consecutive frames and to extract temporal information about objects such as trajectory, posture, speed and direction.Tracking is detecting the objects frame by frame in video. It can be used in many regions such as video surveillance, traffic monitoring and people tracking. In static environment segmentation of object is not complex. In dynamic environment due to dynamic environmental conditions such as illumination changes, shadows and waving tree branches in the wind object segmentation is a difficult and significant problem that needs to be handled well for a robust visual surveillance system.