The object is defined by its location and extent in a single frame. In every frame that follows, the task is to determine the object’s location and extent or indicate that the object is not present.. Tracking is usually performed in the context of higher-level applications that require the location and/or shape of the object in every frame. The detection of moving object is important in many tasks, such as video surveillance and moving object tracking. We proposed a novel approach is to detected the moving objects should be presented to higher-level analysis tools in order to identify further events and behaviors of interest typical of observation systems for open spaces. Then the background subtraction will be done by the algorithms to track object. The methods using for background subtraction are: Video surveillance applications, which combine recent scheme for the improvement of the system recital and system convergence, and a novel heuristic for better initializing the constraint for new created forms, and it evades the emergence of overdominating form. Background subtraction function is constructed that gives the likelihood that a given pixel belongs to the allocation of background pixels. For the running average the previous known pixel values were fitted to the model of distribution. It allows the pictures by eliminating the background and analyzes the object exactly. In a further different frame activity has captured to analyze the varying objects in the video then the difference image is converted into gray image and then translated into binary image. Moreover, we discuss the important issues related to kalman filter is used to track the object and morphological operations are done to detect the object perfectly.