Object detection may be elementary step for machine-driven video analysis in several vision applications. Object detection in a video is usually performed by object detectors or background subtraction techniques. To automatize the analysis, object detection becomes an important task. However existing motion-based ways are typically restricted, while handling advanced scenarios like nonrigid motion and dynamic background. This paper have a tendency to show that the on top challenges may be self-addressed in a much unified framework named DEtecting Contiguous Outliers in the LOw-rank Representation (DECOLOR). This formulation integrates object detection and background learning into one method of improvement, which might be resolved by an alternating algorithmic program efficiently. It helps to detect the moving object from dynamic background by means of outliers in the low rank representation. This algorithm deals with the complicated foreground and dynamic background and also non rigid shapes very effectively. A comparative analysis also made with Principal Component Pursuit (PCP) to prove the Efficiency of DECOLOR. Experiments on each simulated knowledge and real sequences demonstrate that the output performs the progressive approaches and it will work effectively on a good vary of complex situations.