Interest point detection is the vital aspect of computer vision and image processing. Interest point detection is very important for image retrieval and object categorization systems from which local descriptors are calculated for image matching schemes. Earlier approaches largely ignored color aspect as interest point calculation are largely based on luminance. However the use of color increases the distinctiveness of interest points. Subsequently an approach that uses saliency-based feature selection aided with a principle component analysis-based scale selection method was developed that happens to be a light-invariant interest points detection system. This paper introduces color interest points for sparse image representation. In the domain of video-indexing framework, interest points provides more useful information when compared to static images. Since human perception system is naturally influenced by motion of objects, we propose to extend the above approach for dynamic video streams using Space-Time Interest Points (STIP) method. The method includes the process of calculating the interest points in 3D domain (i.e., for feature extraction, this means that the main 2D concepts for images are extended to 3D). STIP renders moving objects in a live feed and characterizes the specific changes in the movement of these objects. A practical implementation of the proposed system validates our claim to support live video feeds and further it can be used in domains such as Motion Tracking, Entity Detection and Naming applications that have abundance importance.