Due to the enormous increase in image database size as well as its vast deployment in various applications the need of Content Based Image retrieval (CBIR)systems becomes an crucial part of today’s cutting edge technology. In this paper we have proposed a CBIR system using color and texture feature. To reduce the semantic gap between the low level feature and high level semantics, relevance feedback mechanisms have been applied to the system. In most existing CBIR   systems the current RF techniques still lack satisfactory user interaction to improve the search accuracy and interaction as well. In this paper, we proposed a user interaction model using Support Vector Machine. Experimental results show that our system gives improved results as compared to other systems available in the literature.