Keying has become a common image processing operation in TV and film production to separate elements from a background. Chroma keying is a robust and important technique for image processing or video which is widely used in magazine covers, cinema films, video game industries and also television programs such as live talk show, weather forecast. Chroma-key method is proposed in this paper using technique Coarse and fine filter in Real-time. Based on K-means clustering algorithm, the improved method is proposed namely Coarse and Fine Filter. Hardware Architecture for proposed method is implemented on Virtex 5 FPGA board. Later optimization of coarse and fine filter is done for reduction of resources utilization. Experimental results shows that proposed design can give better quality of composite image, requires less buffer size within very less time.