Brain MRI plays a very important role for radiologists to diagnose and treat various brain diseases. One of the most important applications of graph partitioning is image segmentation. Various graph based methods for image segmentation application was developed, but it produces unbalance parts and NP Complete. To address such limitations, we have investigated and developed swarm intelligence based approach in which a Tri Level Particle Swarm Optimization (TLPSO) can be applied for partitioning the graph, obtained from an image to be segmented. The proposed classification method includes three stages namely conversion, implementation, selection and extraction. To check the performance of this proposed algorithm, we carried out quantitative as well as qualitative evaluation. Segmentation by graph partitioning in which PSO technology is combined with three levels helps to reduce partitioning imbalance and considers local as well as global features for segmentation. This method generates better segmentation quality and time of convergence is reduced by considerable extent.