Food processing industry requires precise and accurate quality assessment of products. This assessment effects the price of product and also the storage and transport decisions. This paper intends to address the automation of quality assessment in mangoes using computer vision. The paper addresses two issues, namely the number of surface injuries and the time-to-maturity of the fruit. A visible spectrum image of the fruits is considered to extract the Region of Interest(RoI), which contain the fruit with minimal background. Each RoI is further subjected to segmentation techniques so as to identify the injured patches as well as the portion of immature(green) area on the fruit surface. The percentage of these areas is used to determine the grade and the time-to-maturity respectively for each product.