Flower identification and recognition are tedious and difficult tasks even for humans. Image segmentation based on automatic flower extraction is an essential step for computer-aided flower image recognition and retrieval processes. Furthermore, there is a challenge for segmentation of the object(s) s) from natural complex background in color images. In this study, a novel performance optimization approach for image segmentation, i.e. simulated annealing-based mean-shift segmentation (SAMS), is proposed and implemented. It is based on the simulated annealing solution of quadratic assignment problem model treated as an image segmentation process using feature-based mean-shift (MS) clustering on color images. The proposed approach is designed to realize a global and unsupervised (i.e., fully automatic) segmentation. It is a modified and optimized version of Backprojection-based mean-shift segmentation (BackMS) method. In conducted segmentation experiments, the performance results of SAMS approach are compared with the ones of BackMS method. Comparison of overall performance results and statistical analysis (i.e., Wilcoxon signed rank median test) show that SAMS approach improves the performance of BackMS method. It is measured as 49.33% when using object bounding boxes and as 51.33% when using object pixel regions. (C) 2013 Elsevier B.V. All rights reserved.