A new performance metric for user-preference based multi-objective evolutionary algorithms

In this paper, we propose a metric for evaluating the performance of user-preference based evolutionary multiobjective algorithms by defining a preferred region based on the location of a user-supplied reference point. This metric uses a composite front which is a type of reference set and is used as a replacement for the Pareto-optimal front. This composite front is constructed by extracting the non-dominated solutions from the merged solution sets of all algorithms that are to be compared. A preferred region is then defined on the composite front based on the location of a reference point. Once the preferred region is defined, existing evolutionary multi-objective performance metrics can be applied with respect to the preferred region. In this paper the performance of a cardinality-based metric, a distance-based metric, and a volume-based metric are compared against a baseline which relies on knowledge of the Pareto-optimal front. The experimental results show that the distance-based and the volume-based metrics are consistent with the baseline, showing meaningful comparisons. However, the cardinality-based approach shows some inconsistencies and is not suitable for comparing the algorithms.