Preference learning for multi-objective optimisation problems

Choosing a desired option from a large number of diverse alternatives is a difficult task for anyone, regardless of context. For a decision-maker seeking a preferred solution to a multi-objective optimisation problem, selecting a solution from the numerous possibilities is extremely difficult and mentally demanding. Choosing a solution requires comparing many alternatives and assessing their attribute levels and the trade-offs between competing objectives. This thesis is therefore interested in supporting decision-makers in this process by reducing the cognitive burden and fatigue they experience when providing preferences for multi-objective optimisation. The research in this thesis utilises established multi-objective evolutionary optimisation algorithms (MOEAs) to find solutions, with the search directed by preferences acquired interactively. MOEAs use a population-based approach, allowing a diverse set of solutions to be found with a single algorithm run. Furthermore, the iterative process used by MOEAs to generate and improve their population of candidate solutions also facilitates an interactive means of preference acquisition. Interactive preference-based MOEAs aid the decision-maker in developing and defining their preferences while simultaneously directing the solution search toward a region of interest. The result is a more informed decision-maker (having been exposed to feasible solutions during optimisation), and a more manageable choice amongst a small set of relevant solutions. A significant advantage of the interactive approach is that it allows and even encourages the decision-maker to change their preferences after optimisation of the problem has begun. Multi-objective evolutionary algorithms incrementally improve their population of solutions, and preference-based versions will eventually be able to re-focus solution generation toward a new area of interest. However, the number of function evaluations required is variable, and the additional time taken to move the population contributes to a decision-maker’s fatigue. To speed up the process of finding solutions in a new area of interest, we propose a novel use of archives to maintain a diverse set of solutions that can be called upon to “seed” the current population in the new area of interest. Evaluations on several archive types using benchmark problems reveal statistically significant improvements in responding to preference changes in a reference point-based MOEA. With the decision-maker and their preferences included in the optimisation process, the decision-maker has greater ownership of resultant solutions, which are more likely to be implemented. Unfortunately, expressing preferences over alternatives with multiple attributes in large search spaces with an unknown Pareto Front is extremely difficult. Two strategies present themselves: asking the decision-maker a small number of complicated queries involving thresholds, objective trade-off levels and other numerical measures; or asking a large number of easy-to-answer queries such as preference over a pair of alternatives. While both approaches are demanding of the user, the latter is more prevalent in the literature where elicitation is interactive. We propose an active learning framework that aims to minimise the number of pairwise comparisons required to identify highly preferred solutions. By using Bayesian optimisation to model a decision-maker’s preferences, we identify queries that maximise information gain and reduce uncertainty. The preference model is integrated with a reference point-based MOEA and used to identify a region of high preference in the search space of a series of multi-objective optimisation problems. In using Bayesian optimisation with an appropriate acquisition function, the search space can be explored, and promising regions exploited in a principled manner. The result is a unique combination of techniques that can learn a decision-maker’s preferences with a minimal number of low-burden queries, and accurately identify preferred solutions. The final contribution in this thesis extends the use of Bayesian preference acquisition by using historical data to warm-start the model and speed convergence toward preferred solutions. The approach, called accelerated Bayesian preference learning (ABPL), substantially reduces the number of queries needed to find preferred solutions and minimises expensive algorithm evaluations. We identify promising solutions exhibiting similar preference characteristics from historical data and warm-start the Bayesian model. The approach uses the newly acquired information during the optimisation process to find additional solutions and present them to the user in conjunction with suggestions from the Bayesian optimisation model. Again, using an extensive set of synthetic and real-world multi-objective problems, the approach finds highly preferred solutions with substantially fewer user queries than cold-start techniques.