Comparing SVM, Gaussian Process and Random Forest Surrogate Models for the CMA-ES

0447920 - ÚI 2016 RIV DE eng C - Conference Paper (international conference)
Pitra, Z. - Bajer, Lukáš - Holeňa, Martin
Comparing SVM, Gaussian Process and Random Forest Surrogate Models for the CMA-ES.
Proceedings ITAT 2015: Information Technologies - Applications and Theory. Aachen & Charleston: Technical University & CreateSpace Independent Publishing Platform, 2015 - (Yaghob, J.), s. 186-193. CEUR Workshop Proceedings, V-1422. ISBN 978-1-5151-2065-0. ISSN 1613-0073.
[ITAT 2015. Conference on Theory and Practice of Information Technologies /15./. Slovenský Raj (SK), 17.09.2015-21.09.2015]
R&D Projects: GA ČR GA13-17187S
Grant - others:GA MŠk(CZ) ED2.1.00/03.0078; ČVUT(CZ) SGS14/205/OHK4/3T/14; GA MŠk(CZ) LM2010005
Institutional support: RVO:67985807
Keywords : black-box optimization * surrogate modelling * CMA-ES * Gaussian process * random forest
Subject RIV: IN - Informatics, Computer Science

In practical optimization tasks, it is more and more frequent that the objective function is black-box which means that it cannot be described mathematically. Such functions can be evaluated only empirically, usually through some costly or time-consuming measurement, numerical simulation or experimental testing. Therefore, an important direction of research is the approximation of these objective functions with a suitable regression model, also called surrogate model of the objective functions. This paper evaluates two different approaches to the continuous black-box optimization which both integrates surrogate models with the state-of-the-art optimizer CMAES. The first Ranking SVM surrogate model estimates the ordering of the sampled points as the CMA-ES utilizes only the ranking of the fitness values. However, we show that continuous Gaussian processes model provides in the early states of the optimization comparable results.
Permanent Link: http://hdl.handle.net/11104/0249674