Large scale estimation of distribution algorithms with adaptive heavy tailed random projection ensembles

Research output: Contribution to journalArticlepeer-review

Standard

Large scale estimation of distribution algorithms with adaptive heavy tailed random projection ensembles. / Sanyang, Momodou; Kaban, Ata.

In: Journal of Computer Science and Technology, Vol. 34, No. 6, 16.11.2019, p. 1241-1257.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Bibtex

@article{74d8e851a37949ad83c43ddcd0197683,
title = "Large scale estimation of distribution algorithms with adaptive heavy tailed random projection ensembles",
abstract = "We present new variants of Estimation of Distribution Algorithms (EDA) for large scale continuous optimisation, that extend and enhance a recently proposed random projection (RP) ensemble based approach. The main novelty hereis to depart from the theory of RPs that required (sub-)Gaussian random matrices for norm-preservation, and instead for the purposes of high dimensional search we propose to employ random matrices with independent and identically distributed entries drawn from a t-distribution. We analytically show that the implicitly resulting high dimensional covariance of the search distribution is enlarged as a result. Moreover, the extent of this enlargement is controlled by a single parameter, the degree of freedom. For this reason, in the context of optimisation, such heavy tailed random matrices turn out to be preferable over the previously employed (sub-)Gaussians. Based on this observation, we then propose novel covariance adaptation schemes that are able to adapt the degree of freedom parameter during the search, and give rise to a exible approach to balance exploration versus exploitation. We perform a thorough experimental study on high dimensional benchmark functions, and provide statistical analyses that demonstrate state-of-the-art performance of our approach when compared with previously existing alternatives in problems with 1000 search variables.",
keywords = "covariance adaptation, estimation of distribution algorithm, random projection ensemble, t-distribution",
author = "Momodou Sanyang and Ata Kaban",
year = "2019",
month = nov,
day = "16",
language = "English",
volume = "34",
pages = "1241--1257",
journal = "Journal of Computer Science and Technology",
issn = "1000-9000",
publisher = "Springer",
number = "6",

}

RIS

TY - JOUR

T1 - Large scale estimation of distribution algorithms with adaptive heavy tailed random projection ensembles

AU - Sanyang, Momodou

AU - Kaban, Ata

PY - 2019/11/16

Y1 - 2019/11/16

N2 - We present new variants of Estimation of Distribution Algorithms (EDA) for large scale continuous optimisation, that extend and enhance a recently proposed random projection (RP) ensemble based approach. The main novelty hereis to depart from the theory of RPs that required (sub-)Gaussian random matrices for norm-preservation, and instead for the purposes of high dimensional search we propose to employ random matrices with independent and identically distributed entries drawn from a t-distribution. We analytically show that the implicitly resulting high dimensional covariance of the search distribution is enlarged as a result. Moreover, the extent of this enlargement is controlled by a single parameter, the degree of freedom. For this reason, in the context of optimisation, such heavy tailed random matrices turn out to be preferable over the previously employed (sub-)Gaussians. Based on this observation, we then propose novel covariance adaptation schemes that are able to adapt the degree of freedom parameter during the search, and give rise to a exible approach to balance exploration versus exploitation. We perform a thorough experimental study on high dimensional benchmark functions, and provide statistical analyses that demonstrate state-of-the-art performance of our approach when compared with previously existing alternatives in problems with 1000 search variables.

AB - We present new variants of Estimation of Distribution Algorithms (EDA) for large scale continuous optimisation, that extend and enhance a recently proposed random projection (RP) ensemble based approach. The main novelty hereis to depart from the theory of RPs that required (sub-)Gaussian random matrices for norm-preservation, and instead for the purposes of high dimensional search we propose to employ random matrices with independent and identically distributed entries drawn from a t-distribution. We analytically show that the implicitly resulting high dimensional covariance of the search distribution is enlarged as a result. Moreover, the extent of this enlargement is controlled by a single parameter, the degree of freedom. For this reason, in the context of optimisation, such heavy tailed random matrices turn out to be preferable over the previously employed (sub-)Gaussians. Based on this observation, we then propose novel covariance adaptation schemes that are able to adapt the degree of freedom parameter during the search, and give rise to a exible approach to balance exploration versus exploitation. We perform a thorough experimental study on high dimensional benchmark functions, and provide statistical analyses that demonstrate state-of-the-art performance of our approach when compared with previously existing alternatives in problems with 1000 search variables.

KW - covariance adaptation

KW - estimation of distribution algorithm

KW - random projection ensemble

KW - t-distribution

M3 - Article

VL - 34

SP - 1241

EP - 1257

JO - Journal of Computer Science and Technology

JF - Journal of Computer Science and Technology

SN - 1000-9000

IS - 6

ER -