Artificial neural networks as feature extractors in continuous evolutionary optimization

Stephen Friess, Peter Tiňo, Zhao Xu, Stefan Menzel, Bernhard Sendhoff, Xin Yao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent years have seen the advancement of data-driven paradigms in population-based and evolutionary optimization. This reflects on one hand the mere abundance of available data, but on the other hand also progresses in the refinement of previously available machine learning methods. Surprisingly, deep pattern recognition methods emerging from the studies of neural networks have only been sparingly applied. This comes unexpected, as the complex data generated by evolutionary search algorithms can be considered tedious and intractable for manual analysis with mere practical intuitions. In this work, we therefore explore opportunities to employ deep networks to directly learn problem characteristics of continuous optimization problems. Particularly, with data obtained during initial runs of an optimization algorithm. We find that a graph neural network, trained upon a graph representation of continuous search spaces, shows in comparison to more traditional approaches higher validation accuracy and retrieves characteristics within the latent space which are better at distinguishing different continuous optimization problems. We hope that our study can pave the way towards new approaches which allow us to learn problem-dependent algorithm components and recall these from predictions of inputs generated during the run-time of an optimization algorithm.
Original languageEnglish
Title of host publication2021 International Joint Conference on Neural Networks (IJCNN)
PublisherIEEE
Pages1-9
Number of pages9
ISBN (Electronic)9781665439008, 9780738133669
ISBN (Print)9781665445979 (PoD)
DOIs
Publication statusPublished - 21 Sep 2021
Event2021 International Joint Conference on Neural Networks (IJCNN) - Shenzhen, China
Duration: 18 Jul 202122 Jul 2021

Publication series

NameInternational Joint Conference on Neural Networks (IJCNN)
PublisherIEEE
ISSN (Print)2161-4393
ISSN (Electronic)2161-4407

Conference

Conference2021 International Joint Conference on Neural Networks (IJCNN)
Period18/07/2122/07/21

Bibliographical note

Funding Information:
This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 766186 (ECOLE). It was also supported by the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (Grant No. 2017ZT07X386), Shenzhen Science and Technology Program (Grant No. KQTD2016112514355531), and the Program for University Key Laboratory of Guangdong Province (Grant No. 2017KSYS008).

Publisher Copyright:
© 2021 IEEE.

Keywords

  • Machine learning algorithms
  • Manuals
  • Machine learning
  • Artificial neural networks
  • Feature extraction
  • Prediction algorithms
  • Search problems
  • knowledge transfer
  • algorithm selection
  • representation learning
  • Feature learning
  • graph neural networks

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Artificial neural networks as feature extractors in continuous evolutionary optimization'. Together they form a unique fingerprint.

Cite this