Hyper-Parameter Optimization Using MARS Surrogate for Machine-Learning Algorithms

Yangyang Li, Guangyuan Liu*, Gao Lu, Licheng Jiao, Naresh Marturi, Ronghua Shang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)


Automatically searching for optimal hyper parameters is of crucial importance for applying machine learning algorithms in practice. However, there are concerns regarding the tradeoff between efficiency and effectiveness of current approaches when faced with the expensive function evaluations. In this paper, a novel efficient hyper-parameter optimization algorithm is proposed (called MARSAOP), in which multivariate spline functions are used as surrogate and dynamic coordinate search approach is employed to generate the candidate points. Empirical studies on benchmark problems and machine-learning models (e.g., SVM, RF, and NN) demonstrate that the proposed algorithm is able to find relatively high-quality solutions for benchmark problems and excellent hyper-parameter configurations for machine-learning models using a limited computational budget (few function evaluations).

Original languageEnglish
Article number8735959
Pages (from-to)287-297
Number of pages11
JournalIEEE Transactions on Emerging Topics in Computational Intelligence
Issue number3
Publication statusPublished - Jun 2020

Bibliographical note

Publisher Copyright:
© 2017 IEEE.

Copyright 2020 Elsevier B.V., All rights reserved.


  • dynamic coordinate search
  • Hyper-parameter optimization
  • machine learning
  • MARS

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computational Mathematics
  • Control and Optimization


Dive into the research topics of 'Hyper-Parameter Optimization Using MARS Surrogate for Machine-Learning Algorithms'. Together they form a unique fingerprint.

Cite this