Machine learning brain activation topography for individual skill classification: Need for leave-one-subject-out (LOSO) cross-validation

This chapter presents an in-depth exploration of the leave-one-subject-out (LOSO) evaluation of a convolutional neural network (CNN)-based approach, aiming to discern and compare the brain states of experts and novices during laparoscopic surgery skill acquisition. The study contributes to understanding individual cognitive and perceptual processes involved in skill acquisition. Electroencephalography (EEG) data were collected using the wireless LiveAmp system, and preprocessing involved frequency filtering, noise removal, and independent component analysis (ICA). The spatiotemporal patterns were then transformed into a three-dimensional tensor, and a 3D CNN model, ESNet, was employed for analysis. Evaluation methods included fivefold cross-validation and LOSO cross-validation. Results from fivefold cross-validation demonstrated effective training and stable performance, whereas LOSO cross-validation indicated random fluctuations in loss and accuracy, emphasizing the model's unpredictability during cross-validation. Classification accuracy curves for novices and experts revealed limited learning, raising concerns about the model's ability to capture essential individual features. Further exploration and refinement are suggested to enhance the ESNet model's individualization capabilities to discern and compare the individual brain states of experts and novices during laparoscopic surgery skill acquisition.