A lightweight transformer with time dimension enhancement and global attention for health state monitoring in track circuits

As a typical safety-critical system in vehicle-ground signal transmission, accurate identification and prediction of the health state of track circuits are crucial for enhancing efficiency and safety of High-Speed Railway (HSR) amid ongoing digital transformation. From the perspective of vehicle-ground collaboration, track circuit data exhibits multimodality. However, existing deep learning models face significant limitations in effectively capturing degradation rules of track circuits. To accurately model the mapping relationship between feature parameters and the health state of track circuits, we integrate Convolutional Neural Network (CNN) with multi-head self-attention mechanisms, innovatively proposing a lightweight Transformer network improved by Bayesian Optimization (BO). Specifically, a Time Dimension Enhancement Module (TDEM) and a Global Attention Module (GAM) are embedded into the Transformer model. The TDEM leverages a temporal convolutional network with an exponentially increasing dilation factor to efficiently expand receptive fields, enabling the extraction of latent temporal features and the capture of multi-scale temporal dependencies. Meanwhile, GAM refines cross-channel and depth-wise separable convolution by introducing multi-scale parallel separable convolution and broadcast operations to enhance the multi-head self-attention mechanism. Experimental results demonstrate that the time dimension enhancement and global attention network for track circuit health monitoring achieves a test accuracy of up to 98.43%.